Information Sheet # 131

April 17, 2001

GRADING BY PRIMARY TRAIT ANALYSIS

[This information sheet on primary trait scoring is a revised and condensed version of an issue of Word Works, published by the Writing Center at Boise State University.  That document, in turn, was based on the following sources:  Richard Lloyd Jones, "Primary Trait Scoring," in Evaluating Writing, edited by Cooper and Odell; Barbara Walvoord, "Using the Grading Process for Departmental and General Education," a presentation for the 1998 Assessment Institute at Indiana University Purdue University; and Barbara Walvoord and Virginia Johnson Anderson, Effective Grading: A Tool for Learning and Assessment, published by Jossey Bass.  Attached are three examples of primary trait scoring guides.  The first is adapted from the Boise State publication; the other two samples from John C. Bean’s Engaging Ideas are designed for short assignments where the evaluation can focus on a small number of criteria.  These examples are not intended to offer specific criteria that you might use for a class but rather to model how such evaluation instruments can be constructed. Reprinted below is a grading strategy that begins with a simple primary-trait analysis but then turns into a holistic evaluation.  This approach is adapted from an article by Harry Shaw in an anthology entitled Teaching Prose (1984).]

Many faculty who use writing assignments dislike having to figure out grades for student papers: so many papers have such mixtures of strengths and weaknesses they are hard to fit into any grading slot without doing some injustice. Another reason many teachers dislike grading is that grades don't contribute much to learning. It's true that a grade is a form of feedback, and students who take the time to analyze their work when they get it back can learn a lot about why it received the grade it did. But for most students, grades alone are not an informative way to identify strengths and weaknesses in writing.

One effective grading system is Primary Trait Analysis (PTA), designed to show students their specific strengths and weaknesses. It is a point system, similar in some ways to other point based grading systems. The differences are that (1) the point scale is divided into several categories of equal weight, and (2) the points in the scale are linked to specific traits.

Example of a PTA Scale

Here’s the beginning of a PTA Scale for grading science reports that adhere to the following structure:  

        I. Introduction

                A.  Provides history and context.

                B.  Contains the hypothesis to be tested.

        II. Materials and methods

               A.  Outlines general approach for testing the hypothesis.

               B.  Enumerates equipment and commodities and outlines procedures whereby a knowledgeable student could replicate the work.

III. Etc. (separate categories for Findings, Discussion, Conclusion, References)

Printed below is the detailed set of level descriptions for the first two parts of the report’s introduction.  The numbers (1-4) are points the report can earn in each category. The points for all categories are added up and converted to a letter grade.

Introduction A (Provides history and context)

4 -    History well researched. Major contributions presented with discrimination, balance. Controversies outlined & weighed.

3 -    History adequately outlined. Role of major contributions recognized. Relative merit of conflicting opinions somewhat unclear.

2 -    Historical outline present. Contextual development and relative merit of contributions unrecognized or ragged.     Presentation of conflicting ideas absent.

1 -    Historical outline absent or garbled. Contributions listed as in a diary; consideration of merit absent. Notions of conflicting ideas ignored.

Introduction B (Contains the hypothesis to be tested)

4 -    Hypothesis clearly recognized or well crafted and elegantly stated in testable form.

3 -    Hypothesis recognized or well stated. Contextual connections evident.

2 -    Hypothesis detectable but may not be stated in testable form. Contextual connections tenuous.

1 -    Hypothesis undetectable or garbled so as to violate scientific principles. Context absent or ignored.

How to Create a PTA scale

If you glance at the sample scale for a technical communication memo on the third page of this information sheet, you may feel daunted. It looks like a lot of work, writing out all those descriptions. But the work is not so hard if it's done in small steps; it need not all be done at once. The sample scales were developed over a period of time, as the instructors gradually clarified their expectations. Here are hints for making the project doable, without taking up a huge amount of time.

•  Start with your knowledge of past student performance. Jot down the typical strengths and weaknesses you see in student work. Use them as a starter for developing the scale.

•  Identify your main categories. How many will there be? What traits can you lump together? Consider logical principles behind categories. You can categorize according to parts of the finished paper or the overall rhetorical features of assignment.

•  Begin writing the descriptions with the highest level of achievement in each category. Once you have that much done, you can use it to grade student work, adding the other levels later.  You probably want to write the lowest level next and then fill in the middle level(s). The middle levels are the hardest to write because you have to think about a large number of different variables.  If you use a four-point scale, you'll have to write two middle levels. But if you use a five-point scale, you can get by describing levels 5, 3, and 1 only. Levels 4 and 2 will be understood as combining traits from the levels above and below them. A six-point scale can combine levels in the following pattern:  5-6, 3-4, 1-2.

• Experiment with having your students collaborate with you on developing a scale.  The class can select the main categories (or give them the categories), then assign each category to a different group to describe each trait. Quite possibly they will come up with a more demanding scale than you would, and you'll have to persuade them to make it more reasonable. As one student remarked when constructing such a scale, it was like being asked “to build a cross for his own crucifixion.”

Scales are Works in Progress

Treat your PTA scale as an ongoing draft, always in need of more work. As you use it you will find various ways to fine-tune (or completely rethink) the scale:

• If there are too many high scores, maybe some criteria are missing.

•  If there are too many low scores, maybe you expect students to know something they have not been taught.

•  There's a famous story of a biology department in which students were having difficultly using microscopes, even in upper-level classes. The faculty eventually figured out that students were having problems because they had no course which taught students how to use microscopes.

•  If some categories are difficult to score, maybe they are not well defined, or the wrong traits have been lumped together. The categories may need to be redefined.

•  Ask your students for feedback on the scale. If they don't understand some of the level descriptions, then the descriptions probably should be reworded or rethought.

Advantages of PTA

A PTA scale offers benefits both for students and instructors. Some of the prominent advantages are:

•  It makes grading more consistent. It helps to insure that the same criteria are being applied to all students' work.

•  It serves as a corrective against subjective impressions.

•  It gives students more information about what they do well and where they are deficient. They see their work analyzed feature by feature, so they can see more exactly where they are strong and where they are weak.

•  It helps teachers articulate for themselves exactly what their expectations are for the quality of student work.

•  It prompts teachers to think harder about what they teach and how they teach it.

•  It enables departments to assess teaching and learning across multiple sections of a course.

• It can save substantial time in grading papers, particularly when you have large numbers of papers on the same topic.

The disadvantage of PTA scales is that they take time to write. But when you consider that they can be developed gradually in easy stages, and put to use when only partially developed, this is not necessarily a major disadvantage.[

* * * * *

A Strategy for Assigning Letter Grades

In grading “thesis papers” ask yourself the following questions:

(1) Does the paper have a thesis?

(2) Does the thesis address itself to an appropriate question or topic?

(3) Is the paper free from long stretches of quotations and summaries that exist only for their own sake and remain unanalyzed?

(4) Can the writer produce complete sentences?

(5) Is the paper free from basic grammatical errors?

If the answer to any of these question is “no,” give the paper some kind of C.  If the answer to most of the questions is “no,” its grade will be even lower.

For papers which have emerged unscathed thus far, add the following questions:

(6) How thoughtful is the paper?  Does it show originality?

(7) How adequate is the thesis?  Does it respond to its questions in a full and interesting way?  Does it have an appropriate degree of complexity?

(8) How well organized is the paper?  Does it stick to the point?  Does every paragraph contain a clear topic sentence?  If not, is another organizing principle at work?  Are the transitions well made?  Does it have a real conclusion, not simply a stopping place?

(9) Is the style efficient, not wordy or unclear?

(10) Does the writing betray any special elegance?

(11) Above all, can you hear a lively, intelligent, interesting human voice speaking to you (or to another audience, if that’s what the writer intends) as you read the paper?

Depending on answers to such questions, give the paper some kind of A or B.     

Sample Primary Trait Scale for a Technical Communication Memo

Rhetorical situation; writer-reader relationship; honesty (ethos, pathos)

     5-6 Document takes into account who the readers are and how the readers probably feel about the subject.  Shows awareness of how the document is going to affect primary and secondary readers. Responds accurately to requirements of the assignment. Text and graphics are honest; they don't distort facts or withhold important information the writer does not want the reader to know. The subject line is informative, specific, concise.

     3-4 Document is adequate in its relationship to the rhetorical situation. There may be noticeable but not damaging lapses in tone. In some minor way it may not quite answer the assignment. May have minor lapse in honesty, perhaps inadvertent. Subject line is informative but not as specific or concise as it could be.

     1-2 Document is flawed in its relationship to the rhetorical situation. It may be written in a tone inappropriate or offensive to the audience. It may miss the point of the assignment. It may be obviously dishonest, by misrepresenting or omitting facts, using misleading graphics, or creating a bias for or against some idea. The subject line is too vague or confusing to be informative.

Choice of information and ideas; accuracy, conciseness, comprehensiveness (logos)

     5-6 Document contains all the information and analysis the reader needs, and nothing that the reader does not need (taking into account, however, the varying needs of different readers). The information is accurate. The analysis and conclusions are logical and clearly related to the facts. General assertions are backed up with sufficient reasons, evidence, and specific examples. When appropriate, the writer draws creative, imaginative, and inventive conclusions from the facts.

     3-4 Document contains adequate information, but readers feel they need to know more. There may be extraneous information, but not enough to cause serious confusion. There may be minor inaccuracies in the information. The analysis and conclusions make sense in light of the facts presented, but they may appear unimaginative; they only point out the obvious. General assertions are mostly backed up with reasons, evidence, and specifics, but not to the reader's complete satisfaction.

     1-2 Document may be missing important information, or have much extraneous material that confuses the reader. Information may be inaccurate. It may make assertions and reach conclusions not warranted by the facts, not supported by specific examples, or not reasonably developed.  It may do an inadequate job of drawing conclusions from the facts; the reader can easily see meaning in the facts that the author failed to point out.

Organization, visual elements; accessibility; professional appearance

     5-6 Document design is clear, attractive, and professional. Information and ideas are easily accessible to the reader through coherent organization, advance organizers, paragraph divisions, section headings, etc.  Headings and paragraph indentations follow a clear hierarchical scheme. Lists are used where possible but not over‑used. Document follows the proper conventions of format. Graphics are clear and placed near the relevant text. Pages look balanced and open, with appropriate use of white space.

     3-4 Document is adequately designed, but there may be a few lapses in design or professional appearance. The reader may have difficulty finding different parts of the document and kinds of information. The hierarchy of headings and paragraph indentations may be unclear or confusing.  Lists are generally used well, though there may be an over-or under-usage of them. Graphics are mostly clear and related well to the text, though there may be some problems. Pages look balanced and open for the most part, but they may be somewhat inconsistent.

     1-2 Document design is confusing or looks unprofessional. The reader has serious trouble finding different parts of the document and kinds of information. Advance organizers may not be used where needed; paragraphing may fail to give clear indications of changes in topic; section headings may be missing or misleading.  May lack hierarchy of headings or paragraph indentations, or there may be a confusing one. Graphics may be unclear or hard to relate to the text. Pages look crowded or sloppy, or too much white space may make the document look choppy and disunified.

Coherence, clarity, correctness

     5-6 Writing style is direct, literal (rather than figurative or metaphoric), and unambiguous. Words are kept simple and used precisely.  Sentences are grammatically correct and stylistically effective, particularly in the use of emphasis points, parallel structure, subordination, and the given new contract. Punctuation is correct and is used effectively for separation and emphasis. There are no spelling errors.

     3-4 Writing style is mostly clear but may be occasionally too indirect, figurative, ambiguous, or vague. May be a few words misused or too much jargon.  May have a few ineffective sentences ineffective because of problems with sentence emphasis, subordination, parallel structure, or the given new contract. There may be spelling or homonym errors but no more than two per page.

     1-2 Writing style is too indirect, overly figurative, ambiguous, or vague. Words are inappropriately used, or there may be too much jargon for the intended audience. Sentences have errors that seriously interfere with easy reading. There may be several stylistic flaws in sentence emphasis, subordination, parallel structure, and the given new contract. Punctuation is haphazard and shows no sense of how punctuation tells readers how to read.  There are errors in spelling and confusion of sound-alike words.

Total points _______ Grade ________ [Scale:  A = 23-24; A- = 22; B+ = 21; B = 19-20; etc.]

Scoring Guide for Assignment on The Secret Sharer

Your essay should provide a supported answer to the following question:

How has the experience with Leggatt changed the captain so that the person he is at the end of the story is different from what he was at the beginning?

In order to do well on this paper, you need the accomplish the following:

(1) Underline sentence that serves as thesis statement, providing a precisely worded answer to this question.

(2) Support your thesis statement with convincing arguments and appropriate textual details.

(3) Make your essay clear enough for a reader to understand with one reading.

____________________________________________________________________________________________

Criterion 1.  Does your essay have a thesis statement that answers the question regarding changes in the captain?

No thesis or unclear thesis     clear thesis  
4   6 10

____________________________________________________________________________________________

Criterion 2.  Is your thesis supported with convincing arguments and appropriate, significant details from the story?

Weak arguments and/or lack of details as support     Strong arguement and Good details as support  
4   8 10

____________________________________________________________________________________________

Criterion 3.  Is your paper easy for a reader to follow?

Paragraphing and transitions 2 4 6 8 10
Clear, effective sentences 2 4 6 8 10
Accurate grammar, spelling, punctuation, mechanics 2 4 6 8 10

Grading Criteria for Thesis Support Microtheme in Finance

Support of Theses (each item graded S/U)

A.  Clarity of support:   _____
B.  Logic (relationship of support to thesis):   _____
C.  Sources of support:    
  (1) Quantity: _____
  (2) Quality: _____
Final Microtheme Grade:    _____

Specific features to focus on when resubmitting microthemes receiving Unsatisfactory evaluation:

        _____ Grammatical errors are numerous enough to interfere with reading of your response.

        _____ Organization of your response is confusing or difficult to follow.

        _____ Logical support of your thesis is confusing, misleading, contradictory, or incomplete.

        _____ Conclusions are not warranted by your support.

        _____ Support is imprecise or too general.


This website created and maintained by the Coe Writing Center. Copyright 2001.
Email Dr. Bob Marrs with any questions, comments or suggestions.