Grading Techniques/Creating a Rubric
Each rubric will differ based on the assignment you ask your students to complete, but you should focus on the specific learning objectives you wish students to develop3. Think about the observable attributes you want to see from them, as well as those that you don’t.
If you are creating a rubric for an existing assignment that has been used in past offerings of the course, it may be useful to re-read some of those assignments to gain a perspective of the typical spectrum of answers that students provided; it is helpful to predict the kind of answers you expect to see when designing a rubric to cover as many of these as possible, as well as keeping in mind what students must typically show in their assignments to achieve low, middle and high grades.
When creating rubrics (particularly if they are holistic), it is helpful to divide each criterion into levels, such that you create categories that relate to different responses that reflect the progression from novice to expert-like writing (e.g. a score of 1 might represent emerging ability, whereas a score of 5 might represent the quality of an expert.
Defining these categories can be useful for students who wish to monitor their learning progress, especially because the scores/marks will not represent linear progression (e.g. a student who scores 4 out of 5 for logical development does not mean that student has twice as much ability as one who scores 2 out of 5).
If you are creating a rubric for a new assignment, you should keep the following tips in mind:
1. First divide the total marks for the assignment into different sections (e.g. five marks for the depth of content, five for the quality of sources used, five for integration of these sources, five for quality of argument, and five for paragraph structure and transitions). Note that most rubrics focus on six to seven criteria4, but the absolute number should depend on what the assignment5 asks of students; in most cases, limiting the number of criteria is more practical, but sufficient scope should exist within these criteria to distinguish the full range of likely student abilities.
2. Provide a detailed explanation of what an answer would need to show to be awarded any mark/score within the range available for each section (e.g. if you have allocated up to five marks for the depth of content, you should clearly state what a student must cover to gain one, two, three, four and five marks).
3. Do not use potentially ambiguous explanations. For example, do not propose a scale of 0 – 3 marks where weak, fair, good and very good are the descriptors used to differentiate between scores of 0, 1, 2, and 3 because different graders will likely differ on their interpretation of what is weak, fair, good or very good. Instead, try to provide objective definitions (e.g. less than one primary source = 0, one or two primary sources = 1…). If your rubric is holistic rather than analytic, you should provide detailed summaries (with examples) to clearly distinguish between marks/scores.
4. Add a qualitative description to each of these marks/scores if you plan to share the rubrics with students at any stage, so that they can assess their own learning development based on the grade they receive for the assignment.
5. Share the rubric with colleagues and TAs and ask for feedback. Expect to make changes based on this feedback.
6. If you plan to share the rubrics with students, ask these same students whether it is clear to them before the assignment begins. If they misinterpret any part of it, this may be a sign that you should make changes.
By following the above steps, you should design rubrics that are straightforward and objective to use. However, when more than one person will grade the assignments, it is important for all graders (e.g. instructors and TAs) to meet to troubleshoot any issues at an early stage. Ensuring your graders can use your rubric objectively is a crucial part of the design process, which is why the following steps are very important before you grade the assignments from an entire class:
7. Try to select a range of assignments from students of varying abilities and then take turns to grade each one using the rubric.
8. Compare the grades you each assigned the same assignments and calculate your inter-rater reliability (see below).
9. If you have a low reliability (e.g. you have awarded different grades to the same assignments), this indicates issues with the rubric or the way it is being interpreted.
10. In such a scenario, work through each section and pinpoint where – and why – graders have awarded different scores based on the rubric, and then rephrase the rubric to make it more objective.
11. Grade a new set of assignments in the same way before comparing grades until you are satisfied that your inter-rater reliability is sufficiently high, and you are confident that you are all using the rubric in the same way.