Strategies for Teaching Writing/Using Peer Review

Peer Review

Peer review is a natural fit for science communication settings, because it is such a vital component of the scientific publishing process. Thus, involving students in peer review may not only improve students’ writing, but also their understanding of science. Peer review can also help to make students more responsible and reflective when it comes to their written work¹.

When incorporating writing tasks into science classes, one of the main concerns educators have is the time required to grade and provide effective feedback. This is a greater concern with large classes. If your goal is to provide multiple writing opportunities to students to allow them to practice their writing skills, you may want to consider using peer evaluation to reduce the grading burden. However, it is important to be aware that some students can be mistrusting of peer review, thinking that it places their grades in the hands of unqualified peers ². To reduce the likelihood of your students thinking this, it is important to explain the many positive outcomes that should result from them engaging with peer review.

In this guide, we consider some of the challenges and best practice strategies, before focusing specifically on using peer review to provide students with formative feedback and for grading purposes.

Incorporating Peer Review into Your Class

Some of the greater concerns about incorporating peer review into a science class are highlighted below (Table 1), but you can employ the strategies matched with these to negate or offset them.

Table 1: Best Practice Strategies to Deal with Specific Peer Review Implementation Concerns

Concern/Issue – How to…	Associated Best Practice Strategy
Organize the distribution of student work to peers	Consider using a software system Ensure friends/group members receive work from other individuals/groups
Communicate instructions and feedback	Consider running a practice session first [see our guide and materials here] Provide guidelines with sample language Provide a rubric or detailed list of tasks
Motivate students to take the task seriously	Provide an incentive (small initially, rising in value with experience) as part of a grade Provide a bigger incentive for a second draft of work (after peer review)
Motivate students to incorporate peer feedback into their edits
Decide whether reviews should be anonymous	Judge the dynamic of your class In most cases, face-to-face interviews can clarify confusion and do not cause problems

Using Peer Review for Formative Assessment

Some specific strategies and things to consider that may help you handle peer review and address related concerns in your class include how you will:

Communicate the purpose of the activity with students to gain their support. Obviously, it is not a good idea to suggest that the students are “doing your work for you”. Instead, highlight the benefits of reading and critically analyzing other’s work. Discuss how being a peer reviewer will help them understand grading criteria better and spot weaknesses in writing structure and argumentation, which will in turn help improve their own work³. Emphasize how important it is for people to get feedback on their writing during the revision stage, and that you are helping facilitate that for the entire class.
Provide an appropriate incentive to students to encourage them to take the review process seriously. Depending on your class size, you may not have the resources to grade students’ reviews of each other’s work. However, you could set aside a component of the assignment grade to be determined by student perception of how useful the peer review is in helping them revise their work.
Train students on providing effective feedback ⁴. You may want to provide detailed guidance in the form of a checklist or rubric so that students know exactly what they should be considering when reviewing someone else’s work or at least provide prompts to make sure they pay attention to important elements of their peers' writing^{5, 6}.
Decide how many assignments to ask students to review. If they review two or more assignments, they can compare and contrast them as well as receive a wider range of feedback on their own work. Sometimes comparing different pieces of work helps students provide more objective feedback because they are better able to assess the strengths and weaknesses of writing when they see more than one example. While some research suggests students hold positive attitudes towards peer review after the purpose has been explained ⁷, other work shows that students trust the process more as they get more experience of it ⁸. As a result, incorporating more than one round of peer review should increase the chances that you students will have greater faith in what they are doing, and why they are being asked to do it.
Decide whether to include a face-to-face discussion to go along with the paper-based or software-based reviews. It can be useful for students to hear directly from their peers about what was unclear and excellent about their writing assignment. Face-to-face discussions allow for clarification and may even lead to paired brainstorming about revisions. Of course, this would mean that reviews are not anonymous, but there is little evidence that suggests blinding peer reviews is more useful to learners.
Encourage students to incorporate their reviewers’ comments into their work. You may want to have students write a cover page, similar to a letter to the editor of a journal, discussing how they incorporated their reviewers’ comments so that they must tackle this task.
Organize the logistics of the whole process. Software systems can do just this. These tools will greatly reduce the burden of organizing the reviews if you have a large class.

An Example of Peer Review in Communicating Science (SCIE 300) at UBC

Here we describe an example of how peer review is used in a third-year communicating science class at our institution. Students work individually to prepare a research paper written in scientific journal style. Their papers are based on a small-scale scientific investigation they performed in groups of 3-4. Four days after the draft papers are due as submissions to the course learning management system, students bring two copies of their paper to class. While students are working on an in-class activity, the instructor distributes the papers to the reviewers. We make sure that students in the same group are not assigned to review each other’s work and ensure that the review class peer discussions will be able to occur without anyone being left out of the discussion.

We give students a rubric and ask them, as homework, to mark up the papers as needed. Four days later, we pair students up in class to discuss their reviews. Each pair has ten minutes to discuss one paper, and then they switch.

The key features that make this process run smoothly are:

Students are accountable and therefore invested in the process because they need to submit their draft online before coming to class. There are a few buffer days for late submissions.
We provide a rubric to guide students’ reviews.
We provide guidance about exactly what the pairs should be discussing.

Do you have examples of effective peer review that you would like to share? Please contact us here if so.

Using Peer Review for Grading Purposes

While having students grade each other’s work can be a time-saving measure, it takes time and thought to incorporate peer evaluation into your class successfully. Here we describe some best practices to consider before implementing peer evaluation with your students. Benefits to peer evaluation include increased responsibility and autonomy, improvement in critical thinking, and improvement of structured thinking.

Many of the same considerations must be given when using peer review for grading purposes as for formative purposes (discussed above). However, there are some additional things to consider, and these are listed below:

1. As above (in the Using Peer Review for Formative Assessment section), it is not a good idea to suggest that the students are “doing your work for you”. Instead highlight the benefits of reading and critically analyzing others’ work.

Be aware that that some students may have negative perceptions of peer evaluation and perceive it as unfair (Smith et al. 2002, Kaufman and Schunn 2011). However, these perceptions may change positively as a result of participating in peer evaluation (Wen and Tsai 2006), so it is important to ensure that students get something positive from their first experiences of the process.

2. As above, it is important to provide an incentive to students to encourage them to take the grading process seriously.

However…

3. It is very important to limit the peer evaluation component of the students’ overall course grades to reduce the perception that their grade is mostly left up to a non-expert or to chance. This is critical when you are piloting peer evaluation in your class.

 4. Be sure to have a plan in place for how to deal with student complaints. You may want to set up a formal grievance process.

5. Consider whether your peer evaluation is going to be anonymous or not. Most of the software systems below are anonymous. If not anonymous, consider how you will deal with conflicts of interest, such as two best friends being assigned to each other’s work.

Using Software or Calibrated Peer Review Software Systems

What follows is a brief overview of software systems you may want to consider. The benefit of using a software program to handle peer evaluation is that they can simplify the logistics of assigning, collecting, and returning grades, and providing feedback. Using these systems requires a considerable initial time commitment to set up the assignment and grading rubric(s), but the time savings come later when they construct grades for you and provide feedback directly to students. It is also very important to ensure you make it clear to your students that any software systems used for peer review offer learning benefits to them ⁹, as well as smoothing the assignment logistics for you.

Calibrated Peer Review. The CPR software was created at UCLA and takes users through a sequence of submitting their own work, training on grading other’s work (“calibration”) and grading their peer’s work. There is a bank of assignments and it is easy to create your own. Check out the extensive list of publications on the CPR website.
iPeer (UBC). iPeer is an open-source web-based software application that allows instructors to create assignments and rubrics, send reminders to students, and provide feedback. Evaluations can be based on rubrics, and the system can be used for evaluating the contributions of group members in team projects.
PeerMark (Turnitin). This component of Turnitin distributes student work for peer grading according to instructor-given criteria. Reviewers may comment and add editing remarks. Your institution may have a license with Turnitin, and it may be integrated with your learning management system (LMS). At UBC, Turnitin is not integrated with the LMS because Turnitin stores the data in the United States. Be sure only anonymous student work is uploaded to Turnitin if your institution is in Canada.
MyWritingLab (Pearson). This proprietary software allows for “facilitated peer review”, which means that students may provide comments on other students’ work, as well as grade it using an instructor-provided rubric.
Peerceptiv (formerly SWoRD Peer Assessment). Peerceptiv engages students in double-blind reviews using instructor-created rubrics. This system claims to motivate students, eliminate bias, and generate useful analytics for instructors. Student work is graded by three to six peers and students have the opportunity to “back evaluate” to rate the helpfulness and specificity of the review. SWoRD has been heavily researched by the University of Pittsburgh and is offered by Panther Learning.

Do you have other examples of peer evaluation software that you have had success with? Please contact us here if so.

Useful References

1. Dochy, F, Segers, M, Sluijsmans, D. The use of self-, peer and co-assessment in higher education: A review. Stud High Educ. 1999; 24(3):331–350.

2. Kaufman, JH, Schunn, CD. Students’ perceptions about peer assessment for writing: their origin and impact on revision work. Instr Sci. 2011; 39:387-406.

3. Cho, YH, Cho, K. Peer reviewers learn from giving comments. Instr Sci. 2011; 39:629-643.

4. Stanley, J. Coaching student writers to be effective peer evaluators. J Sec Lang Writ. 1992; 1(3):217-233.

5. Cho, K, Schunn, CD, Wilson, RW. Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. 'J Educ Psych. 2006; 98(4):891-901.

6. Gan, MJS, Hattie, J. Prompting secondary students' use of criteria, feedback specificity and feedback levels during an investigative task. Instr Sci. 2014; 42:861-878.

7. Keaten, JA, Richardson, ME. A field investigation of peer assessment as part of the student group grading process, paper presented at the Western Speech Communication Association Convention. 1992; (pp. 1–34). Presented at the Western Speech Communication Association Convention, Albuquerque.

8. Smith, H, Cooper, A, Lancaster, L. Improving the quality of undergraduate peer assessment: A case for student and staff development. Innov Educ Teach Int. 2002; 39(1):71–81.

9. Wen, ML, Tsai, CC. University students’ perceptions of and attitudes toward (online) peer assessment. High Educ. 2006; 27(18):27–44.

The Giving Effective Feedback - Peer Review - Student Handout

Giving Effective Feedback – Peer Review

Tips and Example Feedback

Try to follow some of these tips when giving feedback to your peer(s), and always try to imagine things from their perspective; ask yourself whether your peer(s) will be able to use your feedback to improve their writing.

1. Be supportive as well as honest

“This paragraph confuses me because…”
“You have an interesting detail here that I almost missed because…”
“Your opening thesis statement told me exactly what you were going to argue, but the logic of your argument was hard to follow because…”

2. Be specific (why is something confusing?)

“This word means something different to me because…”
“I am not sure which example you are referring to here because…”
“This sentence might be clearer if you wrote it in the active voice because…”

3. Write comments on the draft and use additional paper if you need to

Make sure these comments are specific and easy to interpret
Try not to edit someone’s work; instead, tell them how they can edit it
Use a coding system to highlight related errors/issues (e.g. circles around tense issues, underline sections where the logic doesn’t flow…)

4. Keep feedback confidential

5. Focus on the paper, not the person

6. Use plenty of “I” statements to de-personalize your feedback (rather than “You” statements, which can make people uncomfortable)

“At this point, I thought the essay was going to next consider…”
“I would probably find this section easier to understand if…”
“I like the way this point is connected to the next one, and would find the argument easier to follow if all the points were connected like this…”

7. Provide suggestions rather than commands

“Consider moving this point up one paragraph so that it follows on directly from this point…”
“I think this section would be easier to read if it was written in the active voice…”
“It might be worth finding another example that provides support for this claim to make the argument more convincing…”

Essay For Feedback - Peer Review Exercise

Should limited research funds be allocated to basic or applied research projects?

Scientific research can be conducted as basic or applied, basic research is more general and includes learning about topics such as insect behaviour whereas applied research is instead performed with a more specific goal in mind to solve particular problems such as how to prevent the spread of malaria with different kinds of drugs in areas affected by malaria. Both research types are different but both are important for improve human quality of life.

Basic research is very important becase without building a foundation of knowledge it is very hard to tackle specific problems in the world. For example, basic research like people investigating the manifold ways in which chemical compounds react and bind with one another in solutions and how these ways are affected by the molecular structure of the compounds is very important. Deadly diseases have been responsible for huge numbers of deaths in years gone by but the field of medicine has successfully developed medicines that treat and cure many of these. All treatments must be tested carefully in applied testing trials before it is used to save lives, however it is very rare that it could be developed at all without there being prior knowledge about how it might be made to specifically tackle the disease agent which is why it is so important to have deon the initial applied work. So it is very important that basic research is performed, that can show things such as in what conditions the drug might be good, whether there might be similar compounds from other sources that might do the same thing etc. Applied research builds on this detailed, broad knowedge in a more specific way. A good example is that knowledge of the molecular structure of certain compounds can be used by those with a mind to work out why they might react different in different environments. If certain compounds reacted differently with water than with dry earths then agriculturalists might research different mixes of compounds used as fertilisers to be used by farmers in different environments that are wet or dry. So they might predict and be right that one is more useful in the pacific northwest than in California and arizona and save money and produce more crops in the two regions. Basic research does often enhance knowledge that can be used to solve problems in an unexpected way too though, and this is a major reason why it is very important to improve science as a whole. For example, scientists performed basic research into the way that sharks skin cells aligned with each other after they noticed how rough they felt to the touch. Further down the line, unrelated research teams used the data to perform applied research and design clothing materials for people working in harsh enivieonrments like extremely cold places or where there is a need to not tear clothing like in a building where corrosive or poisonous solutions will be regularly used.

To summarize, I think that basic and applied research are both important and scientists should continue to work at both types so that more general and specific discoveries can be made to significantly improve the quality of life that we as people will enjoy because if only one type of research had been performed in the past we might not have done many important things such as reduced the spread of malaria, investigated the likelihood of life existing on Mars, or even invented the iPad.

Peer Review Rubric

PEER REVIEW ASSIGNMENT – RUBRIC

Author’s Name: __________________

Your Name: __________________

Instructions to author: When the peer review begins, take notes. These notes are for your own benefit when revising and do not need to be submitted. Listen to your peer and avoid getting defensive or apologizing.

Instructions to peer reviewer: Read this peer review form first, and then read your peer’s paper. You can make annotations on the paper, and/or on this form. The paper’s author will keep this form and the annotated paper. You will also have the chance to present your comments, clarify your points and make suggestions during a conversation with your peer.

Identify the paragraph or section of the paper that you think is most effective, and draw a box around it. For this section of the paper, please answer the following questions: 1) What makes this section the most effective? Be specific in your answer. 2) What is the role of this section in helping you understand the science that the author is reporting?

Identify the paragraph or section of the paper that you think is least effective, and draw a circle around it. For this section of the paper, please answer the following questions:
	Yes	Somewhat	No
Does this section advance the point the author is trying to make?
Is the content of the section problematic?
Is the organization of the section problematic?
Is the writing in this section problematic?
Provide suggestions for improvement of this section. Make a minimum of one suggestion.

If the paper includes errors in any of the following, circle it here and on the paper draft. Try to include helpful comments about the errors.
topic sentences	colon
paragraph structure	semicolon
active & passive voice	capitals
numbers/units	apostrophe
comma	hyphen
plurals	abbreviation/acronym

Keeping in mind the target audience, answer the following questions:
	Yes	Somewhat	No
Is the tone of the paper appropriate?
Is the level of scientific knowledge required of the reader appropriate?
Is the technical language appropriate (e.g. too much jargon)?
Are headings used logically to divide up the paper?
Are there any instances of awkward or ambiguous wording?
Is the paper largely free of grammatical or spelling errors?

Check individual parts of paper and flag problem areas:
	Yes	Somewhat	No
Abstract: Are the core contents of the paper concisely described?
Introduction: Is the rationale or motivation explained?
Introduction: Is sufficient background provided for the study?
Methods: Is there enough detail to allow another scientist to repeat the study?
Results: Are the results presented in a clearly organized manner?
Discussion: Are the findings accurately interpreted? i.e. are there errors in the science?
Discussion: Are interesting implications of the findings described?
Discussion: Is each conclusion supported by sufficient evidence (data, examples)?
Discussion: Are limitations or remaining questions assessed?
Figures/Tables: Do these improve the readability of the paper?
If you answered ‘No’ to any of the above, provide the author a brief explanation below.

Use this space to make additional comments about formatting or style and/or to note concerns you have about this paper not meeting the expectations or goals of the assignment.