Science:Science Writing Resources/Podcasts

Peer Review

In-Class Practice Session

Peer Review: In-Class Practice Session

This in-class peer-review practice session is designed to give students some experience of the process before working with real examples provided by their peers. It should take approximately 50 minutes. The main aims are to consider what sort of feedback is most useful, what sort of language offers constructive help, and how to go about providing feedback to avoid editing an entire piece of writing.

You should hand out copies of the sample Essay for Feedback – Peer Review Exercise and the Giving Effective Feedback – Peer Review – Student Handout documents found in Using Peer Review.

Session Timeframe

The session can be broken down as follows:

1. Five minutes: Put students into groups of 3-5 and share the sample essay and the assignment details so the students know what the original task was for the essay author:

The author needed to write a short argumentative essay in which he/she answered the following prompt: ‘Should limited research funds be allocated to basic or applied research projects?’ They should have taken a stance and defended it by making claims and supported them with specific pieces of evidence.

2. Five minutes: Review higher and lower-order concerns and ask student to prioritize their responses accordingly.

Higher-order concerns include things such as poor logic, organization, and not backing claims up with evidence, whereas lower-order concerns include things such as poor grammar, over-use of jargon, and a lack of transitions and everyday analogies.

3. Five minutes: Talk about using constructive and productive language, even when it may be challenging, and refer students to their Effective Feedback handout for further guidance and some examples.

Ask the students to think about how they could best help the author improve their work as a direct result of the feedback they provide. Stress the importance of being supportive yet honest, and being very specific in their feedback.

4. Fifteen minutes: Have each group read the sample essay and talk about what feedback they would give the author. Check in with each group throughout, answer questions, etc. Ask each group to come up with specific feedback.

5. Ten minutes: Each group must delegate a speaker to come up and share that feedback with the class as if the class was the student who wrote the piece.

6. Ten minutes/debrief: Compare the feedback offered by each group, and ask the class as a whole which feedback worked, and which didn’t? Why? How did the class feel in general when the feedback was shared with them?

The Giving Effective Feedback - Peer Review - Student Handout

Giving Effective Feedback – Peer Review

Tips and Example Feedback

Try to follow some of these tips when giving feedback to your peer(s), and always try to imagine things from their perspective; ask yourself whether your peer(s) will be able to use your feedback to improve their writing.

1. Be supportive as well as honest

“This paragraph confuses me because…”
“You have an interesting detail here that I almost missed because…”
“Your opening thesis statement told me exactly what you were going to argue, but the logic of your argument was hard to follow because…”

2. Be specific (why is something confusing?)

“This word means something different to me because…”
“I am not sure which example you are referring to here because…”
“This sentence might be clearer if you wrote it in the active voice because…”

3. Write comments on the draft and use additional paper if you need to

Make sure these comments are specific and easy to interpret
Try not to edit someone’s work; instead, tell them how they can edit it
Use a coding system to highlight related errors/issues (e.g. circles around tense issues, underline sections where the logic doesn’t flow…)

4. Keep feedback confidential

5. Focus on the paper, not the person

6. Use plenty of “I” statements to de-personalize your feedback (rather than “You” statements, which can make people uncomfortable)

“At this point, I thought the essay was going to next consider…”
“I would probably find this section easier to understand if…”
“I like the way this point is connected to the next one, and would find the argument easier to follow if all the points were connected like this…”

7. Provide suggestions rather than commands

“Consider moving this point up one paragraph so that it follows on directly from this point…”
“I think this section would be easier to read if it was written in the active voice…”
“It might be worth finding another example that provides support for this claim to make the argument more convincing…”

Essay For Feedback - Peer Review Exercise

Should limited research funds be allocated to basic or applied research projects?

Scientific research can be conducted as basic or applied, basic research is more general and includes learning about topics such as insect behaviour whereas applied research is instead performed with a more specific goal in mind to solve particular problems such as how to prevent the spread of malaria with different kinds of drugs in areas affected by malaria. Both research types are different but both are important for improve human quality of life.

Basic research is very important becase without building a foundation of knowledge it is very hard to tackle specific problems in the world. For example, basic research like people investigating the manifold ways in which chemical compounds react and bind with one another in solutions and how these ways are affected by the molecular structure of the compounds is very important. Deadly diseases have been responsible for huge numbers of deaths in years gone by but the field of medicine has successfully developed medicines that treat and cure many of these. All treatments must be tested carefully in applied testing trials before it is used to save lives, however it is very rare that it could be developed at all without there being prior knowledge about how it might be made to specifically tackle the disease agent which is why it is so important to have deon the initial applied work. So it is very important that basic research is performed, that can show things such as in what conditions the drug might be good, whether there might be similar compounds from other sources that might do the same thing etc. Applied research builds on this detailed, broad knowedge in a more specific way. A good example is that knowledge of the molecular structure of certain compounds can be used by those with a mind to work out why they might react different in different environments. If certain compounds reacted differently with water than with dry earths then agriculturalists might research different mixes of compounds used as fertilisers to be used by farmers in different environments that are wet or dry. So they might predict and be right that one is more useful in the pacific northwest than in California and arizona and save money and produce more crops in the two regions. Basic research does often enhance knowledge that can be used to solve problems in an unexpected way too though, and this is a major reason why it is very important to improve science as a whole. For example, scientists performed basic research into the way that sharks skin cells aligned with each other after they noticed how rough they felt to the touch. Further down the line, unrelated research teams used the data to perform applied research and design clothing materials for people working in harsh enivieonrments like extremely cold places or where there is a need to not tear clothing like in a building where corrosive or poisonous solutions will be regularly used.

To summarize, I think that basic and applied research are both important and scientists should continue to work at both types so that more general and specific discoveries can be made to significantly improve the quality of life that we as people will enjoy because if only one type of research had been performed in the past we might not have done many important things such as reduced the spread of malaria, investigated the likelihood of life existing on Mars, or even invented the iPad.

Peer Review Rubric

PEER REVIEW ASSIGNMENT – RUBRIC

Author’s Name: __________________

Your Name: __________________

Instructions to author: When the peer review begins, take notes. These notes are for your own benefit when revising and do not need to be submitted. Listen to your peer and avoid getting defensive or apologizing.

Instructions to peer reviewer: Read this peer review form first, and then read your peer’s paper. You can make annotations on the paper, and/or on this form. The paper’s author will keep this form and the annotated paper. You will also have the chance to present your comments, clarify your points and make suggestions during a conversation with your peer.

Identify the paragraph or section of the paper that you think is most effective, and draw a box around it. For this section of the paper, please answer the following questions: 1) What makes this section the most effective? Be specific in your answer. 2) What is the role of this section in helping you understand the science that the author is reporting?

Identify the paragraph or section of the paper that you think is least effective, and draw a circle around it. For this section of the paper, please answer the following questions:
	Yes	Somewhat	No
Does this section advance the point the author is trying to make?
Is the content of the section problematic?
Is the organization of the section problematic?
Is the writing in this section problematic?
Provide suggestions for improvement of this section. Make a minimum of one suggestion.

If the paper includes errors in any of the following, circle it here and on the paper draft. Try to include helpful comments about the errors.
topic sentences	colon
paragraph structure	semicolon
active & passive voice	capitals
numbers/units	apostrophe
comma	hyphen
plurals	abbreviation/acronym

Keeping in mind the target audience, answer the following questions:
	Yes	Somewhat	No
Is the tone of the paper appropriate?
Is the level of scientific knowledge required of the reader appropriate?
Is the technical language appropriate (e.g. too much jargon)?
Are headings used logically to divide up the paper?
Are there any instances of awkward or ambiguous wording?
Is the paper largely free of grammatical or spelling errors?

Check individual parts of paper and flag problem areas:
	Yes	Somewhat	No
Abstract: Are the core contents of the paper concisely described?
Introduction: Is the rationale or motivation explained?
Introduction: Is sufficient background provided for the study?
Methods: Is there enough detail to allow another scientist to repeat the study?
Results: Are the results presented in a clearly organized manner?
Discussion: Are the findings accurately interpreted? i.e. are there errors in the science?
Discussion: Are interesting implications of the findings described?
Discussion: Is each conclusion supported by sufficient evidence (data, examples)?
Discussion: Are limitations or remaining questions assessed?
Figures/Tables: Do these improve the readability of the paper?
If you answered ‘No’ to any of the above, provide the author a brief explanation below.

Use this space to make additional comments about formatting or style and/or to note concerns you have about this paper not meeting the expectations or goals of the assignment.

Grading and Providing Feedback

Introduction

Grading student papers with efficiency is a vital skill for instructors to hone, but providing effective feedback on these papers is every bit as important. Without detailed, personal feedback, students may feel as though their efforts have gone unnoticed, and, more importantly, they may not know how to address conceptual weaknesses in future work. However, instructors and TA's have limited time available for grading, especially when teaching large classes.

In this podcast, we hear from two instructors with experience in balancing the grading load and providing feedback effectively and efficiently. They pass on their own tips to adopt or adapt, and also discuss other best-practice ideas to help plan how to provide feedback after grading different types of assignment.

We have created two complementary resources that include useful, summary tips that may prove helpful in shaping your grading and feedback plans, and in designing and calibrating rubrics to limit grading queries when the papers are already piled high on your desk. These are both available to download once you have contacted a site administrator here. Once you have provided your details (including a verifiable academic institution email address) you will receive a password that will enable you to download the materials.

Grading and Providing Effective Feedback Handy Hints

Grading and Providing Effective Feedback Handy Hints

The following two handy hints should help you provide effective feedback while saving as much time as possible when grading.

1: Use an Annotation Key

When providing feedback on open-ended questions and essays, you might wish to use an annotation key to highlight similar errors, rather than explaining them again and again in your written feedback (e.g. choose one shorthand symbol for grammatical issues, another for content weaknesses, and another for poor logical development or argumentation etc.).

Spend some time explaining each error that has its own symbol when it first occurs, but afterwards simply mark repeated errors with the same annotated symbol. For example, if you notice a student shifts tense repeatedly within his/her sentences, and this makes the writing hard to follow, you should explain why this is the case at the point in the writing where this first occurs. You may wish to then provide an edited version of the sentence to make it clear what you mean, but try to refrain from editing too much of a student’s work (this is very time consuming and isn’t your job). When the student next shifts tense inappropriately, simply use the same symbol to indicate this.

Consider using different coloured pens for different errors. Depending on the assignment and its length, this can be more trouble than it is worth, and it does make things more time consuming. However, if an assignment is relatively short or if you are only assessing skills relating to a few concepts, it may provide a quick and easy way to group similar errors and/or feedback for improvement together.

Always remember to include the annotation key so that students understand which specific error/weakness relates to each symbol and/or colour.

Use the same symbols and/or colours for the same errors/weaknesses for all assignments that you grade. This will help you save time and may also encourage students to work together to improve their own writing via peer review.

2: Provide a Brief Feedback Summary

When you provide a final summary of feedback, try to make comments that could be used to improve the piece of writing, rather than attempting to edit the writing or list each and every specific weakness, which your annotation key and its symbols should already do.

Focus your feedback on one or two key areas of improvement. If a student’s work needs discussion beyond this, consider asking/requiring them to addend office hours to go over feedback in person because this will often be quicker and easier for both parties. When choosing what to focus on, target higher-order concerns over lower-order concerns (see Table 1).

Table 1: Higher-order concerns are more important in dictating the quality of written work, whereas lower-order concerns can usually be addressed more easily in future writing assignments once students are aware of their specific areas of weakness.

Example Higher-Order Concerns	Example Lower-Order Concerns
Lack of organization (paragraph issues)	Overuse of jargon
Poor thesis and development statements	Overuse of the passive voice
Weak evidence provided for any claims	Use of inappropriate analogies
No citations, or errors in citations	Lack of punctuation
Poor logical development/argumentation	Inappropriate style for numbers and units
Poor grammar	Spelling errors

It is easy to overlook the need to praise students for things they do well, especially when time is limited, but it is important to let them know what they are good at. This not only encourages them but also makes sure they don’t spend time trying to improve something that is already satisfactory, so make sure you add positive elements to your feedback.

Encourage students to keep lists of the things they need to improve, so that they focus on these the next time it comes to completing a similar assignment. Suggest that they should read your feedback from past assignments when editing/revising their work to reduce the chance that they make similar errors again (this is common).

Grading and Providing Feedback: Training Teaching Assistants - Calibration

Grading and Providing Feedback: Training Teaching Assistants – Calibration

A major concern when grading assignments in large classes/sections is the need to divide the grading load among instructors and teaching assistants, while maintaining consistency. This guide is designed to help you run a calibration workshop to achieve that consistency.

Before

1: Design a specific rubric

Design a rubric that provides an objective guide as to how every mark should be distributed. Start by stating what a student needs to include in their answer to obtain 1 mark in a question. Then, show specifically what additional material they need to include to obtain all variations in marks, up to the maximum available. Such an approach should leave a grader in no doubt as to what they should award any given answer.

If possible, do this for open-ended essay questions, as well as short-answer questions. Providing ambiguous criteria, such as award between 5 marks (for exceptional transitions) and 0 marks (for no transitions or very weak transitions) will result in very confused graders. Conversely, stating something quantifiable (such as count the number of individual errors in transitions and deduct 1 mark for each from a starting 5 marks) makes it easier for graders to be objective.

2: Select a handful of papers to be graded by everyone

If the assignment has been completed in previous years or terms, select a variety of papers that showcase a range of grades (some high, some low, and some in the middle). Try to choose 3-5 papers to give enough of a sample and provide variety without providing so many that graders are overworked before the real grading begins.

If the assignment is a new one, compile some fictional, expected answers that should obtain a range of grades when graded with your rubric.

Ask your graders to work alone and use your rubric to grade these assignments.

3: Hold a workshop where all graders discuss the grades they assigned

Go through each graded question with everyone, and discuss the grades that were assigned. Where these differ, discuss the reasons for such differences with the graders and troubleshoot the rubric (and graders’ interpretations) together until everyone is confident they would award the same grades for each question.

4: Revise rubric

It may be necessary to make changes to the rubric where graders provided different grades for certain questions. If the same answer to a given question sees a wide range of grades from graders, this is an indication that the rubric needs to be improved for this question.

Make sure all graders contribute to this process and leave with the new rubric, confident that they can use it objectively.

Although this stage seems like adding another big time commitment to the process, it can save a lot of time further down the line; you will need to spend considerably less time double-checking grading, answering questions about interpreting the rubric and/or results etc., if you tighten up the rubric before grading begins.

During

5: Encourage open discussion among graders as they grade

There will likely be ambiguities that only arise when students complete assignments due to a lack of clarity in certain sections of the rubric, or due to unforeseen – but not technically incorrect – answers arising.

If graders communicate these with one another (over email, in focus groups, or in class planning time), they will help to devise objective solutions much faster. Flagging troublesome answers with one another will draw in feedback from other graders, and a majority vote may prove helpful in deciding how to grade such responses.

After

6: Compare mean grades and variation around these for each grader

There will probably be inherent variation in grades (and in the mean grade and the variation around this) in different sections. However, when section sizes are large, this should be relatively minimal.

If one section does have a significantly higher mean grade, it would be wise to review a handful of papers graded from that section (perhaps even passing these out to graders from other sections). If these different graders grade the papers a little lower, it might be that a standardised adjustment needs to be made to all papers. At the least, some further investigation should take place to make sure these students have not received artificially high grades.

7: Compare random papers from each grader

Even if mean grades are similar between sections, it is good practice to compare papers from different graders to make sure there is close agreement between them. Some variation is to be expected, but if different graders differ greatly in the grades they provide for each assignment, then further investigation is merited.

General Tips

A: Consider randomly assigning papers to graders

Making all assignments anonymous to the grader removes any subjectivity in how they will approach their grading. It can also pay to split assignments up from different sections, or have graders only grade assignments that come from sections they don’t teach.

B: Eliminate all ambiguity/subjectivity from rubrics

Try to provide clear, non-ambiguous scales for all questions, so that every mark can be accounted for. Try to avoid framing subjective scales and instead provide a quantifiable means of grading every answer. For example, try not to ask graders to award 1 - 3 marks based on an answer being poor – excellent; instead ask the graders to award 3 marks for an answer with no mistakes, 2 for one with one or two mistakes, and 1 for an answer with more than two mistakes, for example.

C: Advise graders to grade the same question back-to-back

When grading an assignment, it will save time and enhance objectivity if graders work through the same question sequentially on different students’ assignments, rather than working through each assignment from Question A to Question X. It is easier to memorize the specific grading criteria for a given question on the rubric than to work with different questions, one after another.

Tutoring

Encouraging Students to Seek Tutoring Help

Encouraging Students to Seek Tutoring Help

Many students think tutoring for writing is only meant for those with major deficiencies in their writing, or for those who are transitioning into a non-native language. In reality, tutoring is suitable for almost all students, including those who are strong writers. Writing centres and writing tutors work with postgraduate, undergraduate, domestic and international students who wish to improve specific elements of their writing.

A particularly good time to remind students that they can access writing tutoring is when you set new assignments or grade old ones. Students are often aware of elements of their writing that they can improve on, but there is nothing like a new challenge or the assessment of an old one to spur them on to address these issues.

At UBC, students are able to register and book appointments at the Writing Centre online, up to three weeks in advance. They can book up to one hour per week. Most other universities have their own writing centres and/or specialist support services.

Preparing Students for Tutoring Session

Preparing Students for Tutoring Sessions

Although the onus is firmly on the student to prepare for a tutoring session, you can help them do this effectively by providing feedback on areas of their writing that could be improved. Additionally, tutoring sessions are most effective when students target one or two areas, so encourage students to look over old writing assignments to try to build a picture of where their weaker areas lie.

Remind students that tutors will not proofread or edit their work, nor will they check over homework. They will help students become better writers by challenging them to think about how best to present their topics and by focusing on the writing process.

You should mention to students that generalist writing tutors will help them assess how well they develop pieces of writing (e.g. the logic of their arguments, the organisation of their writing). They will also help with general writing skills, such as grammar and the use of mechanics. Specialist science writing tutors will instead focus more on discipline-specific requirements, such as how to integrate sources (which is very different from the way non-scientists do this). They will also be able to help students identify different niche requirements for science writing. Whether students book tutoring time with generalist or specialist tutors depends on the goals they have for developing as writers.

Working with Students to Support Tutoring Sessions

Working with Students to Support Tutoring Sessions

Before students attend their first tutoring session, encourage them to:

Gather examples of their writing (after it has been graded, and when it incorporates feedback if possible) that showcases the specific areas that they wish to work on.
Speak to you (as their instructor) or to a TA to clarify any feedback and issues that may be present with their writing, so that they are clear on what they wish to develop (and why) before attending a session.
Reflect on their writing process so that they are better able to explain any difficulties they have (e.g. do they struggle to develop a writing outline before starting to write, do they find it difficult to create an effective thesis, or do they need help to design a better editing method?)

After students have attended their first tutoring session, encourage them to:

Write a brief summary of their session and share this with you as something of a battle plan. This will help you keep an eye out for specific elements of their writing that they are working on, so that you may give even more tailored feedback in future assignments. It will also create a form of accountability, which will encourage students to work on what they have agreed with their tutor(s).
Book a follow-up session with their tutor(s) for the same reason. Adding extra motivation and accountability to the process will help students to continue developing their writing skills.

If requested, provide students with examples of other work that is particularly strong in the areas that they are trying to improve, so that they can view examples while honing their skills. This need not be from the same class if you are worried about confidentiality or plagiarism issues.