Documentation:World Cafe/Mid-Term Evaluations of Teaching

Events

Part of Celebrate Learning Week 2012: Presenters: Kiran Mahal - AMS VP-Academic; Simon Bates - CTLT Academic Director/ Facilitators: Cindy Underhill and Janet McCracken (CTLT).
Further background details on the project on the AMS website.

Context

Done midway through the delivery of a course, midterm evaluations offer the potential for substantial benefits to both instructors and students: it engages both in a dialogue about aspects of the course, can significantly reduce the 'distance' between cohort and instructors and provides an opportunity to make changes in real-time, to impact the current cohort taking the course. There are many ways in which such informal, partnership evaluations can be undertaken, and significant activity already at UBC.

Order of events

Participants were split into groups of 4 by table, and were asked to discuss 2 questions regarding midterm evaluations: 1. What does a successful midterm evaluation of teaching look like? 2. What questions should you (or should you not) ask in a midterm evaluation? Thoughts and opinions were noted down on sticky notes and put onto a big poster at the front of the room. The poster itself was divided into different sections according to the questions. Also included were the "student parking lot" and a "faculty parking lot" sections, where challenges from either perspective regarding midterm evaluations could be placed there and discussed. All the groups then got together to talk about the different issues and opinions that came up. This process was repeated for both questions.

Summarized findings

What does a successful mid-term evaluation of teaching look like?

Dialogue more important than if any actual changes are made. Direct action plan according to results may have tendency to backfire.
Identify common issues between students and instructors.
Sometimes there are questions on instructor evaluations that are like "gimmes". Useless questions that don't really give an important answer are a waste of space, and need to be further specified in order to get data that actually means something.
Focus not on content but rather on teaching styles - Students are not in the position to judge content, but they can have a say in whether or not they are understanding the content, and if the style of delivery/teaching is effective in helping them grasp the content.
Sufficient resources outside of the class: A student may want access to supplementary resources (resources not explicitly covered in the course) to further understand the material being taught.
Work together to build the course by having time set aside for discussion on the importance of course evaluation - important to set up a culture of evaluation at the start in order to become comfortable when trading feedback.
Evaluations should not just be seen as a means of informing the professor of things that need to be done, but it should also be a means of encouraging awareness among students of the things that should be thought about when evaluating what they are taking from the course.
Timing should be in the middle of the term (according to calendar), not necessarily after the midterm exam.
Important to follow-up on feedback in a concrete, clear, and intentional way

- One participant noted that going through the evaluations with students and focusing on outlining an action plan for change can backfire and yield negative results. Whether or not anything changes is of less concern than the actual professor - student dialogue that takes place.

Put students into focus groups - a quick way of getting the general feel of issues.

Important Themes

Share results back to students
Quick feedback
Culture of dialogue established in advance
Identify challenges across curriculum
Timing is important
Are students relating to understanding content (Relevance)

What questions should you (or should you not) ask on a midterm evaluation?

Kinds of questions that should be asked

Questions must be clear and concise
Was adequate feedback provided for assignments and assessments?
Questions should be open-ended
Were there sufficient in-class resources (ie. professor/TA office hours, access to readings)
How approachable/helpful was the professor/TA?
Have sufficient amounts of additional resources been identified?
Questions that ask about student's academic characteristics (Is this a required course or an elective for you? What year are you?, etc...)
Which learning activities (ex. group vs individual) engage you the most?
Questions should focus not on the content, but on topics related to student learning and effective/non-effective teaching styles.
What can you as a student do to improve your learning + What can the instructor do to support it?
Were standards and expectations communicated in advance for assignments?
How is the pace of the course? What are some areas that we should slow down and focus more on? What are some areas that need to be revisited?
Was the access/use of the learning tools (ex. slides, learning modules, etc...) provided in this class effective?
How did the course workload compare with that of your other classes?
What should I, as the instructor, start, stop and continue doing?

Kinds of questions that shouldn't be asked

Do you like the location venue of the class? This question more important at the end of course, so prof can try to get a better room next year. But when it comes down to it, it's not up to the professor, can't make any promises

- Perhaps use online space to compensate and create a focus on learning, showing students that even though teh venue may not be good, you should/can still learn through alternative, creative means.

Questions that relate specifically to the professor.
Questions that ask students if they liked the professor.
Questions the pre-empt / replicate the formal SEoT items

Faculty Parking Lots (Challenges)

Instructors may not be willing to change course content to accomodate feedback, as the content may have been set at the beginning of the term
Normalizing regular feedback - creating a culture of evaluation
Getting student participation in evaluations
Timing of evaluation - students must have enough experience in the course in order to give good feedback, but doing so too late in the course may cause the evaluation to lose value.
There should be course-to-course statistical analyses

Student Parking Lot (Challenges)

Student concern with giving feedback before end of term - may cause awkwardness if the anonymity is somehow lost.
Degree of anonymity
"Special Issues" left for additional comments box
Students may not be aware of the instructor's other obligations - this may lead to a difference in understanding when it comes compromising

Results

Challenge of asking faculty to do more work/give up classtime to discuss the importance of professor and student communication. Things to be discussed could be the importance of office hours, and the student's majority opinion on the most effective teaching styles. - You may find many little things that you have not been aware of previously.

Discovered that the midterm evaluations, rather than just being an evaluation of teaching, becomes much more complex when we look at the relationship between "teaching" and "student learning". Hence, a question that may potentially reshape our approach to these midterm evaluations is: Is this a midterm evaluation for teaching or student learning?

Combination of both - opportunity for feedback in both directions - Could be for instructors to find out about their course and teaching and how it impacts the promotion of student learning

More about those 2 demands fitting together, rather than each of them on their own

Midterm evaluation could be more about "how do I make student learning successful", rather than "How has my teaching style been effective/not effective so far". In other words, we could take on a more student centric perspective, rather than a instructor centric perspective.

Poster Photos

Note: Poster photos are ordered from left-right. The very center circle of the poster relates to the question "What does a successful midterm evaluation look like?" The next layer out relates to questions that should be asked on a midterm evaluation. The outer-most layer relates to questions that should NOT be asked on a midterm evaluation.

Poster far-left (Faculty parking lot)
Poster left
Poster middle
Poster right
Poster far-right (Student parking lot)