The second meeting of the TA training CoP was held on January 17, 2012. The topic of the discussion was Evaluating TA Training Programs. This page gathers notes and thoughts about this meeting. Thanks to Natasha (PHYS) for recording this meeting.

Evaluating TA Training Programs

UBC TA Training; Community of Practice
Site content
	Main Page
	Overview by Departments
	CoP Meetings
	Site map

Purpose of getting feedback

To improve and inform us
To apply for more money the next year – documenting what we have done, and that we’re having some kind of impact or effect
Effectiveness of our implementations
Improve materials we use to evaluate the program
Get a sense from the TAs what they think might be lacking from the program we’re offering
Identifying needs
Since it changes from year to year
Having evidence that the plan has been implemented
Keeping ourselves accountable

Biology evaluation methods

Ask for feedback at the end of each session – 5 sessions offered – must attend 4 or 5 to get the certificate. Some of the questions include questions about the previous session. A graduate student is hired to go through evaluation forms and summarizes them. Lists different responses and then indicates which ones come up more frequently. Analyzes numerical data. Committee members discuss what changes need to be made for the next year, based on the feedback. This goes into budget applications. E.g. Students requested that it be moved into an all-day session, with a few extra hours so it’s not as rushed. Always include the summary sheet with the annual report, not necessarily with the new application though. Summary statistics: Just averages, and go by the comments. Feedback is often quite positive, but there are a lot of suggestions. Therefore, suggestions are used more often than the quantitative results.

Types of evaluations performed

Program evaluations
Student Evaluations
- Statistics moved all surveys online using the UBC survey tool for student feedback at midterm, but have also tried using Google Docs (benefits: it produces summary statistics for you). Create one form for each TA, then students go and evaluate their TAs. Get the students to fill it out in the lab.
- How to evaluate drop-in/help room TAs?
  - Chemistry use an online system for students to evaluate the TAs. Draw for a gift card to fill out survey.

Methods of carrying out evaluations

Online: Consensus that you only get a small fraction of participants if you post them online or ask them to do it afterwards.
- Chemistry went to online 2 years ago. Your name will go into a draw – and then participation went up.
Pre- and post-surveys:
- Chemistry use a post survey after TAs have had time in class. Pre – survey is before they come to TA training, asking about what’s one question you have about your teaching duties? How comfortable do you feel about creating a marking scheme? Look for shifts pre- to post.
- Psychology give pre-training and post-training surveys (at the end of the workshop), which gives numerical data and space for comments. Catherine’s cleared surveys through ethics in case she want to publish the results. Surveys, which will be posted, provide very fine grained data, such that she can observe changes in fear, levels of support, comfort levels, etc. “Are we doing anything on their emotions?”
- Pre- and post (at the end of the session). Response rate later on crumbles.
Open-ended questions versus quantitative questions:
- Worries that there exists subjectivity of numbers
- Neutral can stay as “No opinion” and keep it out of the data analysis – you don’t need a number there.
Long-term analysis – after TAs have some time in the class:
- BioTAP throw a celebratory event to celebrate the end of the term, and throw in a feedback session then. Or opportunity to voice feedback and concerns.
- Physics program runs regular meetings through course-specific training and give end-of-term surveys then.
- Stats hold a mid-February feedback session (with food) so TAs can voice their opinions.
- Physics had experience with TAs getting resentful about all the feedback -> What’s in it for the TAs? -> CTLT often post the results and send them to the participants, so participants know that the results are going back to them. -> Also show “these are the results we got last year, and this is what we changed.”
Formal program review
- English wants the TA training program get reviewed on a regular basis – the same way departments get reviewed.
TA evaluations by instructors/supervisors
- Chemistry: Provide supervisor with the goals of the training (e.g. using questioning), then supervisor writes how well the TA does that.
  - Union said that TAs need to be evaluated by their supervisors -> If you have a problem with their TA and based on the student evaluations they’re not performing well, with students they have a stake in that, you have to have student and supervisor evaluation. – ie Course Instructor.
- What if the instructor isn’t there?
- Often, if instructors are supposed to visit TAs in class, they just don’t. Could appeal to grad students to ask their instructors to come in to evaluate them. “I’d really like some feedback…”
- Instructors observing TAs in class is part of a union regulation if problems arise with that TA in the future. TA evaluations can be used to flag problem areas, which allow you to provide help to the TA through mentoring-type activities, then allow the instructor to write a report to be submitted to the department.

Need to think about whether there’s a particular question you’re trying to answer to motivate your evaluation efforts.

E.g. want particular questions about diversity and inclusivity?
Is a totally different evaluation than “What skills are the TAs getting out of the program?”

Side notes, thoughts and all

We could look into the PEAR program to get help create more effective evaluations.
If you are interested in the Living Lab to help you with your TA training program you can get in touch with them (or through Joe)
The call for proposals is due March 12th, the next session will discuss this
The next meeting will be sometimes early February