Quality assurance designing quality online course evaluation

Phase Five: Evaluation (click to expand)

Evaluation phase takes place once the course has successfully come to an end for the first time (sometimes this phase might happen while the course is running depending on the urgency of the issues and demands/feedback from students). In order to prepare the course for the second offering, the course development team will meet again to evaluate the course based on the first offering, the student evaluations, peer feedback, instructor's notes and reflection, feedback provided by the course instructor/course author and the team.

5. EVALUATION

• • Collect, review and act on learners feedback
• • Plan and conduct mixed-method evaluations
• • Request peer review
• • Take time to reflect
• • Meet course development members to revisit the course design and content

Note:

For evaluation, consider different assessment strategies including, but not limited to: reflecting on your class (it is recommended to do this from the beginning of the course), documenting students' academic achievements in the course; keeping records of student retention, monitoring faculty and students satisfaction throughout the course, collecting students feedback on the course a few times throughout the course, maintaining the cost effectiveness of the program, measuring students competence using nationwide standard assessments as a comparing base, comparing students performance with the course/program outcomes and having a peer or an academic reviewer to provide you feedback.

Tasks to be completed by the project manager/instructional designer (click to expand)

1. After the first offering of a course, the Project Manager/Instructional Designer may ask to meet with the course author/instructor to debrief how the course went. At this meeting, the team will debrief experiences in terms of learner experiences/evaluations, instructor experiences/concerns and make note of any items that may need modification. It is recommended to make notes of any issues brought up by students in the first offering. Instructor's reflection is also very helpful in evaluation process.
2. Project Manager/Instructional Designer may provide a list of new strategies and changes based on his/her own reflection as well as feedback received.
3. The team will review overall course content/assignments and discuss any student feedback that is available. The instructor will discuss his/her concerns regarding any issues that were brought up by students throughout the course to come up with new strategies and design for the next offering.
4. Along with assessing content issues that may have arisen, technical or delivery issues that may have negatively impacted the course are looked at in order to ensure that they are resolved before the next run of the course. For example, if majority of students had expressed concerns about one of the assignments and its delivery, the assignment and its design will be revisited and might be redesigned.
5. All revisions to be done for the next offering will be discussed and noted upon the available budget and resources. It is recommended that the original course author be the person to make the revisions to the course if possible.

Supporting Quality Assurance Documentation (click to expand)

The online course evaluation links below offer a range of rubrics and guidelines to assist with the development of effective practices for evaluating online courses.

1. Self Review of Online Teaching (http://www.online.pitt.edu/faculty/documents/SelfReviewGuide.pdf)
2. A tool for Faculty Peer Review of Online Teaching - The Sloan Consortium (http://sloanconsortium.org/effective_practices/tool-faculty- peer-review-online-teaching)
3. Peer Review of Teaching - University of British Columbia (http://ctlt.ubc.ca/programs/all-our-programs/peer-review-of-teaching/) and (http://wiki.ubc.ca/Peer_Review_of_Teaching)
4. Reflective Practice - University of British Columbia (http://wiki.ubc.ca/Reflective_Practice_%28Teaching_and_Learning%29)
5. Evaluating Online Courses - Michigan State University (http://fod.msu.edu/oir/evaluating-online-courses)
6. Criteria for Evaluating the Quality of Online Courses - Wright, C. R. (http://elearning.typepad.com/thelearnedman/ID/evaluatingcourses.pdf)
7. Class Observations - University of Central Florida (http://www.fctl.ucf.edu/TeachingAndLearningResources/ClassroomManagement/ClassObservations/)
8. Rubric for Online Instruction (ROI) - Chico State University (http://www.csuchico.edu/roi/
9. Faculty Peer Review of Online Teaching- Penn State University (http://facdev.e-education.psu.edu/evaluate-revise/peerreviewonline)
10. A Peer Review Guide for Hybrid Courses at Penn State (https://www.e-education.psu.edu/files/files/PeerReview_HybridCourses_PSU_Guide_31July2012_form.pdf)
11. Gathering Feedback from Students (Vanderbilt University)
12. Midterm Feedback Evaluation (Carleton University)
13. Teaching Evaluation (NYU Stern)
14. Conducting a Midterm Evaluation (University of California, Berkeley)