MET:Building Quality Assurance in Online Learning Assessment
This page originally authored by Sophia Barton-Bucknor (February 2012).
Theory of Online Learning
Terry Anderson's (2008) model of online learning depicts the two major modes of online learning: a) collaborative and community-of-inquiry; and b) independent study within which effective learning occurs. Here effective learning can be examined within an overlapping framework of four optics: learner-centred, community centred, knowledge centred, and assessment centred (Anderson, 2008, citing Bransford, Brown and Cocking, 1999). While assessment is the key figure within the assessment centred optic, it plays a supporting cast role within the other optics to varying degrees.
Why Assess Online Learning
Learning is increasingly being situated online (e-learning) due to a combination of anytime, anywhere convenience (Palloff & Pratt, 2009) and various Web 2.0 constructivist affordances.In their Community of Inquiry model, Garrison, Anderson, and Archer (2000) posit that effective online learning [educational community] includes three critical presences: cognitive, social, and teaching. Anderson (2008) identifies assessment as key to activating teaching presence within the learning context. Assessment refers to the process of measuring against stated learning objectives, outcomes and competencies. Jenkins (2004) states that "assessment is one of the most powerful drivers ... as it defines the goals of both learners and teachers." Carnevale (2001) confirms that assessment is "taking centre stage as online educators experiment with new ways of proving that they are teaching effectively" and look for new ways to use assessment activities to motivate and engage learners. Learners continue to rely on assessment activities for feedback on understanding and achievement as well as to "pace and guide their learning activities" (Jenkins, 2004). Assessment is often categorized as formative or summative. Formative assessment refers to the ongoing or continuous assessment activities and is often seen as assessment for learning and as learning. Summative assessment refers to the final or capstone assessment decision; consideration is given to formative assessment findings.
Approaches to Quality Assurance in Online Learning Assessment
Wikipedia defines quality assurance (QA) as a systematic measurement that involves comparison with a standard, monitoring of processes and an associated feedback loop that is used to continuously improve performance. QA models are often based on well-established quality assurance principles: documented policies and systematic procedures for ensuring compliance with the procedures (Phipps & Swift, 2004). Wang (2004) and Suzuki and Tada (2009) highlight the necessity of assuring and improving the quality of online or e-learning. Assuring quality for online learning assessment is often approached, situated and examined within program evaluation; assessment is one of the considerations embedded in such a model.
There is no widely accepted principles or guidelines for assuring quality e-learning and assessment, though many have been posited (Suzuki and Tada, 2009) by accrediting and regulatory bodies such as the Canadian Community Association For Community Education (CACE) and the Council for Higher Education Accreditation (CHEA) - see CHEA's Monograph Series 2002, Number 1 (May 2002). Other models and approaches include:
Comprehensive Approach to Program Evaluation in Open and Distributed Learning - CAPEODL Model
Badrul Khan (2007) discussion of Program Evaluation in E-Learning posits one such model -- Comprehensive Approach to Program Evaluation in Open and Distributed Learning (CAPEODL). Khan’s framework examines e-learning against the following eight dimensions of open and distributed learning environments: pedagogical, technological, evaluation, management, resource support, ethical, and institutional. Evaluation within this framework refers to both assessment of learners and the instruction and learning environment.
5 Pillars of Quality Approach - Sloan Consortium
The Sloan Consortium identifies 5 pillars of quality against which e-learning programs can be examined: learning effectiveness, access, student satisfaction, faculty satisfaction, and cost-effectiveness (Wang, 2006). Assessment is considered in learning effectiveness to determine that direct assessment of student learning is equivalent or better.
Holistic Quality Assurance Framework - Kelly, Phipps and Swift
Kelly, B., Phipps, L. and Swift, E. (2004) propose a holistic quality assurance framework that encompasses learner needs, accessibility, usability, local factors, infrastructure, and learning outcomes. Here, assessment is part of the discourse for learning outcomes.
A Five-E Model Approach - Suzuki and Tada
Suziki and Tada (2009) highlight that due to the multi-facet nature of online learning, the discourse around quality issues of online learning has been done separately. Recognizing the need for a cohesive framework premised on various models, they locate their discussion and approach in A Five-E Model discourse based on: environment design, content design, information design, motivation design, and learning design.
Application of the Principles of Quality Assurance in the Online Learning Environment
While it is important to situate e-learning evaluation within a holistic framework within which assessment is one component, it is also important to examine assessment by itself because of its importance. The difficulty lies with assessment not being mutually exclusive; it is not in a neutral zone. It is both impacted by the multi-faceted integration and interrelationships of the principles of quality within the context of learning as well as being an outcome measure of the process. However, quality assurance in online learning assessment should not be prescriptive no matter the model or approach used. Understanding and leveraging the online context should result in assessment methods designed around the constructivist Learningaffordances of Learn 2.0 and Web 2.0 within which learning and assessment may occur. Quality assurance features premised on principles of quality (reoccurring within many approaches such as the CAPEODL and 5 pillars of qualitymodels: effectiveness, accessibility, appropriateness, adaptability, efficiency, security, timeliness, collaboration and authenticity should be integrated into the online learning and assessment.
Anderson (2004) in Teaching in an Online Learning Context argues that timely, detailed formative feedback -- whether from objective methods such as multiple-choice questions or through more subjective type assessment of blogs, wikis, or journal entries -- provided as near in time as possible to the performance of the assessed behaviour is most effective in motivating and shaping behavior and mental constructs. One concern with assessment of online learning is the possibility of cheating. A focus on formative and not summative assessment mitigates this issue as well an improve authenticity of the process. Summative assessment such as of e-portfolios and project-based activities may be just as effective in assuring quality, especially where formative assessment informs summative assessment decisions.
Objective assessments, such as those provided in online multiple-choice test and true and false questions that can be quickly marked by a machine with instantaneous feedback provided to learners may be very effective learning devices (Anderson, 2008, citing Prensky, 2000); however, more subjective methods and direct communication and feedback from teacher to the student (Anderson, citing Laurillard, 1997) may increase learning effectiveness and deepens collaboration or interaction.
Cuter (citing Wiggins, 1993) defines authentic assessments as those embed assessment in real-world contexts; tasks and procedures in which students are engaged in applying skills and knowledge to solve “realworld” problems, giving the tasks a sense of authenticity. Assessment activities should be developed based on the cultural and technological context of online learning along with the educational interactions dynamic of the teacher-learner, learner-learner, and learner-content (Anderson, 2008) premised on principles our stated dimensions of quality.
Rochester Institute of Technology's Best Practices for Assessing Student Learning Outcomes in an Online Environment: https://www.youtube.com/watch?v=h4NKDG6NFQY
Web technologies afford imaginative and meaningful assessment premised of many principles of quality identified in various models and frameworks. Jenkins (2004). Self, peer, group-based, teacher within a combination of subjective and objective formative and summative assessments are possible. However, teachers may view quality assurance models and guidelines as prescriptive. There are some aspect of 'prescription' that usually comes with attempts to build and improve systems.
As well, there is the potential for increase in workload (Anderson, 2008), especially during initial implementation stages. Subjective assessments are time consuming to review and discuss. However, this may be lessened through the use of student guided or led assessment objectives and subjective assessment. Other challenges come with the technological dependent nature of the learning context. Technological and human resources capacity would need to be developed or built through training and or purchase that would impact principles of quality such as accessibility and support. A lack of these resources may negatively impact on the quality of assessment.
- Anderson, T. (2008). Toward a theory of online learning. In T. Anderson & F. Elloumi (Eds.) Theory and Practice of Online Learning, Chapter 2 (pp. 45-74). Available online at: http://cde.athabascau.ca/online_book/pdf/TPOL_chp02.pdf
- Anderson, T. (2004). “Teaching in an Online Learning Context.” IIn: Anderson, T. & Elloumi, F. Theory and Practice of Online Learning. Athabasca University. Available online at: http://cde.athabascau.ca/online_book/ch11.html#fig1
- Barker, K. (2012, February 25). “Creating Quality Guidelines for On-line Education and Training.” Community Association for Community Education. Available online at http://futured.com/form/pdf/english.pdf
- Custer, R. “Authentic Assessment.” Retrieved 2012, February 28 from http://www.calpro-online.org/eric/docs/custer/custer2.pdf
- Jenkins, M. (2004). “Unfulfilled Promise: formative assessment using computer-aided assessment.” Learning and Teaching in Higher Education. 1, 67-78. Available online at http://www2.glos.ac.uk/offload/tli/lets/lathe/issue1/articles/jenkins.pdf
- Khan, B. H. (2007). “Program Evaluation in E-Learning.” Avalailable online at http://asianvu.com/bk/elearning_evaluation_article.pdf
- Lankshear, C., & Knobel, M. (2008). The “twoness” of learn 2.0: Challenges and prospects of a would-be new learning paradigm. Closing keynote presented at the Learning 2.0: From Preschool to Beyond, Montclair State University, Montclair, NJ. Available online at http://everydayliteracies.net/Montclair_Talk
- Palloff, R. & Pratt, K. (2009). Assessing the Online Learner. San Francisco, CA: Jossey-Bass
- Rochester Institute of Technology, Office of Educational Effectiveness Assessment. (2017). Assessing Student Learning Outcomes in an Online Environment. Available online at: https://www.rit.edu/academicaffairs/outcomes/online-student-learning-outcomes-assessment
- Kelly, B., Phipps, L. & Swift, E. (2004). “Developing A Holistic Approach For E-Learning Accessibility.” Canadian Journal of Learning and Technology, 30 (3). Available online at http://opus.bath.ac.uk/425/1/accessibility-elearning-paper.pdf
- Postsecondary Education Quality Assessment Board. (2011). “Quality Assessment Panel Report Guidelines.” Available online at http://www.peqab.ca/Publications/QAGuidesOnlineCapacityReviewWeb.pdf
- Wang, Q. (2006). “Quality assurance – best practices for assessing online programs”. International Journal on E-Learning. Available online at http://www.thefreelibrary.com/Quality+assurance--best+practices+for+assessing+online+programs.-a0143618063
- Community of Inquiry Wikipedia Entry
- E-learning Wikipedia Entry
- Sloan_Consortium Wikipedia Entry
- Formative assessment Wikipedia Entry
- Summative assessment Wikipedia Entry