MET:Building Quality Assurance in Online Learning Assessment

From UBC Wiki

This page originally authored by Sophia Barton-Bucknor (February 2012).

Theory of Online Learning

File:Terry Anderson's Theory of Online Learning Diagram.jpg
alt text

Terry Anderson's (2008) model of online learning depicts the two major modes of online learning: a) collaborative and community-of-inquiry; and b) independent study within which effective learning occurs. Here effective learning can be examined within an overlapping framework of four optics: learner-centred, community centred, knowledge centred, and assessment centred (Anderson, 2008, citing Bransford, Brown and Cocking, 1999). While assessment is the key figure within the assessment centred optic, it plays a supporting cast role within the other optics to varying degrees.

Why Assess Online Learning

Learning is increasingly being situated online (e-learning) due to a combination of anytime, anywhere convenience (Palloff & Pratt, 2009) and various Web 2.0 constructivist affordances.In their Community of Inquiry model, Garrison, Anderson, and Archer (2000) posit that effective online learning [educational community] includes three critical presences: cognitive, social, and teaching. Anderson (2008) identifies assessment as key to activating teaching presence within the learning context. Assessment refers to the process of measuring against stated learning objectives, outcomes and competencies. Jenkins (2004) states that "assessment is one of the most powerful drivers ... as it defines the goals of both learners and teachers." Carnevale (2001) confirms that assessment is "taking centre stage as online educators experiment with new ways of proving that they are teaching effectively" and look for new ways to use assessment activities to motivate and engage learners. Learners continue to rely on assessment activities for feedback on understanding and achievement as well as to "pace and guide their learning activities" (Jenkins, 2004). Assessment is often categorized as formative or summative. Formative assessment refers to the ongoing or continuous assessment activities and is often seen as assessment for learning and as learning. Summative assessment refers to the final or capstone assessment decision; consideration is given to formative assessment findings.

Approaches to Quality Assurance in Online Learning Assessment

File:Developing a Holistic Approach for E-Learning Diagram.jpg
alt text
File:Comprehensive Approach to Program Evaluation in Open and Distributed Learning.png
alt text
File:Sloan Consortium 5 Pillars of Quality.jpg
alt text

Wikipedia defines quality assurance (QA) as a systematic measurement that involves comparison with a standard, monitoring of processes and an associated feedback loop that is used to continuously improve performance. QA models are often based on well-established quality assurance principles: documented policies and systematic procedures for ensuring compliance with the procedures (Phipps & Swift, 2004). Wang (2004) and Suzuki and Tada (2009) highlight the necessity of assuring and improving the quality of online or e-learning. Assuring quality for online learning assessment is often approached, situated and examined within program evaluation; assessment is one of the considerations embedded in such a model.

There is no widely accepted principles or guidelines for assuring quality e-learning and assessment, though many have been posited (Suzuki and Tada, 2009) by accrediting and regulatory bodies such as the Canadian Community Association For Community Education (CACE) and the Council for Higher Education Accreditation (CHEA) - see CHEA's Monograph Series 2002, Number 1 (May 2002). Other models and approaches include:

Comprehensive Approach to Program Evaluation in Open and Distributed Learning - CAPEODL Model

Badrul Khan (2007) discussion of Program Evaluation in E-Learning posits one such model -- Comprehensive Approach to Program Evaluation in Open and Distributed Learning (CAPEODL). Khan’s framework examines e-learning against the following eight dimensions of open and distributed learning environments: pedagogical, technological, evaluation, management, resource support, ethical, and institutional. Evaluation within this framework refers to both assessment of learners and the instruction and learning environment.

5 Pillars of Quality Approach - Sloan Consortium

The Sloan Consortium identifies 5 pillars of quality against which e-learning programs can be examined: learning effectiveness, access, student satisfaction, faculty satisfaction, and cost-effectiveness (Wang, 2006). Assessment is considered in learning effectiveness to determine that direct assessment of student learning is equivalent or better.

Holistic Quality Assurance Framework - Kelly, Phipps and Swift

Kelly, B., Phipps, L. and Swift, E. (2004) propose a holistic quality assurance framework that encompasses learner needs, accessibility, usability, local factors, infrastructure, and learning outcomes. Here, assessment is part of the discourse for learning outcomes.

A Five-E Model Approach - Suzuki and Tada

Suziki and Tada (2009) highlight that due to the multi-facet nature of online learning, the discourse around quality issues of online learning has been done separately. Recognizing the need for a cohesive framework premised on various models, they locate their discussion and approach in A Five-E Model discourse based on: environment design, content design, information design, motivation design, and learning design.

Application of the Principles of Quality Assurance in the Online Learning Environment

While it is important to situate e-learning evaluation within a holistic framework within which assessment is one component, it is also important to examine assessment by itself because of its importance. The difficulty lies with assessment not being mutually exclusive; it is not in a neutral zone. It is both impacted by the multi-faceted integration and interrelationships of the principles of quality within the context of learning as well as being an outcome measure of the process. However, quality assurance in online learning assessment should not be prescriptive no matter the model or approach used. Understanding and leveraging the online context should result in assessment methods designed around the constructivist Learningaffordances of Learn 2.0 and Web 2.0 within which learning and assessment may occur. Quality assurance features premised on principles of quality (reoccurring within many approaches such as the CAPEODL and 5 pillars of qualitymodels: effectiveness, accessibility, appropriateness, adaptability, efficiency, security, timeliness, collaboration and authenticity should be integrated into the online learning and assessment.

Anderson (2004) in Teaching in an Online Learning Context argues that timely, detailed formative feedback -- whether from objective methods such as multiple-choice questions or through more subjective type assessment of blogs, wikis, or journal entries -- provided as near in time as possible to the performance of the assessed behaviour is most effective in motivating and shaping behavior and mental constructs. One concern with assessment of online learning is the possibility of cheating. A focus on formative and not summative assessment mitigates this issue as well an improve authenticity of the process. Summative assessment such as of e-portfolios and project-based activities may be just as effective in assuring quality, especially where formative assessment informs summative assessment decisions.

Objective assessments, such as those provided in online multiple-choice test and true and false questions that can be quickly marked by a machine with instantaneous feedback provided to learners may be very effective learning devices (Anderson, 2008, citing Prensky, 2000); however, more subjective methods and direct communication and feedback from teacher to the student (Anderson, citing Laurillard, 1997) may increase learning effectiveness and deepens collaboration or interaction.

Cuter (citing Wiggins, 1993) defines authentic assessments as those embed assessment in real-world contexts; tasks and procedures in which students are engaged in applying skills and knowledge to solve “realworld” problems, giving the tasks a sense of authenticity. Assessment activities should be developed based on the cultural and technological context of online learning along with the educational interactions dynamic of the teacher-learner, learner-learner, and learner-content (Anderson, 2008) premised on principles our stated dimensions of quality.

Best Practices

Rochester Institute of Technology's Best Practices for Assessing Student Learning Outcomes in an Online Environment: https://www.youtube.com/watch?v=h4NKDG6NFQY

Challenges

Web technologies afford imaginative and meaningful assessment premised of many principles of quality identified in various models and frameworks. Jenkins (2004). Self, peer, group-based, teacher within a combination of subjective and objective formative and summative assessments are possible. However, teachers may view quality assurance models and guidelines as prescriptive. There are some aspect of 'prescription' that usually comes with attempts to build and improve systems.

As well, there is the potential for increase in workload (Anderson, 2008), especially during initial implementation stages. Subjective assessments are time consuming to review and discuss. However, this may be lessened through the use of student guided or led assessment objectives and subjective assessment. Other challenges come with the technological dependent nature of the learning context. Technological and human resources capacity would need to be developed or built through training and or purchase that would impact principles of quality such as accessibility and support. A lack of these resources may negatively impact on the quality of assessment.

References

  • Barker, K. (2012, February 25). “Creating Quality Guidelines for On-line Education and Training.” Community Association for Community Education. Available online at http://futured.com/form/pdf/english.pdf
  • Lankshear, C., & Knobel, M. (2008). The “twoness” of learn 2.0: Challenges and prospects of a would-be new learning paradigm. Closing keynote presented at the Learning 2.0: From Preschool to Beyond, Montclair State University, Montclair, NJ. Available online at http://everydayliteracies.net/Montclair_Talk
  • Palloff, R. & Pratt, K. (2009). Assessing the Online Learner. San Francisco, CA: Jossey-Bass

External Links

Retrieved from "https://wiki.ubc.ca/index.php?title=MET:Building_Quality_Assurance_in_Online_Learning_Assessment&oldid=697759"