MET:Cognitive Tutors

Authored by Karon Wong (Winter 2012)

Cognitive Tutors (CT) is an AI-based system that uses real-time to guide students through a math problem (Anderson, 1995). CT is a software that has the capability to observe and watch students solve problems while they are on the computer (Anderson, 1995). CT distinguishes its uniqueness by integrating a production-rule model into the tutor. The model generates different correct solution paths to recognize and connect the students’ work when they follow one of the paths. CT allows students to work independently without any interruptions if they are in the correct path. However, if students get off-path, the instructions offered by the tutor will get the students back on path. During feedback and help, CT offers two kinds of instructions. One instruction is designed to prompt the student with a message and an explanation when an error is made. Another instruction with a message to guide the student into the correct path is activated only when the student asks for help (Anderson et al. 1995). The feedback which is offered to students at the end of a problem eliminates time loss in a traditional textbook chapter method while students must wait to receive feedback when the teacher finishes marking the solutions. With its higher interactivity level, CT provides step-to-step in-depth and explicit feedback instead of having the student to wait until he gets to the final solution to the problem (Koedinger & Aleven, 2007). The interactive instructional loops offer step-by-step solution-specific hints. As stated by Lepper and Malone (1987) (as cited in Koedinger & Aleven, 2007) CT is high in interactive level and works similarly to a human tutor in supporting students.

File:CT.jpg

History

In 1984, some high school students were put through a geometry tutor and computer science mini-course at Carnegie Melon University with the LISP Tutor. The pilot project was well received with a positive response. It was found that students had learned quite well with the geometry tutor. For evaluation purpose, the LISP mini-course was separated into two groups. One group used the LISP tutor to do the exercises while the other group used a standard LISP environment. The group with the aid of LISP tutor finished the exercise 30% faster and scored higher than the control group. The experience showed that the metacognitive strategy was an effective one where students were able to learn by doing and explaining with a computer-based Cognitive Tutor (Anderson, Corbett, Koedinger & Pelletier, 1995).

Wenger (1987) (as cited in Aleven & Koedinger, 2002) stated that CT is an intelligent tutoring system with an educational goal to provide guided learning to high school and college students who have difficulties in mathematics. The CT software is catered to help individual students who exhibit continuous difficulty in performing satisfactorily in mathematics. CT contributes to a mastery learning criterion (Anderson, Corbett, Koedinger, & Pelletier, 1995) by monitoring each step a student makes towards arriving at the solution, prompts the student with feedbacks and hints whenever the student makes an erroneous attempt. More specifically, each CT is a tutoring building which is based on a cognitive model (Ritter, 2007) to predict what students can or cannot do as it interprets the student’s behaviour by correctly representing his knowledge. The basic production rule of the CT system analyzes the student’s problem-solving technique and uses “knowledge tracing” (Corbett & Anderson, 1995) to derive results from the analysis to monitor the student’s mastery of each crucial production rule. CT tests the principal beliefs of the "ACT-R theory"(Adaptive Control of Thought-Rational Theory) and manifests in various programs, such as the Pittsburgh Urban Mathematics Projects (PUMP) Algebra Tutor also known as PAT and the LISP Tutor (Koedinger & Aleven, 2007). With the system’s strong interactive and theoretical base, its on-demand hints, on-line glossary for definitions and terms, statements and the use of illustrated examples in the definitions of important theorem, CT specializes in helping students in algebra and geometry (Koedinger & Anderson, 1997).

File:CT2.jpg

The Goals of Cognitive Tutors

Like the ACT (Adaptive Control of Thought) theories, CT aims to promote procedural-declarative distinction, knowledge compilation and strengthening of both declarative and procedural knowledge. While declarative knowledge is “knowing that” something is (Woolfolk, Winne, & Perry, 2006), the system provides verbal information to students by displaying visual hints and prompts. The procedural knowledge of “knowing how” to perform a task (Woolfolk et al. 2006) is improved as students convert knowledge into production rules through practising the necessary skill. According to the theory, it is believed that utilizing declarative knowledge in a problem-solving activity will be the only way in learning production rules. It also targets to empower students the ability to interpret various procedures, including following and comparing instruction, generating problem-solving behaviour through connecting declarative knowledge and task goals (Anderson et al. 1995). The conversion of interpretive problem solving to production rules thereby enables CT to foster knowledge compilation. Through practice, both declarative and procedural knowledge can be strengthened. Having knowledge successfully encoded and practising of learned knowledge; students will be able to create faster, more accurate and smoother execution (Anderson et al. 1995).

Principles

CT is formed by model tracing and knowledge tracing. CT traces each step a student makes and follows through all the strategies that the student uses in the problem-solving scenario (Anderson et al. 1995). The model tracing becomes most effective when it corrects students by providing an error message whenever they make a mistake. The message will prompt on the error the student has incurred in the choice of strategy or in a particular reasoning step. Knowledge tracing, on the other hand, determines the selection of problems in a curriculum based on a student’s needs. When the system recognizes that a student is showing proficiency in mastering the skill of a certain level, it will allow the student to proceed in challenging a different curriculum (Koedinger & Aleven, 2007). Procedural knowledge is goal-oriented and relies on practice. It seldom creates awareness. Declarative knowledge, on the contrary, is more dependent visually and verbally and can be used for specific goals. It usually consists of verbal and visual knowledge that is linked through perception, instruction or reading (Anderson et al. 1995). The basis of CT is from the ACT-R theory of cognition. It tests the key principles that complex cognitive domain can be understood by small independent knowledge called protection rules. CT shows prominent advantages compared to common learning environment that do not use computer tutors (Koedinger & Aleven, 2007).

Represent student competence as a production set: CT differs from other models that operate on a more behaviourist approach to a computer-based instruction by its flexibility in setting unique curriculum objectives and its ability to analyze the performance of the students on a more abstract and accurate level. CT sets appropriate curriculum objectives based on the system’s interpretation of the student’s ability by decomposing the student’s skill into different components to carry out componential analysis (Anderson et al., 1995).

Communicate the goal structure underlying the problem solving: One of the instructional objectives of CT is on exposing and communicating goals and sub-goals (Anderson et al. 1995). CT recognizes the need to communicate efficiently with the students to enhance their problem-solving skill. In the algebra tutor, for example, the system communicates its goals to the student by displaying various topics that a student can attempt, such as Patterns and Multiple Representation or Solving Linear Equations. The system also displays sub-goals as its objectives such as prompting the student to write and use a two-step equation and a graph to establish solution of equation (Carnegie Learning Company).

Provide instruction in the problem-solving context: One challenge that CT faces is the process of providing instructions between sections in the tutor. Although the provision of instructions in the problem-solving context was structured to allow an opportunity for the students to refer back to the instructions, some students find it interfering with their problem-solving (Anderson et al. 1995).

Promote an abstract understanding of the problem solving knowledge: While students often exhibit over concerns on specific knowledge in problem-solving, CT allows students to use detailed theoretical interpretation to apply in problem-solving. To help students to navigate, the system’s glossary provides a list of theorems and their explanations for students to use in order to reach the desired solution (Anderson et al. 1995).

Minimize working memory load: Derived from Sweller, CT minimizes irrelevant information to the target production to prevent interference that would cause a high working memory load (Anderson et al. 1995).

Provide immediate feedback on errors: Students using CT receive the benefits of immediate feedback and coaching. The "Just-in-time" help, "On-demand" help, and positive reinforcement minimizes the time gap between the time a student makes an error and the time the error is corrected (Anderson et al. 1995). It is important because the longer it takes for the student to correct from his error, the longer the student will need to solve the problem. CT’s spontaneous feedback efficiently gives the student a sense of direction and prevents the student to waste his current efforts that are not goal-oriented (Woolfolk et al. 2006).

Facilitate successive approximations to the target skill: The more guided practice a student gets, the better chance he will succeed in solving the problem. When a tutor hints and provides necessary and spontaneous guidance to the student when he does the practice, the student will be more able to succeed in performing a particular skill. Given the proper help and sufficient practice, the student will be able to work independently in time without the guidance of the tutor (Anderson et al. 1995).

Real-world situations: CT allows students to be engaged in real-world situations while they develop the skill in problem solving, communication and reasoning. Real-world situations are incorporated in problems which encourage students to make connections using multiple representations (Anderson et al. 1995). In the PUMP curriculum, for example, real world situations are integrated to make materials more accessible to students. Students benefit from experiencing mathematics used in everyday life. This is consistent to the recommendations of the National Council of Teachers of Mathematics (NCTM, 1989) which recommends computer utilities and mathematics communications (Koedinger & Anderson, 1997).

Comparisons

CT is different from "LOGO", a tool which follows a constructivist approach to help students in constructing knowledge. The LOGO tradition acts as a tool that motivates students in learning and has been seen as a useful medium in teaching geometry. Although both CT and LOGO are student-centered, Schofield, Evans-Rodes, and Hubber (1990) found that in with the implementation of LOGO classrooms are more student centered with teachers taking a greater facilitator role in supporting students as-needed on particular challenges. CT allows students to be in more control. Wertheimer (1990) highlighted that Cognitive Tutor effectively engages students and it gives the facilitator more free time to provide individualized assistance to those who needed the help. On the other hand, CT offers feedback and hints that LOGO does not provide. Telementoring has often been confused with CT because they both share common features such as an ongoing relationship between a student and the telementor as an expert. Telementoring often mentors interactions revolving around problems that the student brings up. O’Neill and Harris’s article explains why telementoring is important – it “facilitates students’ best thinking by problematizing their work.” CT, on the other hand, assigns students with problem sets selected by the tutor. Students will work towards solving the problem with the assistance of real time hints and guidance. The objective of CT is for students to master a precise domain such as geometry or algebra (O’Neill & Harris, 2005).

The Objective and Application of Cognitive Tutors

CT offers a dynamic problem-solving environment to enhance reasoning and problem- solving ability. For example, the Algebra Cognitive Tutor provides different types and levels of help to students, including help on solving problems on a worksheet, or a symbolic equation solver and a grapher (Stylianou & Shapiro, 2002). To help students create appropriate analyzes, CT provides multiple levels of hints, including a spontaneous feedback on why a student has entered a wrong step, a hint on useful problem-solving principle, and a bottom-out hint as the last hint to provide the student with the answer or the next step (Koedinger & Aleven, 2007). The Geometry Cognitive Tutor, for example, supports self-explanation. In a problem scenario when students have to compute angle measures, a student would have to explain his answers step by step, such as entering the name of the geometry theorem or definition that he has chosen. The model also provides a glossary with geometry knowledge containing theorems and definitions in it, where students can use in order to solve the problem. For example, when a student entered “triangle sum” in his explanation to solve the problem which he needs to compute the degree of the angle, the model will generate a feedback indicating that the explanation is wrong. A hint from the tutor upon the student’s request will reveal the correct reason (Koedinger & Aleven, 2007). In addition, the self-explanation feature of CT works towards fostering better understanding while students learn. The feedback of the tutor helps students come up with better explanations and encourages the student to make step-by-step correct explanations. The tutor feedback encourages students to think systematically and allow them to analyze what their steps are through self-explanation. The explanations of students signify problem-solving principles as students could often explain the steps by relating them with a problem-solving principle. Hence, students benefit from acquiring the ability to explain things in their own words (Koedinger & Aleven, 2007). Through explanation, students develop the skill to compare their explanations to what they already know. In the self-explanation process, students will be able to build and strengthen their existing knowledge (Koedinger & Aleven, 2007). When production rules are successfully applied during problem-solving, learning will occur. There is no learning occurrence if students are permitted to spend extended time to follow through an incorrect path. Hence, the production rule of CT allows students to be focused in learning through its immediate feedback. It encourages students to find a good balance between withholding explanatory information and the condition. It is believed that students develop better judgement when they benefit from the more explicit information rather than relying on the feedback of the tutor. Hence, CT shows major advantages compared to common learning environments (Koedinger & Aleven, 2007).

Evaluating Cognitive Tutors

CT’s yes/no feedback offers effective help to students. The yes/no feedback is given when the student makes steps to solve a problem. Should the student’s attempt be inappropriate, CT immediately offers an effort feedback message. Detailed-solution specific hints are also provided to the student for each step taken towards the solution. Although CT may seem to be heavy in providing information for the student, it actually withholds a considerate amount of information (Aleven & Koedinger, 2002). First of all, the system presents problems rather than worked-out examples. In this way, students take their responsibility to generate the solution steps. Secondly, depending on settings, a student can request for hints instead of receive hints at the tutor’s initiative. When a student requests for further help, the information in the hint sequences slowly reveals itself with subsequent hint levels being displayed. As such, CT deliberately forms the equilibrium between the benefits of information giving and withholding (Aleven & Koedinger, 2002). The Geometry Proof Tutor that is used in classrooms has shown great learning gains. Students who worked with this tutor scored about one standard deviation better than other students who were taught by the same teacher but did not use the tutor. Tutors are more efficient in supporting individual learning than collaborative learning. Anderson et al. (1995) stated (as cited in Koedinger & Aleven, 2007) the LISP tutor, for example, indicted that the learning gains were 30-43% higher and 30-64% more efficient in learning in comparison of working with the standard LISP programming environment. In Pittsburgh and Milwaukee, students in the curriculum of Algebra Cognitive Tutor and the standard algebra were compared. Students from the Cognitive Tutor curriculum scored 15-25% higher on items taken from standardized texts and 50-100% higher on test items based on problem solving and the use of representation. Finally, students in middle school math classes using the CT curricula learned better than students using other curricula. (Aleven & Koedinger, 2002)

See Also

• Intelligent Tutor System

• Cognitive Model

• Logo

References

• Aleven, A.W.M.M., & Koedinger, K.R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science. 26, 147-179.

• Anderson, J.R., Corbett, A.T., Koedinger, K.R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4(2), 167-207.

• Carnegie Learning: The Cognitive Tutor Company Web site. Retrieved July 9, 2008, from http://www.carnegielearning.com/products.cfm

• Koedinger, K.R. (1998).Intelligent Cognitive Tutors as Modeling Tool and Instructional Model.

• Koedinger, K.R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Education Psychology Review. 19, 239-264.

• Koedinger, K.R., & Anderson, J.R. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education. 8, 30-43.

• Lehrer, R., Randle, L., & Sancilio., L (1989). Learning preproof geometry with LOGO. Cognition and Instruction. 6(2), 159-184.

• O'Neill, K., & Harris, J.B. (2005). Bridging the perspectives and developmental needs of all participants in curriculum-based telementoring programs. Journal of Research on Technology in Education.

• Ritter, S., Anderson, J.R., Koedinger, K.R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249 - 255.

• Stylianou, D.A., & Shapiro, L. (2002). Revitalizing algebra: the effect of the use of a cognitive tutor in a remedial course. Journal of Education Media. 27.

• Woolfolk, A., Winne, P., & Perry, N. (2006). Third Canadian Edition of Educational Psychology. Toronto: Pearson.