MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT

The Effect of Mastery Learning on Student Achievement and Engagement Kevin Moroney University of New England

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Abstract

To determine whether historical claims about the efficacy of a Mastery Learning instructional model in increasing academic performance and student engagement were valid, the author conducted an 8-week investigation in two 9th grade special education math classes. The subjects were 27 randomly-assigned students functioning an average of five years below grade level in math. Three curriculum units were taught using the Mastery Learning approach of instruction-assessment-reinstruction-reassessment to evaluate the impact of this pedagogical method on mathematical reasoning, computational accuracy, and active involvement in the learning process. Units were broken down into component skills and sequenced for increasing complexity. The minimum standard for mastery of each skill was raised from 65% to 80% to promote higher achievement. Student behavior was monitored for both active participation and avoidance during instruction. The findings showed a significant increase in students responsiveness to in-class and at-home problems, with a corresponding decrease in avoidance behaviors during class. While overall accuracy in both comprehension and calculation increased dramatically from baseline to final testing for the 24 students who completed the 8-week program, interim proficiency levels fluctuated in accordance with the complexity of the subject matter in the individual units. The time needed to effectively implement Mastery Learning protocols reduced the volume of material covered during the investigational period. The researcher concluded that, while a Mastery Learning approach produces beneficial academic outcomes, multiple factors influence its effectiveness, including available instructional time, appropriate staffing ratios, and the abilities or disabilities of the students.

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Table of Contents

Introduction

.. 4

Review of Literature . 12 Methodology .. 25 Results Conclusions Action Plan References .. 31 .. 39 .. 40 .. 43

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Thirty years ago, President Ronald Reagan proposed abolishing the U.S. Department of Education (USDOE) (Reagan 1982). His rationale for this shocking plan was that the Federal government was constitutionally barred from interfering with the rights of the various states to educate their children as they saw fit (Sunderman 2009). Fifteen years later, this proposal was an official plank in the Republican Partys platform: The Federal government has no constitutional authority to be involved in school curricula This is why we will abolish the Department of Education [and] end federal meddling in our schools (DeRugy & Gryphon 2004). Yet the opposite has happened. The USDOE has not only remained a part of the federal government but its influence has grown dramatically in the decades since it was targeted for demolition (DeRugy & Gryphon 2004; Sunderman 2009). Every presidential administration since Reagans has used USDOE funding schemes to control State education policies, ostensibly to improve student achievement across the country. Unfortunately, every effort so far has failed. In 1994, the U.S. Congress enacted Goals 2000: Educate America Act, which had as its stated intent: To improve learning and teaching by providing a national framework for education reform; to promote the research, consensus building, and systemic changes needed to ensure equitable educational opportunities and high levels of educational achievement for all students; to provide a framework for reauthorization of all Federal education programs; to promote the development and adoption of a voluntary national system of skill standards and certifications; and for other purposes. (U.S. House of Representatives 1993). Among its many objectives, this legislation proposed that: By the year 2000, all students will leave grades 4, 8, and 12 having demonstrated competency over challenging subject matter including English, mathematics, science, foreign languages, civics and government, economics, arts, history, and geography, and every school in America will ensure that all students learn to use their minds well, so they may be prepared for responsible citizenship, further learning, and productive employment in our Nation's modern economy. (HR 1804 102(3)(A))

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT

The year 2000 came and went with this goal still unmet. The new millennium gave rise to an improved federal education initiative, the No Child Left Behind Act (NCLB), which expanded Washingtons influence over State education policies and standards by requiring all schools receiving federal funds to meet annual progress goals (Sunderman 2009). Schools that failed to achieve adequate yearly progress would be subject to sanctions, including changes of school personnel, restructuring of the schools administration, transfer of its students, State takeover, external expert control, or outright closure (New York City Department of Education [NYCDOE] 2010a; New York State Dept. of Educ. [NYSED] 2011; USDOE 2001 1116(b)(8)). In theory, the threat of sanctions provided incentives to the States to: a. incorporate scientifically-based research strategies that strengthen the core academic program in schools served by the local educational agency, and b. identify actions that have the greatest likelihood of improving the achievement of participating children in meeting the State's student academic achievement standards so that, by the 2013-2014 school year, all students would be performing at the proficiency level in reading, language arts, math, and science (NYSED 2011a; USDOE 2001 1116(c)(7)(A)(iii)). In practice, the incentives didnt achieve what they were supposed to. Although the goals were educationally sound, and unprecedented levels of federal funding were made available to accomplish them, we are no closer to reaching this goal now, halfway through the target 2013-2014 academic year, than we were a dozen years ago when NCLB was passed. In fact, in many ways, we are farther from this ideal than ever. In New York City alone, 30% of schools missed federal achievement guidelines in 2008 (Gonen 2009). A

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT record 27 New York City public schools were shut down for poor performance in 2011, and an additional sixty schools were targeted for closure or overhaul in 2012 (NY1 News 2012). New York is not alone in its failure. Nationwide, according to a study by the American Federation of Teachers, only 11 out of 50 states completely met the criteria for having both strong content standards and documenting that tests align to the standards required by NCLB (Great Schools 2011). The National Assessment of Education Progress (NAEP), a national data collection system that has been monitoring education outcomes since 1969, found that after three decades of education reform there was only minor improvement (< 2%) in mathematics and even less (< 1%) in reading, science and other subjects amongst those students about to enter 9th grade (Van Duzer 2006). In hindsight, the general consensus among educators is that the NCLB legislation made the fundamental mistake of establishing State academic assessments (1116(a)(1)) as the sole criterion for determining both student and school performance. In effect, formal statewide examinations would be the single standard used to determine whether the school was making adequate yearly progress towards the NCLB goals. As a result, the focus of education shifted from mastery of skills to achievement on tests. In its Educator Guide to the New York City Progress Report (2010), NYCDOE spelled out the measurement of school and student performance levels in no uncertain terms: The State assigns Performance Levels 1, 2, 3, and 4 to scale scores on the State ELA [English Language Arts] and mathematics exams. These performance levels reflect the extent to which the student demonstrates the level of understanding expected at his/her grade level. (emphasis added) Student success on these specific tests counts for 25% of the schools overall grade (A, B, C, D, or F). An additional 60% depends on student progress that is, how much the student

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT populations proficiency on the ELA and mathematics examinations have increased as they moved from one grade to the next at the same school. Standardized testing does not assess the students functional competence and comprehension of the tested subject matter. Rather than asking students to generate an appropriate response from their own knowledge, it tests their ability to select a correct answer (typically out of a choice of four). Because the schools are graded on their students ability to test well, school administrators and teachers have what former U.S. Secretary of Education Arne Duncan calls perverse incentives to skew the results of standardized testing as a measurement of educational outcomes (Dillon 2010). As other educators have noted more bluntly, the focus on high-stakes tests gives the states, schools, students, and teachers incentives to cheat (Great Schools 2011). Many schools narrow their curricula to concentrate on only those subjects which will be formally tested by the state (National Center for Fair & Open Testing [NCFOT] 2007), often at the expense of education in subjects that are either untested or fall outside the evaluation criteria established by the State (Great Schools 2011; Zimmerman & DiBenedetto 2008; NCFOT 2007). The subjects that form the core of those criteria English and mathematics are frequently changed to conform to the content and format of the test (NCFOT 2007). This approach, called teaching to the test, has become so pervasive that it has all but replaced skills training as a primary educational goal (Great Schools 2011). This has been going on in American classrooms for over a decade. A survey by Education Week magazine in 2000 showed that 66% of teachers thought that state tests were forcing them to concentrate too much on what was tested (Great Schools 2011). The National Center for Fair and Open Testing (2007) noted that this practice has made the schools and states

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT accountable to a completely unregulated testing industry instead of to the students, the parents, and the goal of equipping students with the knowledge and skills an education is supposed to provide. In 2010, President Obama called for a change to the current practice of using state examinations as the sole criteria for assessing student and school success. In the Blueprint for Reform: The Reauthorization of the Elementary and Secondary Education Act (USDOE 2010), the Obama administration sought to raise standards, improve and expand methods of assessment, promote comprehensive education, implement rigorous and fair accountability, promote a culture of success, and focus less on compliance and more on enabling effective local strategies to flourish (USDOE 2010). The ultimate objective of all these processes is to improve student learning and achievement ... by providing intensive support and effective interventions in American schools, a goal now targeted for success by the year 2020 (USDOE 2010). What makes the Obama plan somewhat different from his predecessors efforts is that it recognizes that the current educational system is actually in decline (Obama 2010; Sunderman 2009). Additionally, it acknowledges the fallacy of using standardized testing as the sole criterion for measurement student performance (Dillon 2010; Obama 2010), and responds, at least in part, to the petitions from 14 states to widen the scope of assessment methods to include growth or progress factors (Zimmerman & DiBenedetto 2008). Finally, block grants to designated Schools in Need of Improvement will be applied for not given outright and continued funding (of up to five billion dollars) will be contingent on the grantees not just setting measurable annual goals but demonstrably meeting them (Associated Press [AP] 2009; Dillon 2010; USDOE 2010).

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Although the administrations determination to re-vamp and re-energize education in this country is clear, the difficulty remains in overhauling the systems in thousands of school districts that have spent decades following a single, ineffective path of education for evaluation instead of education for mastery (Chung-il 2007). Our educational system is entrenched in an old factory model paradigm that is ill-designed to accommodate the kinds of shifts that the president is now proposing (Van Duzer 2006; Chung-il 2007). The Blueprint calls not only for better goals, but more effective practices in achieving them, something prior legislative policy initiatives have never expressly advocated. It emphasizes the use of innovative approaches and insists that they should be backed up by research evidence of success. It will come as no surprise to any educator with a background in pedagogy that there is such a practice one with strong evidentiary support that can meet the criteria established by the USDOE. It has the added advantage of aligning with the NYCDOEs established rubric for high-achieving schools (NYCDOE 2010b), including but not limited to: 1.2(b) Across classrooms, teaching strategies and routines are strategically differentiated so that all learners have multiple entry points, supports, and extensions into the curricula. 1.3(b) The use of teacher and student time is structured to respond to the learning needs of all students so that students engage in challenging academic tasks and develop higher order thinking skills. 1.4(b) Students are interested in their learning and want to succeed 2.2(a) Teams of teachers and individual teachers use or create assessments that offer a clear portrait of student mastery providing meaningful and actionable feedback on the effectiveness of classroom level, curricular, and instructional decisions. 2.2(b) Teams of teachers and individual teachers identify strengths and needs, track progress, and adjust classroom level curricular and instructional decisions.

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT 2.2(c) Teams of teachers and individual teachers supplement summative and Periodic Assessment data, create a picture of individual students strengths and areas of need, and differentiate instructional strategies. 3.1(b) Goal-setting and action planning at the school level are informed by analyzing student outcomes and existing instructional and organizational practices with regard to closing the achievement gap and/or college-readiness expectations. 3.2(b) Individual teachers effectively and consistently analyze data to identify which students need additional supports and extensions, and set differentiated annual and interim learning goals for those students to accelerate their learning so all students are on a path to mastery 3.3(c) The practice of providing feedback to students and families on students progress and opportunities for support and enrichment is consistent and there is evidence that this feedback is understood and used by students and families.
(emphasis added)

10

A practice that was designed to accomplish all of these goals, and which abundant research has proven successful, is called Mastery Learning, an educational approach that was popularized by education pioneer Benjamin S. Bloom in the 1960s and 1970s. It has fallen out of use in the regular classroom because of the paradigm shift to test-driven teaching. This paper revisits the Mastery Learning instructional model, which advocates a recurring pattern of instruction, assessment, reinstruction, and reassessment until students demonstrate a high standard of competence in each curriculum unit. Units are broken down into small component skills and sequenced for increasing complexity. No student advances to the next skill until he or she has demonstrated mastery of the underlying ones (Block 1971). Student engagement in the learning process, rather than investment in mere numerical outcomes, is paramount (Ames and Archer 1988). Ungraded assessments, used as diagnostic tools for refining and reinforcing instruction, are more important than graded ones (Bloom 1968; Bloom 1971).

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Twenty years of test-driven instruction makes these mastery-focused practices seem radical. Current high school students have never been educated under this antiquated paradigm. Students who have spent years failing under the standardized testing model have learned that they cannot learn. They may plod from one test to another, hoping to defeat the test at whatever minimum score is set, but with little expectation of actually mastering the material. This researcher sought to determine whether students who had never known anything except score-driven failure could adapt to the mastery-based model. Would the teach-assesscorrect-reinforce-reassess-repeat pattern improve their overall accuracy in mathematical calculation? Could incremental skill-building techniques improve their comprehension and processing skills? What impact would post-assessment reinstruction have on student engagement and attitudes toward learning? To answer these questions in a tangible way, the study was designed to assess the following six hypotheses: a. If students are taught using a Mastery Learning instructional model, they will increase their in-seat problem-solving activity by 10% and blackboard problem-solving activity by 5%. b. If students are taught using a Mastery Learning instructional model, they will offer verbal or written responses to 4 out of 5 calculation questions. c. If students are taught using a Mastery Learning instructional model, they will offer verbal responses to 4 out of 5 process questions. d. If students are taught using a Mastery Learning instructional model, they will reduce the frequency or duration of avoidance behaviors during class by 20%. e. If students are taught using a Mastery Learning instructional model, they will increase their accuracy in solving arithmetic calculation problems by 25%.

11

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT f. If students are taught using a Mastery Learning instructional model, they will increase their accuracy in solving word problems by 25%. By focusing on both cognitive and attitudinal aspects, the researcher intended to determine whether the historical claims in favor of a mastery learning approach to education were valid.

12

Review of Literature When the mastery learning (ML) approach was introduced as an educational philosophy nearly a century ago, it offered a bold promise: by establishing specific objectives, organizing instruction into well-defined learning units, requiring students to master each unit before proceeding to the next, administering ungraded diagnostic texts after each unit to isolate deficiencies in understanding, providing feedback and corrective instruction in those areas; and affording the students additional time to improve their comprehension of the material, 75% 90% of students could achieve the same high level as the top 25% of students in classes using traditional instructional methods (Block 1971). This premise has been tried, abandoned, and tried again many times in the last ninety years, and the results are almost universally reported to bear out the startling predictions of Carlton Washburne (1922), Henry Morrison (1926), Benjamin Bloom (1968), James Block (1968), and numerous other educational researches throughout the 20th century. In more recent decades, mastery learning has fallen out of favor as an educational approach. With the passage of the Educate America Act (Goals 2000) in 1994, mastery began to be evaluated by the numbers: by the year 2000, the high school graduation rate will increase to at least 90 percent; by the year 2000, all students will leave grades 4, 8, and 12 having demonstrated competency over challenging subject matter including. (103rd Congress of the United States of America 1993). Unfortunately, the operative phrase in that goal is not

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT challenging subject matter but rather having demonstrated competency. The subsequent No Child Left Behind Elementary and Secondary Education Act passed in 2001 established the means by which those children would demonstrate their competency: Each local educational agency receiving funds under this part shall use the State academic assessments to evaluate the progress of each school served under this part to determine whether the school is making adequate yearly progress (U.S. Department of Education [USDOE] 2001). Shall use State assessments means conducting normative performance tests. To evaluate the progress of each school means that the comprehension of the material by individual students is secondary to the overall performance of the entire population of the educational facility. Every state that accepts federal funds for education must measure up to the standard set by the USDOEs rather vague criterion of adequate yearly progress. The consequence, whether intended or not, was a teach-to-the-test mentality that emphasized behavioral performance goals rather than subject matter mastery goals in every classroom. Ames and Archer (1988) distinguish the various elements of the classroom climate based on these two types of approaches:
Classroom Dimension Success is defined as Value is placed on Reasons for Satisfaction Teacher oriented toward Focus of attention is on View of errors/mistakes Reasons for effort Evaluation criteria Mastery Goal approach Improvement; progress Effort; learning Working hard; challenge How students are learning Process of learning Part of learning Learning something new [Individual] progress Performance Goal approach High grades; high normative test scores Normatively/comparatively high ability Doing better than others How students are performing Own performance relative to others Producing anxiety High grades; performing better than others Normative performance

13

The researchers studied the impact of these distinct approaches on the learning strategies and motivation of 176 Illinois teenagers in grades 811. Rather than assess the students specific

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT performance outcomes (as in normative testing), Ames and Archer investigated the effect of each educational perspective on the students process and attitudes in the learning environment: how they approached tasks, engaged in the process of learning, and responded to classroom situations. They hypothesized that there would be a direct correlation between the Goal Orientation of the classroom and the students (a) individual goal orientation; (b) choice of learning strategy; (c) receptivity to challenges; (d) rating of the likability of the class; (e) attribution of causes for success/failure; and (f) their perceived ability in the subject matter. Except in perceived ability, students in the ML classes showed significantly more positive attitudes toward all aspects of the learning experience. They accepted responsibility for their learning process, and attributed success and failure to their own efforts rather than lack of ability on their part or lack of effectiveness on the teachers part. They recognized that mistakes are part of learning. They emphasized their effort and improvement rather than their achievement scores. They were more likely to choose challenging tasks than easy ones, and reportedly engaged in self-planning, self-monitoring, and information processing strategies to meet those challenges. It is clear that early research into the mindset of students based on goal orientation favors the ML approach as a motivating and engaging element of the classroom climate. This aspect was subsequently investigated further to determine if the effect occurs not only when a particular kind of assessment is actually implemented, but also when it is merely anticipated by the students. Ruth Butler (2006) studied 312 junior high school students (ages 13-15) in Jerusalem, Israel, to determine whether the expectation of a certain type of evaluation after instruction was sufficient to alter their approach to the learning process during instruction. Students were

14

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT divided into three experimental conditions: (a) the normative assessment group who would, as was typical, complete ten reasoning problems at the conclusion of instruction, hand in their papers for evaluation, and receive a percentile score for the unit; (b) the temporal evaluation group who would receive feedback contemporaneously with completion of each of the ten problems, in order of presentation; and (c) the no evaluation group, who would not hand in their work and would received no feedback on their performance on the 10 reasoning problems. The students were told at the outset the evaluative criteria that would be used for their group, but not for the others. The groups that would receive feedback were also informed that their evaluations would be received in an envelope bearing their assigned serial number, making the feedback semi-anonymous. The findings of the investigation produced both expected and unexpected results. Like Ames and Archer before her, Butler found no significant difference amongst the experimental groups in their perceptions of their abilities to perform their assigned tasks. Also unsurprisingly, the findings substantiated the hypothesis that those who anticipated an end-of-unit assessment focused more on the ability to solve the problems than on mastery of the subject matter, whereas those who would receive temporal evaluations at the time they were solving the problems focused more on mastery. Those not expecting evaluation at all showed similar, moderate degrees of investment in both ability and mastery outcomes. A less expected result was noted in the effect of goal orientation on performance over time. In the early problems, there was little difference between the Normative and Temporal groups, but as the exercise progressed, the Temporal group surpassed the normative group to a significant degree in both accuracy of the solution and complexity of problem-solving process. Furthermore, in addition to higher achievement on specific tasks, the mastery goals associated

15

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT with the Temporal group correlated with greater improvement in performance as they progressed through the activities. These results demonstrate the value of prompt feedback in improving and extending achievement. An even more surprising outcome related to the intrinsic motivation the students reported toward their tasks. Not only did the Temporal Evaluation group report significantly higher interest than the Normative students, but so did the No Evaluation group. Those who expected no feedback on their performance at all were still more invested in the process and the outcome of their tasks than those who anticipated a percentile evaluation after the end-of-unit assessment. Butler also determined that students who focus on meeting a normative standard appear to have a more conditional approach to sustained effort. They invest in their activity with an eye toward demonstrating superior ability, but their engagement is sustainable only if the desired recognition of their superiority is confirmed by the evaluation. If the test does not bear out the high performance levels they anticipate from their efforts, they cease to maintain the same level of investment in the learning process. In examining this finding, Butler also noted that her results confirmed that context matters to the students, as (she says) it should to the teacher. She asserted that the teachers attitude toward testing influences the students attitude toward learning the subject matter. If the teacher approaches the test as the end of instruction, regardless of the results Okay, were done with that; lets move on to something else now. the students are more likely to believe that mastery of the subject matter is irrelevant or unimportant. Maintaining student motivation is not only an objective of mastery learning but also an essential component of its success. The more complex the learning, the more important it is for instructors to sustain (and, if necessary, create or redirect) their students interest.

16

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Researchers at the University of Memphis investigated how the emotional states of students impact their learning ability and receptivity to instruction (DMello, Taylor & Graesser 2012). While they noted that basic emotions such as anger, fear, sadness, and disgust do not play a significant role in learning, transient affective states such as boredom, confusion, frustration, and flow (engagement), and the less common emotions of delight and surprise, do play a role. Their prevailing theory, supported by prior research, indicated that deep comprehension is most likely to occur when learners confront contradictions, anomalous events, obstacles to goals, salient contrasts, perturbations, surprises, equivalent alternatives and other stimuli or experiences that fail to match expectations. They sought to determine whether affective trajectories shifts in emotion states during the learning process correlated with student achievement. While monitoring the emotional states of 29 college students, they observed that boredom is the greatest hindrance to knowledge acquisition of all the emotional states studied, and a bored learner remains bored and transitions into frustration at rates significantly greater than chance. By contrast, learners in the zone of flow are likely to stay engaged or transition

17

to a state of confusion, which the researchers determined is not a deterrent to learning but is instead positively correlated with it. In other words, if confusion can be provoked, the students are less likely to disengage with the material. The authors note that confusion is sometimes viewed as harmful to learning, but they hold with the alternative view that it can be beneficial, most particularly because it rarely turns to boredom. Frustrated learners, on the other hand, are equally likely to turn to boredom or simply remain frustrated as they are to transition to flow or confusion. Frustration and boredom are the greatest hindrances to learning and the most persistent emotional states in the learning environment studies. The researchers also note that reactive attempts to foster transitions out of

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT these negative states may not suffice. Instead, they propose proactive pedagogical strategies to circumvent the incidence of negative emotions. Other studies support the claim that Mastery Learning is one method of proactively circumventing boredom and frustration and creating the more positive emotional states of confusion, flow, surprise and even delight. Ironsmith and Eppler (2007) investigated the impact of mastery learning techniques on both achievement and motivation of 948 low-aptitude college students, comparing the outcomes of traditional lecture programs with the mastery learning PSI model (Personalized System of Instruction) developed by Fred Keller in the 1960s. As expected, they found that PSI students clearly out-performed their lecture-based counterparts in academic achievement. Not only did PSI students earn higher final exam scores than the control group, but the benefit was greatest among those students with the lowest starting grade point averages. This is especially exceptional because mastery during the pre-final quizzes was measured at 90%. As for motivational factors, the results demonstrated that PSI students may though do not always develop stronger learning goals, a greater sense of control over their learning outcomes, more resilience in the face of academic challenges, and increased intrinsic interest in the subject matter. All of these factors are noted by the authors as fostering improved future learning. Both the advantages in academic achievement and the attitudinal benefits of a mastery learning approach were similarly evident in the Australian study conducted by Paul Francis and his colleagues (2009), but this study also showed some of the disadvantages of the mastery learning approach.

18

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Observing that advanced-level physics students still had major problems with the most basic material, they set out to explore whether instituting a mastery learning approach for the first-year class would generate better understanding than the large, traditionally timetabled class. They broke the curriculum into levels, each providing three opportunities to demonstrate mastery, with detailed feedback and optional drop-in tutorials available at each of the first two assessment points. Students were required to complete at least one assessment per week. The initial findings showed both positive and negative effects of the change in approach. A tiny segment of the 230 subjects began completing coursework at an unprecedented rate, covering half the curriculum units in the first two weeks of the semester. While they spent a great deal of time working at more advanced levels, these students later complained that they were not being supported in the lectures, which were proceeding at the pace of the slower students. Few students took advantage of the flexibility of the system to cover material at a different pace. Some subjects were unable to make the transition to the alternative form of mid-unit assessment. About 20% of the class took the a test and then immediately proceeded to the b test on the same material. The researchers discovered that the students didnt know the two unit tests were covering identical concepts, and didnt realize they were supposed to review the feedback before proceeding to the subsequent assessment. The instructors were able to call attention to the mistake and correct the problem immediately. Additionally, a review of the identified errors demonstrated that many incorrect responses was attributable to carelessness and misreading of the question, rather than poor comprehension of the subject matter, in 40% of the cases. Again, immediate intervention resolved the issue.

19

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT One issue that was not immediately resolvable was the resistance of a small segment of the first-year students who were unable or unwilling to confront their weakness early on when they repeatedly failed to master the basic levels of the course. The drop-out rate increased from 12% to 18% when the students realized that they could not proceed to the more advance levels and make up their marks with the final exam. The authors note that higher drop-out rates are not unique to their own population but rather a generic feature of mastery-based courses. A derivative downside of the mastery learning approach is the time it takes for most students to rise through the instructional levels. Only 70% of the students completed all of the course material. On the more positive side of this study, the students who did complete the course had developed the depth of understanding of the foundational concepts (at 80% mastery) that the researchers sought to promote. One fifth of the students attended the optional tutorials, and these were not simply remediation sessions for struggling participants. Many of those who attended were already receiving good grades and wanted even better ones. The new learning model promoted an improved attitude toward the workload when compared to student feedback from prior years and other lecture-based classes. In what the researchers call a striking change, the study participants typically did 50% more homework with no complaints about it, where lecturebased students had always complained about the workload. This attitude was replicated in the laboratory portion of the course, where students would no longer rush through experiments but [have] the courage to stop and say, no, instead of finishing the whole lab badly, I would rather concentrate on this bit and do it properly. This parallels the level of investment noted by Bloom (1968), Block (1971), Butler (2006), and Ames & Archer (1988).

20

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT It should be noted, however, that the level of investment by the student may not be intrinsic to the mastery learning model but to the students perceive value of the mastery goals. Research by Dompnier, Darnon and Butera (2009), students in mastery learning classes may fake a desire to learn that they dont actually feel because it may seem like a way to be seen as a better student by the teacher. That is, the teachers appreciation of mastery goals is a signal to the students that they can ingratiate themselves with the instructor by adopting, at least on the surface, the same attitude. They themselves might be achievement-oriented, but they feign an interest in mastery to garner favor from the person who will evaluate them. The researchers attempted to discover whether the students perception of mastery goals as either socially desirable (for teacher appreciation) or socially useful (as a function of perceived competence) impacted the students final grades. They discovered that, in their test population of 267 first-year French psychology students, students who endorsed mastery learning as socially desirable also endorsed its goals as socially utile to a significant degree. As social utility perceptions increased, so did the participants grades. This may seem like a chicken-egg riddle, but the impact of mastery learning techniques in less Machiavellian environments is predominantly a positive one. In 1983, after Chicago city colleges had operated mastery learning classes for more than a decade, earned credit ratios (ECR) were compared between students participating in one or more ML classes with those who were not enrolled in any such classes (Bonczar & Easton 1983). The studies found that while a single ML class did not increase the participants ECRs significantly over their non-ML counterparts, enrollment in two or more ML classes did produce a notable improvement in academic success, both in their ML learning classes and in their

21

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT additional non-ML classes. Their data suggested that the skills acquired in ML classes were immediately transferrable to non-ML classes. A meta-analysis of 15 studies on mastery learning demonstrates that the co-operative (small group) approach to classroom instruction and practice, interspersed with individual assessment and ongoing feedback, reduces the incidence of what the researchers call middle grades drop-off, a sharp decline in mathematics proficiency between 5th and 7th grades (Nunnery, Chappell & Arnold 2013). The authors analysis was prompted by dismal statistics showing that American 15-year-olds rank 25th (out of 34 countries) in math literacy, and the massive and permanent crimp that math deficiency creates in students future financial, career, social and decision-making prospects. Among the factors cited by the studies under review, Nunnery and colleagues noted that cooperative learning leads to increased achievement in even low-achieving student, including those with disabilities, limited English proficiency, culturally diverse backgrounds, and traditional gender-discrimination burdens. Success can be achieved, the authors report, if students are structured into groups of 4-5 and presented with both group goals and individual accountability for achieving those goals. Higher functioning students gain group credit and individual reward by assisting lower performing teammates, and low achievers contribute to their teams success as well as their own by each incremental improvement, not by overall scores. Peer interaction was shown to provide practical opportunities for increased modeling, repetition and social support. It also appears to prevent or reduce the common math anxiety that inhibits learning and to increase engagement and motivation in learning tasks. The researchers warn, however, that the success of cooperative mastery-based learning is not based on the group format but on the goals-structuring requirement. Group goals and

22

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT individual accountability are necessary. Otherwise, students might simply share answers rather than try to explain ideas to each other. Sharing answers without explanation, the authors note, has been found to inhibit, not aid, learning of mathematics in cooperative learning contexts. The sharing of information and explanations appears to be the single most important element of effective mastery learning, not just between students in learning clusters but between students and their teachers. The sharing of information is part of every instructional activity in every classroom, but what sets mastery learning apart is the additional explanations the ongoing exchange of tailored feedback coupled with clarifications of where the student lapsed in understanding, and how to improve their knowledge and skills. It is these further interventions that produce the greatest improvement in both comprehension and testable performance. Two studies demonstrate this phenomenon most clearly by assessing the roles of formative testing (James 2012) and Feedback and Remediation (James & Folorunso 2012) on the mathematical competencies of junior secondary school students (ages 11-16) in Nigeria. In 2009, 84% of students failed the Senior Secondary Certificate Examination (James 2012), prompting the Federal Government to investigate the problem and recommend corrective action. The resulting study was conducted to determine whether the mastery learning technique of using formative evaluations would increase performance, retention and transfer of learning in students three to six years from their graduation exam. Formative testing would be used to pinpoint individual and group errors requiring correction, to motivate students, and to overcome learning deficiencies. All 312 subjects followed the identical math syllabus and attended expository classes using the same teaching techniques and materials. Three of the four experimental groups took

23

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT examinations at the end of each instructional unit. The difference between those three groups was in what use was made of the end-of-unit assessment. One group received feedback on their performance as well as remediation on identified areas of deficiency. The second group received feedback on their performance but no remediation (the traditional approach in Nigerian, as well as American, education). The third group received neither feedback nor remediation. The fourth experimental group (the control group) sat through the classes but took no class tests. The findings indicated that in every instance, the testing procedure resulted in significantly improved performance over the control group, but that improvement was dramatically higher in the feedback and remediation group than in the feedback-only and testonly groups. Based on these results, the author recommended that formative tests be used for diagnostic purposes, as a basis for identifying the source of difficulties in mathematical performance and as a way of providing specific corrective action. An additional study by the same researcher and a colleague focused on the effect of the particular feedback and remediation activities (James and Folorunso 2012). Utilizing a threegroup experimental structure (lacking the non-tested control group in the above study) among 240 different Junior Secondary School students, they sought to determine whether particular techniques of diagnosis and directed remediation had a significant impact on student performance and whether other variables such as gender and socioeconomic status were relevant factors in mathematics achievement under the experimental conditions. The techniques that were applied to the first experimental group that were withheld from the second and third groups included the following post-examination activities: (1) feedback on individual student performance (their graded test papers were returned); (2) division of test items into two or three sections; (3) allowing the students with the highest score in each section to lead

24

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT the class for a review of that material; (4) class discussion to identify the correct answer to each item in the section; (5) continuing the discussion with a question & answer period regarding difficult areas; (6) encouraging students to provide answers amongst themselves; and (7) the teacher acting as a guide and/or providing assistance where needed. All of these elements were complete, for each section of the test, before any student moved on to the next unit of instruction. By contrast, the second group received their graded test papers back and moved on to the next unit. The third group did not receive their marked papers back, and no reference was made to the test once it was administered. They moved on to the next unit immediately. After the third formative test, post-tests were administered. All groups showed improvement following whatever instruction was provided, but the group following the mastery learning practices of feedback and student-led remediation showed the greatest jump in achievement, nearly double the performance of the students who received no intermediate feedback during the instructional period. The authors concluded that the current use of continuous testing without feedback and remediation was not only ineffective in producing knowledge and retention but may actually inhibit learning in teenage students. They recommended the discontinuance of that system and the introduction of feedback-based re-instruction tailored to the students needs based on regular assessment. Methodology Subject Population The twenty-seven participants in the current study were randomly assigned to two special education Math Fundamentals classes in a New York City public high school. Each class was staffed by a State-certified special education teacher and a paraprofessional (assigned for data

25

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT recording and supplemental instructional purposes). The staffing ratio was a maximum of 15:1:1 in each class. All subjects were ninth-graders in their first year of high school, and ranged in age from 13 to 16 (mean 14.4). Twelve were female, 15 male. The population encompassed a mix of handicapping conditions: learning disability (74%); emotional disturbance (15%); speechlanguage impairment (7%); and intellectual disability (4%). All students were functioning at least four years below grade level in mathematics based on tests scores reported in their Individual Educational Programs; grade level equivalent (GLE) scores ranged from 2.8 to 5.1, with an average GLE of 4.2. Although the study began with 27 subjects, three students were dropped from the research population in week four for chronic truancy. These students were included in the baseline data and the final summative analysis (shift in performance from baseline levels after instructional week 8), but not in the interim measurements during the investigational period.

26

Data Collection Baseline responsiveness and accuracy. Baseline performance levels of responsiveness and accuracy were obtained for all students using a combination of two methods. First, an untimed, 20-question, fill-in assessment instrument spanning a range of math skills in the Math Fundamentals 1 curriculum from first- to ninth-grade levels was administered under testing conditions. Students were advised that this was an ungraded assessment that would not affect their standing in the class but was being used to only determine the appropriateness of their academic placement. They were expressly told that some of the material would be unfamiliar

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT and that they should do their best to answer each question but not worry if they could not answer any individual problem. The number of responses, non-responses (blank answers), and accurate responses were recorded, and used to calculate two levels of accuracy: (a) the percentage of correct answers out of the responses given, and (b) the percentage of correct answers out of the total number of problems presented. On average, the students responded to 14 of the 20 questions (range: 6-18), a response rate of 70%. Of the questions answered, approximately 74% were answered correctly. Had the test been graded based on all twenty questions and not just those answered, the average score would have been 55% (range: 30-70%). The above procedure represented an assessment of responsiveness and accuracy for material that would span the semester, not just the investigational period. To confirm the validity of baseline data for a more focused segment of the curriculum, additional measures of initial responsiveness and accuracy were taken during class instruction for the first week. A total of 40 in-class problems and 25 homework problems were presented based on daily lessons. Due to time constraints and staffing limitations, it was not practical for the researcher and paraprofessional to check every students in-class written work. Additionally, because students needed their class notes to complete homework assignments, it was not educationally sound to collect their daily work for teacher review. We therefore relied on selfreported completion rates during the course of the study. To avoid inflated reports by the students, overt spot-checks of their written work were made during class, and teacher-assessed nightly homework was added to the data to correct for any significant discrepancies between student reports and reality. During the progress of the investigation, unit exams would also add an element of correction to any skew in the self-reported data.

27

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT The measurement of responsiveness was limited to the number of questions answered, regardless of the accuracy of the answers given; subjects received credit for each response, whether or not it was correct. Students who missed classes were not permitted to make up the in-class problems but were allowed to make up the homework assignments for those days. Absenteeism adversely affected the responsiveness scores for 6 of the 27 subjects (22%) in the baseline assessment week, but the class as a whole maintained an average responsiveness of 78%, which was comparable to the 74% response rate on the written assessment. Accuracy rates during the baseline instructional period (57%) were also comparable to those on the written assessment (55%). Baseline engagement and avoidance. During the initial assessment week, student behavior was monitored and recorded for acts of engagement and acts of avoidance. Engagement behaviors included: in-seat verbal responses; at-board problem solving; explanations offered with regard to mathematical process or reasoning; and corrections made to the work of other students. Voluntary/spontaneous responses were accorded equal weight as ondemand responses. A daily average was calculated based on the number of occurrences over five instructional days and the students attendance for the week. Engagement behaviors ranged from zero to nine for the week (daily average: 0.78) Avoidance, or disengagement, behaviors were similarly monitored and recorded. These included: outright refusals to work; cessation of work after commencement and before completion; visual redirection away from the work; verbalizations unrelated to the work; physical distancing; body reorientation away from the work; and overt gestures of rejection of instruction. Time constraints and staffing limitations made it impractical for the instructor and paraprofessional to record the duration or assess the intensity of the individual behaviors, so each

28

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT occurrence was equally weighted. Where a student demonstrated multiple avoidance behaviors in a single incident, the episode was recorded as one occurrence. A daily average was calculated based on the number of episodes per week and the students attendance during that five-day period. The number of avoidance behaviors ranged from 9 to 29 incidents per week, with daily averages ranging from 1.8 to 8.0 (mean: 4.4). Visual and verbal distractions represented the most common interruptions in the learning process.

29

Instructional Model Beginning in the second week, class instruction was presented using a mastery learning approach. Curriculum units were broken down into individual skills and corresponding tasks. For example, the Word Problems section was divided into decoding the question (involving math literacy skills) and solving it (math numeracy skills). Initial instruction did not involve any calculation of the answer but rather a reading of the problem: understanding the scenario; determining what information is sought; locating the numbers that would be used in the calculation of the answer; distinguishing increase from decrease as an outcome of the process; identifying increase with the operations of addition or multiplication and decrease with the operations of subtraction and division; identifying keywords which signal the appropriate operation to use; etc. Only after these processing skills were mastered would the students be asked to add calculation (numeracy) to their performance Both literacy and numeracy skills were sequenced for increasing complexity and presented in order, with an informal, open-book, untimed diagnostic evaluation after each task and reinforcement where needed. More formal closed-book (but still untimed) unit exams were given when the initial instruction was completed.

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT The criterion for mastery of any individual skill set was 80%. Notably, this success rate (often represented as in 4 out of 5 trials) is the standard performance level required on the students IEPs to indicate their achievement of an academic goal and their readiness for a new objective in their educational plan. However, this minimum passing grade far exceeds the 65% standard to which most students are held, even in general education classrooms. Students who did not demonstrate 80% success on the first unit exam were given additional instruction in the subject matter and retested. Those who did succeed on the first exam were often encouraged to explain the process or demonstrate skills at the blackboard. This teach-test-reinforce procedure was repeated, either on a whole-class, small-group, or individual basis, until each student achieved the minimum performance level. No student would advance to the next instructional unit until s/he had demonstrated mastery of the earlier skill set. Students who did not succeed after the second unit test were either paired with a higher performing student for peer tutoring and/or assigned to the paraprofessional for additional review and reinforcement. Students were not permitted to take the lower score and move on with the rest of the class to the new, more complex skill. Initially, the researchers recorded the number of unit exams required by each subject to achieve mastery as a measure of student progress. However, it soon became clear that this was not a relevant variable, as no student needed a third unit exam past Week 3 of the study, and only one advanced as far as a third exam during that week. Data were thereafter limited to four elements: responsiveness, accuracy, engagement in the learning process, and avoidance behavior. It was anticipated that the first three measures would increase and the last would decrease during the course of the 8-week instructional period.

30

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Results Findings The initial evaluation of the baseline data revealed the following general findings: At least one half of the students had inconsistent computational skills; what they could do in one situation did not carry over into other situations. All subjects were deficient in both quantitative conceptualization (mathematical reasoning or literacy) and practical applications (mathematical numeracy or calculation); All of them lacked the ability to solve word problems independently. When students did not immediately understand the question, they generally responded less than 10% of the time if they; they did not attempt to decipher the question or work out a method of solving it. Students engaged in distraction/avoidance behaviors when they didnt understand what to do; They treated whatever grade they received, even a failing one, as final; they had no concept of seeking correction and doing the work over to improve their score) Based on diagnosed disabilities and a review of grade-level-equivalency scores prior to the instructional period, these results were as anticipated. Following the 8-week instructional period, however, the level to which most of these patterns were positively influenced surpassed the researchers expectations. The data support most of the hypotheses with which the study began, in many ways far exceeding the predicted benefits of a Mastery Learning instructional program. However, while it was clear that the students became more engaged in their learning and achieved unanticipated

31

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT gains in both responsiveness and accuracy, the results revealed that other factors influence performance, and not all progress was incrementally upwards. The first hypothesis proposed that students in Mastery Learning math classes would increase their in-seat problem-solving activity by 10% and blackboard problem-solving activity by 5%. students. Responsiveness which, for the purpose of measuring engagement in the learning process, is distinguished from accurate responsiveness increased for all students who completed the eight-week instructional period. Predictably, those who had the highest baseline responsiveness had the smallest increase at the end of the study, but the improvement in responsiveness was below 10% for only one of the twenty-four students (4%). Six students (25%) more than doubled the number of questions completed from pre-test to post-test. Improvement for the rest ranged from 25% to 96% overall. Overall, this hypothesis is confirmed, though the results are not comparable for all

32

Change in Responsivness
(percentage of assigned problems completed)
30 Number of Students 25 20 15 10 5 0 Week 1 (N=27) Week 2 (N=27) Week 3 (N=27) Week 4 (N=24) Week 5 (N=24) Week 6 (N=24) Week 7 (N=24)

Increased No Change Decreased

Week 8 (N=24)

From Baseline

Figure 1. Data are not adjusted for attendance. While homework problems could be made up, students who missed in-class problems due to absenteeism were unable to complete the missed problems, resulting in reduced responsiveness. Data represent numbers of subjects, not degree of change per student. The three students omitted from the study for truancy after Week 3 are included in the final From Baseline figures as the only three subjects who decreased their responsiveness during the course of the study.

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT It must be noted, however, that the pre- to post-test increases were not consistently reflected in student performance during the individual units. The earliest instructional goals did not involve calculations (numeracy) but concepts (mathematical literacy/reasoning). Math literacy includes decoding skills, understanding a scenario, determining what information is being sought, identifying the numbers that will be involved in the calculation and the operation that will be performed with them. During these non-computational lessons, student responsiveness increased significantly, but when calculation was added to the task, there was a short-term drop in the overall number of completed problems. An additional factor observed to adversely affect responsiveness was attendance. Students returning from an absence tended to take at least a day to return to their prior level of responsiveness. Similarly, after a school holiday, there was a brief decrease in elicited or volunteered answers. Patterns of increases and decreases in weekly responsiveness are represented in Figure 1. The second element of this first hypothesis at-board responsiveness is more difficult to confirm, as only some students were willing to go to the blackboard to demonstrate their problem-solving. Those who were willing went often, but fully a third of the group refused to do board work more than a handful of times over eight weeks. By contrast, these students demonstrated a different sort of increased engagement; they uniformly volunteered more answers verbally and/or offered explanations of processes and/or pointed out errors in their classmates board calculations and/or corrected those errors. During the course of the study, these behaviors were added to board work as indications of improved engagement and responsiveness. Coupled with in-seat responsiveness, these additional behaviors created dramatic increases in overall engagement measurements.

33

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT At the start of the semester, the average daily number of verbal responses ranged from 0.2 (one response in 5 days) to 2.25 (9 responses in 4 days). After 8 weeks of Mastery Learning instruction, that range reached a minimum average of 2 responses per day and a maximum average of 3.6. In achieving this rate of written and verbal responsiveness, the students demonstrated the validity of the second hypothesis that they would answer 80% of the calculation questions presented, either in writing or verbally. In fact, the subjects exceeded this expectation. Once they learned that they received participation credit for any answer, whether or not it was right, they consistently offered their results with increasing frequency, either on demand or spontaneously. The minimum level of responsiveness reported by the students at the studys end was 85%, and 19 of the 24 reported answering 100% of the questions presented.

34

Change in Engagement
(verbal & behavioral participation in learning activities)
30 25 20 15 10 5 0 Week 1 (N=27) Week 2 (N+27) Week 3 (N=27) Week 4 (N=24) Week 5 (N=24) Week 6 (N=24) Week 7 (N=24) Week 8 (N=24)

Increased No Change Decreased

From Baseline

Figure 2. Data represent a daily average of in-seat and at-board responses as well as explanations of process and corrections of classmates work. Differences of less than 0.2 were deemed statistically insignificant and recorded as No Change. Data represent numbers of subjects, not degree of change per student. The three students omitted from the study for truancy after Week 3 are included in the final From Baseline figures as the only three subjects who decreased their engagement during the course of the study.

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT The third hypothesis, that the Mastery Learning student would achieve 80% verbal responsiveness to process questions, was not confirmed by the data. While six (25%) of the students were able and willing to articulate explanations at this higher level of performance, most students were only able/willing to do so in one or two out of five (20% - 40%) situations. Despite demonstrating the ability to apply the process, they declined to verbalize it. Outright refusals to answer a question on demand were among the criteria recorded to monitor avoidance behaviors, the subject of the fourth hypothesis which proposed that the frequency and duration of such behaviors during class would decrease by 20%. During the course of the study, the research team found it infeasible to record the duration of avoidance episodes while keeping up instruction. However, the number of incidents of verbal, visual, and physical disruptions, distractions, and evasions was recorded. Neither the instructor nor the paraprofessional anticipated the extent to which these occurrences would decline. Even amongst the few students with emotional disabilities or attention deficit disorder, the decrease was dramatic.

35

Change in Avoidance
(verbal & behavioral refusals, distractions, or disruptions in learning activities)
30 25 20 15 10 5 0 Week 1 (N=27) Week 2 (N+27) Week 3 (N=27) Week 4 (N=24) Week 5 (N=24) Week 6 (N=24) Week 7 (N=24) Week 8 (N=24)

Decreased No Change Increased

From Baseline

Figure 3. Data represent daily averages of visual, verbal, and physical reactions which constituted a withdrawal from or refusal to engage in classroom learning activities. Differences of less than 0.2 were deemed statistically insignificant and recorded as No Change. Data represent numbers of subjects, not degree of change per student. The three students omitted from the study for truancy after Week 3 are included in th e final From Baseline figures as the only three subjects who increased their avoidance during the course of the study.

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT At the start of the semester, the daily average of avoidance behavior was no less than twice per period and as high as eight incidents. At the end of the study, all students except one had fallen below the prior minimum; that is, 23 participants averaged under 2 disruptions per class, and 11 (46%) averaged less than one. One student in the last week of the program even managed to get through a 5-day school week without one avoidance incident. Increased engagement in the learning process, whether measured by improved responsiveness or decreased avoidance, is only a part of the Mastery Learning approach. As the name of the educational model implies, the goal is for the students to master the subject matter. Toward that end, the fifth hypothesis in this study proposed that students taught with this approach would increase their accuracy in solving arithmetic calculation problems by 25%. As only one student failed to meet this criterion (increasing his accuracy by only 23% of his baseline performance), this hypothesis is considered validated. In fact, the student who narrowly missed the predicted standard of achievement started with one of the highest accuracy rates in the subject population, 65%. Having risen to 80% accuracy (the target goal for mastery), his 15point climb was just shy of a quarter of his initial performance level. For most students, the increase was significantly higher than 25% in calculation. Seventeen subjects (71%) improved calculation accuracy by over 50% from baseline during the ten-week period. It must again be noted that this improvement was not a consistent and steady increase over time. There were fluctuations in performance levels during individual instructional units, and during the cumulative midterm exam, there was a significant drop in test scores for nearly half (42%) of the students.

36

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Additionally, absences and holidays again adversely affected performance, as did the complexity of the material. When students were asked to advance from finding information within a question to manipulating that information to arrive at a solution, scores dropped for a while. Re-instruction became more necessary. Similarly, when topics changed and students had to bring new and different skills to the table, they demonstrated a lapse in accuracy. Combining both literacy and numeracy skills together instead of seeking single-step solutions also resulted in decreased performance. That being said, however, there was a general improvement in homework and test accuracy over the duration of the study. Only four students (17%) failed to attain an ultimate average of 80% across the three curriculum units.

37

Change in Accuracy
(percentage of correct responses)
30 25 20 15 10 5 0 Week 1 (N=27) Week 2 (N+27) Week 3 (N=27) Week 4 (N=24) Week 5 (N=24) Week 6 (N=24) Week 7 (N=24)

Increased No Change Decreased

Week 8 (N=24)

From Baseline

Figure 4. Data are not adjusted for the complexity of the subject matter. Process/literacy questions are equally weighted with calculation/numeracy questions. The three students omitted from the study for truancy after Week 3 are included in the final From Baseline figures as the only three subjects who decreased their accuracy during the course of the study.

One of those curriculum units word problems was the target of the sixth hypothesis. The researcher noted from the baseline test and discussions with more experienced teachers that descriptive, situation-based problems are significantly more challenging for below-grade-level students than solving numerical expressions. Consequently, more time was dedicated to

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT breaking down these problems into nine component skills and building students understanding up from basic levels (or below basic levels) toward true mastery, one skill at a time. All students improved their performance, but not by the 25% predicted for this study. Only half the students could independently solve word problems at the desired level by the end of the study. The sixth hypothesis is therefore rejected, with the following explanation. Those students whose poor math skills were compounded by poor reading skills were at a significant disadvantage in decoding the problems presents. When the questions were read to them (although this service was not part of their Individualized Educational Program), their accuracy improved, as it did even moreso when the problems were discussed before solving. However, mastery was judged on the students ability to solve the problems on their own, and in that regard, half of the students failed to meet the standard. It is important to note that the task breakdown and subsequent skill-building approach was successful in teaching individual skills, but this did not translate into integrating those skills. Subjects could accurately identify less as a keyword for subtraction and split as a keyword for divide, and even write the mathematical equation in numbers and symbols, but still fail to accurately solve the problem. Inconsistencies and fluctuations notwithstanding, there was demonstrable evidence that the students ability to solve word problems improved during that portion of the study. In the first week that word problems were solved (that is, calculated instead of decoded), 25% of the subjects required reinstruction and a second unit exam to demonstrate mastery. In the second week, that number dropped to 17%, and fewer students left problems blank; only one (4%) completed fewer than 80% of the questions.

38

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Conclusions It is clear that the Mastery Learning approach provides a number of beneficial outcomes, but the educational technique by itself is not the sole predictor of success. Multiple factors influence the efficacy of Mastery Learning techniques. These include: the time available for instruction and the flexibility of the schedule; the student-teacher-paraprofessional staffing ratio; the personality and strength of the teacher in establishing a working rapport with the students, implementing the instructional program, enforcing higher performance standards, and encouraging students to persevere in the face of past failures; the disabilities and abilities of the students, as well as their attitudinal factors such as their personalities, prior classroom experiences, and prejudices toward the subject matter; the dynamics of the classroom, including how students interact with each other as well as the teacher; the nature and appearance of the materials provided for instruction; and even the season of the year.

39

This latter element had a surprising impact on the present study, which was implemented in the fall semester. This time of the year presents variables that are beyond the instructorresearchers control: school schedules are more frequently interrupted by holidays, both religious and secular, causing disruptions in the continuity of instruction; students often require greater adjustment in returning to school after the summer and, in the present case, there was the additional adjustment to entering a new school; cold & flu season causes more frequent absences, reducing instruction time and problem-solving practice; and decreased daylight often

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT prompts students to stop work earlier in the evening, reducing the number of homework assignments completed. While these effects must be noted, it was impossible to adjust for these variables or differentiate instruction-related data from circumstantial fluctuations related to external factors. As a result, the significance of the conclusions drawn from the raw data must be tempered accordingly. The most consistent impact was to engagement in the learning process rather than improved proficiency in specific skills. The decline in accuracy during the cumulative midterm exam demonstrated that building on earlier skills is only effective where the complex level is given as much instructional and practice attention as the individual component levels. This study did not allow for the level of precision available to larger research teams with control of all instructional conditions (including time, curricula and materials). The instructor and paraprofessional had to rely on self-reports from the students instead of controlled examination of the work product. Additionally, established and inflexible school periods did not allow for the continuation of instruction to align with the moment-to-moment needs of the students. As a result, the Mastery Learning approach had to be adapted to the time frame instead of the other way around. Still, the results confirm four of the stated hypothesis and suggest the general validity of the other two under particular circumstances.

40

Action Plan The findings of the present study have tremendous potential to improve academic achievement among ninth-grade special education students, many of whom are performing more than five years below grade level in mathematics. Most significantly, the students increased engagement in the learning process and their corresponding decrease in avoidance behavior

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT make the instructional environment more conducive to developing individual and cumulative competency in both math literacy and numeracy. During the course of the research, it became clear that Mastery Learning is a timeconsuming and labor-intensive instructional method. The paraprofessional assigned to my classroom was often involved in data collection, removing her from instructional and reinforcement duties for the duration of the study. Once data collection was complete, however, the addition of a second education professional added to the effectiveness of the Mastery Learning program, particularly in reinstruction and retesting phases. A further benefit was observed in the paraprofessionals ability to encourage the students toward reaching the 80% standard (when they were used to needing only a minimum passing score of 65) and in identifying successful approaches to improving their efforts. The results of this study will redefine this authors use of support personnel in his classroom. A competent paraprofessional is more than a teachers aide and can be effectively utilized as a secondary instructor when provided with appropriate materials and guidance from the teacher. When paraprofessionals are as involved with the students as the primary educator, they develop their own insights into the students behaviors and needs, and often notice things that the teacher might miss while concentrating on instruction. They can bring these matters to the teachers attention and also intervene directly with the student. Going forward, I plan to include my paraprofessional in assessing the students performance and in providing individual or small-group reviews of the material. The paraprofessional can go over unit exams with students who failed to achieve master the first time around, and also administer retests.

41

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT The success of Mastery Learning technique in building individual skills needs to be supplemented with further instruction in combining constituent skills into more complex problem-solving abilities. While the task analysis was successful, multi-step practice remained challenging for a significant percent of the students. Despite achieving the required 80% mastery level on the component pieces of math literacy and computation, they were frequently unable to put those pieces together in solving the more complex problems. Reallocation of time would help resolve this underperformance. The students have demonstrated the ability to learn the parts; they simply need to practice the whole. During the research period, the goal was to cover multiple units and administer assessments for each component skill in the task analysis. In the future, rather than rushing through the units in a race toward the test, I will pace the lessons to allow for incremental skill-building, with repeated reinforcement at each skill level. Additionally, although I had planned on implementing the Mastery Learning approach for the only first three units of the curriculum to engage the students and get them used to the higher standard of success, I see now that it is a disservice to the students to revert to a regular instructional model after the completion of the study. Continuity is essential to preserve what has been gained. I intend to prepare task analyses, Mastery Learning lessons, and multiple assessment instruments for all units in the year-long curriculum. Finally, because students might be assigned to a different math teacher at the change of semester in February, it makes no educational sense to limit the Mastery model to only one third of the Math Fundamentals classrooms. I have spoken with the assistant principal of the special education department about presenting my findings to the other math teachers in lieu of attendance at the next departmental meeting. He has suggested that the presentation be made to

42

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT all of the teachers, as the approach is applicable to any subject. It was too late to schedule the presentation for December, so our intention is to present the information at the January conference, in advance of the new semester. While I believe that most, if not all, of the teachers in my department can make a success of Mastery Learning, I am doubtful that the change in instruction will occur in the short-term. Objections have already been raised regarding the slower pace of instruction and the inevitable failure to cover the curriculum on the set schedule; the need for support staff in classes that are not typically assigned paraprofessionals; and the reactions of underperforming students (who have difficulty achieving 65% accuracy to the demand for 80% performance. I intend to address these concerns in the presentation by reminding my colleagues that: a. The set schedule is designed for general education students who will take the Statewide examinations at the end of the year. Many of our students are Alternative Assessment designees (who dont take the same exams), and those who are required to take the State exams have no chance of passing if they advance through the curriculum at speed without learning anything. Additionally, if the students learn how to learn and acquire a good comprehension of basic/foundational knowledge, prior research in the field of Mastery Learning indicates that the students rate of learning will ultimately increase. The delay is actually an investment in future performance. b. Additional staff is helpful and beneficial, but not essential to the Mastery Learning classroom. After the initial instruction, underperforming students can be assisted by higher achievers, or broken into small groups for rotational instruction and practice.

43

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT c. It is true that some students will balk at the higher demands, but students rise to the level of expectation. Low expectations are not serving them. If the tasks are broken down into more manageable pieces, and sufficient time is dedicated to instruction, any student who can achieve 65% can achieve 80%. In additional, 80% is the traditional standard for mastery of the goals on each students IEP. It makes no educational sense to set 80% as the criterion for achievement and then work toward only 65%. Despite the objections, and even if my responses are not accepted, I hope to instill in my colleagues minds the idea that our students can learn what we teachers can teach. I hope to provide examples of how this was achieved in two of my classes, and then to obtain the cooperation of more experienced subject-matter teachers in adapting the curricula for my three other classes to benefit from this successful approach.

44

References Ames, C. & Archer, J. (1988). Achievement goals in the classroom: students learning strategies and motivation processes. Journal of Education Psychology, 80(3), 260-267. EJ388054. Retrieved from: http://eric/ed/gov/PDFS/EJ388054.pdf Associated Press. (2009, May 11). Obama wants to close 5,000 failing schools. New York Post. Retrieved from: http://www.nypost.com/p/news/national/item_6SAR5kUx59eOoDJPSZ 828J Block, J.H. (1971). Introduction to mastery learning: theory and practice. In J.H. Block (Ed.) Mastery Learning: Theory and Practice. New York: Holt Rhinehart and Winston.

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Bloom, Benjamin S. (1968). Mastery learning. Reprinted in J.H. Block (Ed.) Mastery Learning: Theory and Practice. New York: Holt Rhinehart and Winston, 1971. Bloom, Benjamin S. (1971). Affective consequences of school achievement. In J.H. Block (Ed.) Mastery Learning: Theory and Practice. New York: Holt Rhinehart and Winston. Butler, R. (2006). Are mastery and ability goals both adaptive? Evaluation, initial goal construction and the quality of task engagement. The British Journal of Educational Psychology, 76, 595-611. DOI: 10.1348/000709905X52319. Retrieved from: http://eric/ed/gov/PDFS/EJ750358.pdf Chung-il, Y. (2006). Teachers primary role for education reform: equalizing learning outcomes. Asia Pacific Education Review 8(2), 159-165. Retrieved from http://www.eric.ed.gov/PDFS/EJ776336.pdf DeRugy, V. & Gryphon, M. (2004). Elimination lost: what happened to abolishing the department of education? Retrieved from: http://old.nationalreview.com/comment/ derugy_gryphon200402110914.asp Dillon, A. (2010, March 13). Obama calls for major change in education law. The New York Times. Retrieved from: http://www.nytimes.com/2010/03/14/education/14child.html?_r=1&pagewanted=print Dompnier, B., Darnon, C. & Butera, F. (2009). Faking the desire to learn: a clarification of the link between mastery goals and academic achievement. Psychological Science, 20(8), 939-943. DOI: 10.1111/j.1467-9280.2009.02384.x. Retrieved from: http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?sid=fc061946-edd0-451b-845da0e99d96b0e6%40sessionmgr110&vid=4&hid=106

45

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Gonen, Y. (2009, March 17). State has plan for failing schools. New York Post. Retrieved from: http://www.nypost.com/p/news/regional/item_CQ5f90W6UWTXE4Z3ViUXBN GreatSchools. (2011). Whats so bad about teaching to the test? Retrieved from: http://www.greatschools.org/student/academic-skills/teaching-to-the-test.gs?page=1 Ironsmith, M. & Eppler, M.A. (2007). Mastery learning benefits low-aptitude students. Teaching of Psychology, 34(1), 28-31. DOI: 10.1207/s15328023top3401_6. Retrieved from: http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?sid=a07fe7c2-7616-464eb780-b829c3bedcd5%40sessionmgr112&vid=15&hid=106 National Center for Open & Fair Testing. (2007). How Standardized Testing Damages Education. Retrieved from: http://fairtest.org/facts/howharm.htm New York City Department of Education. (2010a). Educator guide to the New York City progress report. Retrieved from http://schools.nyc.gov/NR/rdonlyres/DF48B29F-46724D16-BEEA-0C7E8FC5CBD5/89549/EducatorGuide_D75_09302010.pdf New York City Department of Education. (2010b). Quality review criteria rubric 2010-2011. Retrieved from: http://text/.nycenet.edu/NR/rdonlyres/8DED4CFE-D87F-4715-AB2AESFC566BAEC8/90078/201011QRubric081110_long1.pdf New York City Department of Education. (2011). Support and intervention at schools. Retrieved from http://schools.nyc.gov/comunity/planning/Support+and+Intervention.htm. New York State Education Department. (2011). No Child Left Behind: Schools in need of improvement. Retrieved from: http://www.p12.nysed.gov/nclb/parents/fssinieng.html NY1 News. (2012). Public City School Closings, Overhauls for 2012. Retrieved from: http://www.ny1.com/content/news/154985/public-city-school-closings--overhauls-for2012).

46

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Obama, Barack. (2010, March 13). Weekly radio address: President Obama to send updated Elementary and Secondary Education Act blueprint to congress on Monday. White House Office of the Press Secretary. Retrieved from: http://www.whitehouse.gov/thepress-office/weekly-address-president-obama-send-updated-elementary-and-secondaryeducation-actReagan, Ronald (1982, January 26). State of the Union address. Retrieved from: http://www.infoplease.com/t/hist/state-of-the-union/195.html Sturgis, C. & Patrick, S. (2010). When success is the only option: designing competency-based pathways for next generation learning. Retrieved from: http://www.eric.ed.gov/PDFS/ED514891.pdf Sunderman, G.L. (2009). The federal role in education: from the Reagan to the Obama administration. Annenberg Institute for School Reform: Voices in Urban Education, 24: summer, 6-14. Retrieved from: http://www.annenberginstitute.org/VUE/wpcontent/pdf/VUE24_Sunderman.pdf United States Department of Education. (2001). Elementary and Secondary Education Act (No Child Left Behind). Retrieved from: http://schools.nyc.gov/RulesPolicies/NCLB/default.htm United States Department of Education. (2010). Blueprint for Reform: The Reauthorization of the Elementary and Secondary Education Act. Retrieved from: http://www2.ed.gov/policy/elsec/leg/blueprint/blueprint.pdf United States House of Representatives, 103rd Congress. (1993). Goals 2000: Educate America Act (HR 1804). Retrieved from: http://www2.ed.gov/legislation/GOALS2000/TheAct/sec102.html

47

MASTERY LEARNING, STUDENT ACHIEVEMENT AND ENGAGEMENT Van Duzer, E. (2006). Overcoming the limitations of the factory system of education. Retrieved from: http://eric/ed/gov/PDFS/ED490530.pdf Zimmerman, B.J. & DiBenedetto, M.K. (2008). Mastery learning and assessment: implications for students and teachers in an era of high-stakes testing. Psychology in the Schools, 45(3), 206-216. DOI: 10.1002/pits.

48