Académique Documents
Professionnel Documents
Culture Documents
DOI 10.1007/s10649-015-9594-2
Abstract This paper examines the contribution of quality mathematics teaching to student
achievement gains. Quality of mathematics teaching is assessed through teacher demonstration
of the five strands of mathematical proficiency, the level of cognitive task demands, and
teacher mathematical knowledge. Data is based on 1907 grade 6 students who sat for the same
test twice over an interval of about 10 months. The students were drawn from a random
selection of 72 low- and high-performing primary schools. Multi-level regression shows the
effects of quality mathematics teaching at both individual and school levels, while controlling
for other variables that influence achievement. Results show that students in low-performing
schools gained more by 6 % when mathematics instruction involved high-level cognitive task
demands, with two thirds of all the lessons observed demonstrating the strands of mathematics
proficiency during instruction. The implication to education is that quality of mathematics
instruction is more critical in improving learning gains among low-performing students.
Keywords Mathematics . Proficiency . Knowledge . Cognitive . Demand . School
1 Introduction
In Sub-Saharan Africa, teachers have a huge task in improving the scholastic outcomes of students
who come from different social backgrounds. The average performance in quality education of
African countries appears much poorer than elsewhere in the world. For example, the five countries
At the time of writing this paper, James Ciera, Peter K. Musyoka and Moses Oketch were working at the African
Population and Health Research Center.
112
(South Africa, Botswana, Ghana, Morocco, and Tunisia) that participated in the Trends in
International Mathematics and Science Study in 2003 were ranked in the last seven of the 45
participating countries. However, it is notable that there exist wide variations in the quality of
education within African countries, which underlines the need to consider the situation at the country
level. One way of improving the quality of education is through quality teaching. Most studies on
quality of teaching and how it influences student achievement are to be found outside Africa.
In Kenya, the Free Primary Education (FPE) program was introduced in 2003 and brought to
public schools over one million children (Government of Kenya, 2005). Consequently, class sizes as
high as 80 exist in public primary schools (Ngware, Oketch, & Ezeh, 2011). However, public
expectations of teachers to produce better grades on examinations done by their students remain high
despite large class sizes. Within the context of FPE, there exist schools in the same neighborhood
that persistently perform well and others that persistently perform poorly in standardized national
examinations. Examining the quality of teaching will improve our understanding of what happens in
the classroom that may explain differences in performance among schools. Teacher-classroom
interactions that aid student learning are often complex processes that hinge on interpersonal and
pedagogical awareness. The teachers pedagogy and interactions with students at the classroom level
can determine how much is learned (Hattie, 2009; Morrison, Bachman, & Connor, 2005).
Research on teaching demonstrates that teaching quality is a critical policy issue in
education reforms, particularly given the investment in the teaching force in low resourced
environments. Teachers are one of the most important school-based factors influencing student
achievement (Hattie, 2009; Rice, 2003). While researchers and policymakers agree on the
importance of teachers in the process of learning, there exists no consensus on how to best
measure quality of teaching. According to Walshaw and Anthony (2008), understanding what
quality mathematics pedagogy looks like is still in its formative stages. In this paper, we
examine the effects of quality of teaching mathematics at the classroom level on student
achievement growth. We do this by analyzing the relationship between teacher mathematical
proficiency, level of cognitive demand of a mathematics task and teacher mathematical
knowledge, and student achievement measured by mathematics scores. Mathematical proficiency refers to the interwoven and interdependent strands necessary for students to learn
mathematics. They include conceptual understanding, procedural fluency, strategic competence, adaptive reasoning, and productive disposition (Kilpatrick, Swafford, & Findell, 2001).
Cognitive demand of a task is important because the level and kind of thinking in which
students engage determines what they learn (Stein et al., 2009). Teacher mathematical
knowledge includes content knowledge, pedagogical knowledge, and pedagogical content
knowledge. A teacher who has a good command of mathematical knowledge has a range of
teaching strategies at his or her disposal while faced with different situations in a classroom
(Darling-Hammond & Bransford, 2005; Kilpatrick et al., 2001; Shulman, 1986).
By examining the teaching of mathematics, this paper contributes to the debate on quality
of teaching and learning achievement. This paper also contributes to filling the gap that exists
in Sub-Saharan Africa on research evidence of what is happening in the classroom. The main
research question answered in this paper is: What is the effect of teachers mathematical
proficiency, level of cognitive demand of a mathematics task, and teacher mathematical
knowledge on students achievement?
2 Literature
Literature on teaching quality focuses on teachers content knowledge, pedagogical content
knowledge, and pedagogical skills exhibited in the class (Carnoy et al., 2008; Georges,
113
Borman, & Lee, 2010; Marshall, 2009; National Research Council, 2001; Ngware, Oketch,
Mutisya, & Kodzi, 2010; Sorto, Marshall, Luschei, & Carnoy, 2009). Literature on quality of
education includes other teacher attributes such as attitudes, background characteristics, and
instructional practices (Akiba, LeTendre, & Scribner, 2007; Harris & Sass, 2007;
Phillips, 2010; Walshaw & Anthony, 2008). Studies by Akiba et al. (2007) and
Rockoff (2004) suggest that overall, teaching quality indicators are positively related
to student achievement. However, as observed by Palardy and Rumberger (2008),
teacher instructional practices have the most proximal association with student
achievement. On one hand, this implies that instructional practices have direct influence on student learning. On the other hand, teacher background characteristics and
teacher attitudes influence learning indirectly through association with instructional
practices. A 46-country study by Akiba et al. (2007) suggests that countries with
better teacher quality produce higher mathematics achievement. The study further
shows that larger opportunity gaps in access to qualified teachers did not predict
achievement gaps between high socio-economic and low socio-economic students.
Improvement of teaching quality is the driving force for improving student achievement and, thus, promoting a nations economic competitiveness (Akiba et al., 2007).
A major finding of qualitative studies on mathematics instruction is that the range
of teaching strategies, alternative mathematical representations, and explanations at the
disposal of a teacher during instruction is largely dependent on conceptual understanding of the subject (Baumert et al., 2010). According to Baumert et al. (2010),
inadequate knowledge limits a teachers capacity to explain and represent the content
to students in a sense-making way. An emerging body of literature on teaching
mathematics has coalesced on five interdependent strands of proficiency in mathematics (Carnoy et al., 2008; National Research Council, 2001; Sorto et al., 2009). The
strands provide a comprehensive view of successful mathematics learning and are
necessary for students to learn mathematics. The National Research Council (2001)
describes the five strands as:
1. Conceptual understandingcomprehension of mathematical concepts, operations, and
relations
2. Procedural fluencyskill in carrying out procedures flexibly, accurately, efficiently, and
appropriately
3. Strategic competenceability to formulate, represent, and solve mathematical
problems
4. Adaptive reasoningcapacity for logical thought, reflection, explanation, and
justification
5. Productive dispositionhabitual inclination to see mathematics as sensible, useful, and
worthwhile coupled with a belief in diligence and ones own efficacy (p. 116)
Stein et al. (2009) argue that learning opportunities are not created simply by good
classroom management practices; instead, the level and kind of thinking in which the
students engage determine learning gains. Mathematical instructional tasks can be
examined in terms of the cognitive level a learning task demands. According to
Stein et al. (2009), cognitive demand refers to the kind and level of student
thoughts required to engage with and work out mathematics tasks. Carnoy et al.
(2008) note that the level of cognitive demand is an aspect associated more with
the student even though the teacher controls and directs the required level for his or
her students.
114
Stein et al. (2009) classify cognitive demand of mathematics tasks to include (see
Appendix 1 for sample items):
1. Memorizationrecollection of facts, formulae, or definitions
2. Procedures without connectionsperforming algorithmic type of problems that have no
connection to the underlying concept or meaning
3. Procedures with connectionsuse of procedures with the purpose of developing deeper
levels of understanding concepts or ideas
4. Doing mathematicscomplex and non-algorithmic thinking where students explore and
investigate the nature of the concepts and relationships (pp. 16, 21)
A Panamanian study by Sorto et al. (2009) found that lessons were too short to engage
students with activities that are associated with higher levels of mathematical proficiency and
cognitive demand. In a South African study by Carnoy et al. (2008), the majority (77 %) of
lessons require students to simply recall rules and definitions or perform algorithms with no
connection to underlying concepts.
What we learn from the literature is that quality of instruction matters in student achievement. Furthermore, the literature suggests that the effect of instructional quality on learning
achievement differs with the entry behavior of students and context. This evidence is important
in informing teaching practices and policies for quality education. However, the majority of
studies focus on developed countries and/or emerging economies. There is little literature on
instructional quality and how the two are related to student achievement in Sub-Saharan
Africa, particularly in Kenya. Little is known about the effects of mathematics instructional
quality on student achievement gains in primary schools in Kenya. Furthermore, a review of
the literature on teaching quality shows just how little attention this phenomenon has received
(Walshaw & Anthony, 2008).
3 Conceptualization
The paper uses a value-added approach to investigate the influence of quality of teaching on
learning gains. A value-added approach was chosen for two main features: First, the dependent
variable is designed to measure the amount of change that occurs in learning during the period
when students are in classrooms under study (i.e., gain score), and second, measures of change
were adjusted for differences across classrooms in the students prior achievement (entry
behavior), students socio-economic background, and other school factors (Harris & Sass,
2007; Rowan, Correnti, & Miller, 2002). According to Rowan et al. (2002), the aim of a valueadded model is to approximate the size of variance changes in student achievement within
classrooms after controlling for the effects of other variables.
We measure quality of instruction by using a framework that enables us to observe the five
strands of mathematical proficiency as defined in the literature, levels of cognitive demand of a
task, and teachers observed mathematical knowledge. Students in a classroom have a wide
range of abilities and learning styles. Suh (2007) argues that while some students enjoy
mathematics, others show signs of avoidance and even mathematics phobias. Not only do
students who succeed with mathematics exhibit the traits measured by the five strands, but
their teachers also demonstrate the strands in teaching practice (National Research Council,
2001; Walshaw & Anthony, 2008).
From this brief background, we conceptualize the effects of quality teaching as shown in
Fig. 1. Figure 1 pulls together various factors including student characteristics, teacher factors,
115
INDIVIDUAL LEVEL
Quality of instruction
Teacher Mathematical
Knowledge (observed)
Mathematic
Proficiency
Level of cognitive
demand of
mathematics tasks
during instruction
Others
School/teaching culture
(absenteeism), class-size,
teacher SES, school rank
& SES, lesson duration,
content
Learning
achievement growth
(IRT-scale score
gains)
Others
peer group score,
seat position,
language used at
home, etc.
Fig. 1 Multi-level framework to test the effects of quality of instruction on student learning
and school/classroom context features that are the focus of this paper. IRT scale gain score was
the dependent variable and with the IRT round one scores adjusted for initial academic
performance that can probably be the source of changes in learning achievement. Figure 1 is
a two-level analysis (students and schools) that assumes classrooms and schools are the same
(one sixth grade teacher per school). Therefore, we use cluster correction in STATA to correct
standard errors in the regression analyses. From Fig. 1, we estimate an educational production
function (EPF), as described in Appendix 2.
4 Methods
Data used in this paper is from a larger study carried out in 2010 by the African Population and
Health Research Center; the data has been analyzed to respond to different research questions
(see, for example, Ngware, Ciera, Musyoka, & Oketch, 2013; Ngware, Oketch, Mutisya, &
Abuya, 2010). For the purpose of selecting the low- and high-performing districts and schools,
the Kenya Certificate of Primary Examination (KCPE) results of the last four available years
(20022005) were used to rank districts and schools. School performance in national examinations (a proxy indicator for student achievement) in Kenya varies by district. Some districts
are persistently at the top of the league tables while others at the bottom. Based on the
distribution of school mean scores in a district, schools were categorized as low performing
and high performing.
Six districts were selected, two from those that were consistently ranked in the bottom 10 %
in KCPE examinations over the past 4 years, two from those that were consistently ranked in
the middle, and another two from those that were consistently ranked in the top 10 % over the
116
same period. Seventy-two schools, 12 in each of the six districts, were selected randomly for
the studysix that consistently ranked in the bottom 20 % and six that consistently ranked in
the top 20 %. A further selection criterion ensured a mix of rural, suburban, and urban schools.
Data for this paper were collected from 72 head teachers, 72 mathematics teachers, and 1907
grade 6 students who sat for the same mathematics test, administered by the study team, in
rounds 1 and 2.
To collect data, three surveys and two assessment tools were developed and pre-tested to
improve their validity and reliability. First was a head teacher questionnaire that solicited
information on school management, staffing, enrollment, and parental participation in
school affairs. This was administered to the head teachers. Second was a teacher questionnaire that solicited information on demographics, qualifications and training, discipline, and
syllabus coverage. This was administered to teachers who taught mathematics to the
participating students. Third was a student questionnaire that collected information on
socio-economic backgrounds of the grade 6 students and their perceptions of the school
environment. This questionnaire was administered to grade 6 students in the selected
schools.
The assessment tools included a grade 6 mathematics teacher test and a student mathematics test for grade 6 students. The teacher mathematics test was a 13-item multiple choice tool
that assessed teachers content knowledge, pedagogical knowledge, and pedagogical content
knowledge. A shorter test for teachers was administered to ensure completeness of the
responses as a lengthy test would deter them from responding to all items. The items were
based on what primary grade 6 mathematics teachers are expected to know in order to teach
grade 6 mathematics. Each of the test items was scored either correct (1) or incorrect (0). The
total raw scores were converted into percentages to provide a measure of teacher mathematical
knowledge. In the case of the student, the student mathematics test was a 40-item multiple
choice tool that assessed students in the five curriculum outcome areas and their ability to carry
out mathematic tasks of different cognitive demand. The five curriculum outcome areas
specified by the Kenyan primary school curriculum included number concepts and operations,
patterns and algebra, measurements, geometry, and basic statistics (Kenya Institute of
Education, 2002). The cognitive demand of mathematics tasks was at four levelsmemorization, performing routine procedures, performing complex procedures, and problem solving.
Items that fall under the first two categories have low cognitive demand, while those that fall
under the last two have higher cognitive demand (Stein et al., 2009). Each of the student test
items was scored either correct (1) or incorrect (0). The total raw scores were converted into
percentages to provide a measure of student achievement in mathematics. Overall, the
questionnaire return rate was 98.6 %.
4.1 Video analysis
Lesson observations were conducted using a high-quality video camera and an observation
checklist. A video analysis rubric was developed to systematically analyze the video recordings. The procedure draws upon classroom interaction research, notably the work of
Chesterfield (1997), Sorto et al. (2009), and a classroom interaction study in South Africa
(Carnoy et al., 2008). The rubric was adapted to suit the study objectives by splitting the broad
activities into readily observable tasks and including additional questions to assess the overall
student-teacher interactions and classroom physical environment. The rubric was also pretested to improve reliability. Videos were systematically analyzed for quality mathematics
teachingstrands of mathematics proficiency, levels of cognitive demand of tasks, and
teachers mathematical knowledge.
117
To improve on the quality of lesson recordings, four research assistants were trained in the
optimal ways of filming using high-quality video equipment. Two teacher trainers with
extensive experience in teacher training and pedagogy analyzed the videos with an external
validation of their analyses conducted by an expert in video analysis from another ongoing
African classroom-based research study. The two internal experts first analyzed each video
separately and then jointly, each providing his/her interpretation of what was observed. The
analysis by the external expert did not significantly differ from that of the internal experts.
4.2 Variable descriptions
In this paper, the dependent variable is gain score while the main explanatory variable is
quality of mathematics teaching. Our focus is on understanding quality of teaching inside the
classroom and how it influences learning. Reviewed literature measures quality using learning
outcome indicators (e.g., see Hattie, 2009). We adapt a similar approach by using student gain
scores in mathematics. It is the variability of what happens in the classroom that largely
explains student test scores (Hattie, 2009; Palardy & Rumberger, 2008; Stein et al., 2009).
Based on this argument, we use indicative measures of what is happening in the classroom as
proxy of quality of teaching. Following the relationship depicted in Fig. 1, we hypothesize that
quality of teaching explains student gain scores. These variables are defined as follows:
Gain score The difference between student scores in rounds 1 and 2 of test administration.
Tests 1 and 2 had the same test items but were administered 10 months apart. For the purposes
of fitting a regression model, Item Response Theory (IRT) scale procedures were used to
compute a gain score from the raw scores for each student. We then computed gain scores from
the IRT scale scores by simply subtracting each students IRT mathematics scale score in test 1
from their score in test 2. This gave us the IRT scale score points students gained between
rounds 1 and 2.
Quality of teaching mathematics Quality of teaching mathematics refers to the extent to which
the following aspects of mathematics instruction, described earlier in the literature, were
present in a lesson: (i) strands of mathematics proficiency, (ii) levels of cognitive demand of
a mathematics task, and (iii) teacher mathematical knowledge. This last aspect refers to the
amount and organization of mathematics knowledge in the mind of the teacher, subject matter
knowledge for teaching including an understanding of what makes the learning of specific
topics easy or difficult, and curricular knowledge. To assess the teachers mathematical
knowledge, we analyzed his or her demonstration of an understanding of content knowledge
(CK), pedagogical knowledge (PK), and pedagogical content knowledge (PCK). In
Appendix 1, we present a sample of teacher test items classified as PK or PCK.
Scores were generated to assess each aspect of mathematics teaching quality (mathematics
proficiency, cognitive demand, and teacher mathematics knowledge). For example, scores of
0, 1, 2, and 3 indicated that features of teaching quality were absent, present at most twice,
present 3 or 4 times, and present for at least 5 times, respectively. The score for each aspect of
teaching was weighted and summed separately. The score range for mathematics competence
was 055; the range for cognitive demand was 030 while the range for mathematics
knowledge was 09. Then, the overall score in each category was aggregated into one of
three broad categoriesweak, moderate, or strong. Appendix 1 presents a sample of classroom mathematic tasks classified as weak, moderate, or strong. These three categories were
computed based on three equal ranges between the minimum and the maximum scores of each
aspect of teaching quality. For example, the weak category had the lowest scores (bottom 1/3)
118
while strong category had high scores (scores placed in the upper 1/3 of the score range). The
scores for teacher mathematics knowledge were skewed towards the right such that all the
weights were in the moderate and strong categories with none in the weak category. Hence, we
re-labeled the moderate and strong categories simply as low and high, respectively. To assess
the net effect of the quality of teaching on the gain score, we controlled for other factors that
were used in the analysis. Table 1 describes how all the variables in our analysis were
operationalized.
5 Results
5.1 Descriptive statistics
The descriptive results are presented in three parts: (i) Overall, combined results for all schools,
(ii) top performing schools, and (iii) bottom performing schools. This separation is important
because the schools were selected on the basis of performance. The overall mean gain score for
all schools was 0.582. The mean gain score for high-performing schools was 0.645 and that for
the low-performing schools was 0.492, a difference of 0.153 equivalent to a higher gain of
31 %. The gain scores for the top schools range from 2.1 to 4.0 while the range for the bottom
schools is 2.1 to 2.8, an indication that top schools have gain scores reaching higher levels
compared to the bottom schools. Table 2 presents the overall mean gain score, percentages,
and significance measures (t test p values) for teaching quality indicators.
For mathematics proficiency and cognitive demand, the bottom performing schools show
higher differences in gain scores compared to the top performing schools (see Appendix 3).
For example, in the bottom schools, the Bmoderate^ and Bstrong^ categories of cognitive
demand show significantly larger means compared to the weak category (p values=0.045 and
0.075, respectively). Similarly, for mathematics proficiency in the bottom schools, the moderate category shows a significantly larger mean compared to the Bweak^ category (p values=
0.045). In top schools, the moderate category under the aspect of mathematics proficiency had
a non-significant and lower mean gain score compared to the weak category (p value=0.966).
5.2 Multi-level models
To investigate the effects of quality of mathematics teaching on students gain scores, we fitted
a multi-level linear regression model while controlling the clustering observed among students
that belong to the same class. From the descriptive analysis in the previous section, it was
observed that top and bottom performing schools scored differently; hence, we modeled the
data for the entire sample of schools and then modeled data by school category (top and
bottom schools) separately. In each case of modeling, we fitted a multi-level model to assess the
gain score against the main explanatory variables while adjusting for the related student,
teacher, and school characteristics described in the literature as influencing student achievement
(Akiba et al., 2007; Baumert et al., 2010; Carnoy et al., 2008; Georges et al., 2010; Gillies &
Collins, 2008; Goldschmidt & Phelps, 2010; Hattie, 2009; Hungi & Thuku, 2010).
The regression results are shown in Appendix 4. The key explanatory variable is quality of
mathematics teaching, which was assessed through the teachers demonstration of mathematics proficiency, levels of cognitive demand of mathematics tasks, and observed teacher
mathematical knowledge during mathematics instruction. The results from the multi-level
linear models indicate that when we use the combined data for top and bottom performing
119
Description
Level/categories
Students sex
0=female, 1=male
Student age
Continuous variable
Mathematics tuition
0=no, 1=yes
0=never, 1=sometimes,
2=all times
Wealth Index
Continuous (0100 %)
Teachers preparedness
to teach
0=inadequate, 1=adequate
Mathematical proficiency
0=weak, 1=moderate,
2=strong
Cognitive demand
0=weak, 1=moderate,
2=strong
Mathematical knowledge
0=low 1=high
Teacher-specific variables
Non-basic teaching
materials
0=no, 1=yes
Lesson duration
Continuous
Proportion of active
learning time
Percentage
Continuous
Student-teacher ratio
Class size
Continuous
Proportion of content
covered
Percentage
120
Table 1 (continued)
Variable
Description
Level/categories
Percentage
Proportion of sub-topics
covered
Percentage
0=often, 1=sometimes,
2=rarely, 3=never
schools, quality of mathematics teaching has no significant effect on score gains. However, we
found that among the bottom schools, high-level cognitive demand of mathematics tasks
Table 2 Quality of mathematics teaching and school mean score gain
Teaching quality
School category Presence strength Mean IRT score gain (se) % Lessona, b
Math proficiency
All
Weak
Moderate
0.570 (0.080)
0.535 (0.032)
Strong
0.686 (0.083)
14.1
Top
Weak
0.751 (0.116)
16.7
Bottom
Cognitive demand
All
Top
Bottom
a
b
18.3
67.6
Moderate
0.514 (0.057)
61.1
Strong
0.773 (0.073)
22.2
Weak
0.415 (0.072)
20.0
Moderate
0.553 (0.036)
74.3**
Strong
Weak
0.336 (0.067)
0.488 (0.097)
5.7
12.7
Moderate
0.548 (0.040)
49.3
Strong
0.607 (0.046)
38.0
Weak
0.690 (0.218)
8.3
Moderate
0.539 (0.061)
52.8
Strong
0.691 (0.073)
38.9
Weak
0.387 (0.084)
17.1
Moderate
Strong
0.559 (0.050)
0.515 (0.043)
45.7**
37.1*
Low
0.541 (0.065)
23.9
High
0.570 (0.032)
76.1
Low
0.694 (0.113)
19.4
High
0.591 (0.051)
80.6
Low
0.434 (0.061)
28.6
High
0.545 (0.037)
71.4*
121
during instruction significantly improves students score gains by between 5.6 and 5.8 %. In
the bottom schools, other aspects of quality of mathematics teachingmathematics proficiency and mathematical knowledge observedhad negative but insignificant effects on score
gains. In the top schools, all the aspects of quality of mathematics teaching had negative and
significant effects on score gains (see Appendix 4).
In top schools, depending on the emphasis given to a strand by the teacher, mathematics
proficiency reduced the score gain by between 90 and 97 %; using tasks with higher levels of
cognitive demand reduced score gain by between 64 and 73 %, and higher teachers mathematical knowledge observed improved score gains by 90 %. We expected all aspects of quality
of teaching mathematicsmathematics proficiency, cognitive demand, and teacher mathematics knowledge observedto have positive effects on score gains, but this was not the case.
Furthermore, another negative effect was also found between students initial academic ability
(measured by test 1 scores) and score gains, although a positive effect was expected. In the
sample schools, a one percentage point increase in students initial academic ability measured
by test 1 scores reduced score gains by 27 %. When data were disaggregated by school
category, initial academic ability reduced score gains by 21 and 36 % in top and bottom
performing schools, respectively.
We carried out further analysis to test the relative importance of the quality of
mathematics teaching on learning gains. We used the likelihood ratio test based on the
entire school sample dataset as well as the separate data for top and bottom schools.
For the entire school sample, the teaching quality variables were not significant. In
top schools, quality of teaching was significant (chi-square=20.35, p=0.009); and, in
the bottom schools, the quality of teaching was also significant (chi-square =11.69,
p=0.039).
6 Discussion
This paper examined the effects of quality of teaching mathematics on learning achievement in
primary schools. The question to be answered was: BWhat is the effect of teachers mathematics proficiency, level of cognitive demand of mathematics tasks, and teacher mathematics
knowledge on students achievement?^ We use teachers mathematics proficiency, cognitive
demand of mathematics tasks, and mathematics knowledge as indicators of quality of teaching.
This is based on the premise that a teacher who has a good command of mathematics has a
range of teaching strategies at his or her disposal while faced with different situations in a
classroom (Darling-Hammond & Bransford, 2005; Kilpatrick et al., 2001; Shulman, 1986).
Our hypothesis was that quality of teaching has a positive and statistically significant effect on
learning gains. We expected this to be true even after adjusting for other factors that influence
learning achievement (e.g., see Hattie, 2009). After adjusting for other factors that influence
learning achievement (see Table 2), our multi-level regression analysis confirmed the negative
and significant effects of teaching quality on learning gains in top schools. However, in bottom
performing schools, cognitive demand of tasks and mathematics knowledge had positive and
significant effects.
The literature reviewed suggests that quality of mathematics teaching has an impact on
student achievement and score gains (Carnoy et al., 2008; Georges, Borman & Lee, 2010;
Goldschmidt & Phelps, 2010; Marshall, 2009; National Research Council, 2001; Ngware,
Oketch, Mutisya, & Kodzi, 2010; Sorto et al., 2009; Suh, 2007). Most of the studies reviewed
do not report the negative effects found in our study. In Panama and Costa Rica, for example,
Sorto et al. (2009) found that mathematics lessons focus more on procedural fluency than in
122
making connections, reasoning why rules and procedures work, or solving problems. The study
further found that Costa Rican teachers were focused more on developing conceptual activities
with less success. In South Africa, a study by Carnoy et al. (2008) found the pattern of the
proficiency in mathematics somewhat balanced. However, in most of the lessons, procedural
fluency was the dominant strand with another half of the lessons showing aspects of conceptual
understanding and adaptive reasoningthese are aspects of mathematics proficiency.
We think that the negative effects in our study may be explained by student
variability in learning achievement in mathematics. Low scorers in test 1 had more
opportunity or room for improvement. High scorers in test 1 had limited opportunity
or room for improvement due to the ceiling effect. It was possible that higher scorers
in test 1 had already approached their maximum potential in performance, hence the
low gain. It should be noted that most of the test items covered content and
competencies required of those below grade 6. Consequently, the negative relationship
between test 1 scores and score gains (low performers gaining more) could explain
the negative effects of quality of mathematics teaching on learning achievement.
Our alternative explanation of the negative effects is the quality of teaching itself,
as demonstrated in the multi-level analysis as well as the descriptive results. After
adjusting for other factors, the ceiling effects in the top performing schools lead to
negative effects of teaching quality on learning gains. This was possible if teachers
were not adding any value to students who were already good performers. For
example, in top schools, teachers mean score in the mathematics assessment test
was 63 %, while that of their students was 53 %a difference of ten percentage
points. The assessments were not exactly the same. However, they were comparable,
and the one done by teachers assessed content and competencies expected of a grade
6 mathematics teacher. Other variables, such as the students academic entry behavior,
may have had a bigger effect on learning gains than the teaching quality, given the
low mathematics knowledge of the teacher. This supports the findings of Baumert
et al. (2010), who argued that inadequate subject knowledge among teachers limits
their capacity to explain and represent the content to students in a sense-making way.
In bottom performing schools, teachers mean score was 58 % while that of their students was
39 %a difference of almost 20 percentage points. In low-performing schools, a teacher is a
valuable resource in spite of his/her low mathematics knowledge, given the low student learning
achievement. This explains the positive and significant effects observed in low-performing
schools, even after adjusting for other variables.
By examining the teaching of mathematics, this paper contributes to the debate on quality
of teaching and learning achievement. This paper also contributes to filling the gap that exists
in Sub-Saharan Africa on research evidence as to what is happening in the classroom. The
main research question answered in this paper is: What is the effect of teachers mathematical
proficiency, level of cognitive demand of a mathematics task, and teacher mathematical
knowledge on students achievement? The results demonstrate that high-quality mathematics
teaching is absent in primary school classrooms in Kenya. What we observe is a negative
effect of teaching quality on learning gains, particularly in schools with high-performing
students. However, in schools with low-performing students, the observed quality of teaching
seems to be adding some value in learning, simply because the students have low entry
academic behavior. Our hypothesis, therefore, holds for schools with low-performing students.
The studys limitations stem from the single-time observation of the aspects of quality
teaching during mathematics lessons. Multiple observations would have allowed us to improve
the objectivity in measuring teaching quality. One area for further research is in the identification of the key aspects of teaching that best measure quality teaching. This has implications
123
to both policy and practice as efforts to improve teaching effectiveness in low resource
environments could first focus on such aspects.
124
125
126
127
A f H; I; S;
Yi Xi i i 1; 2; N
where i is a vector of error components and i N (0,). The response vector for the
ith student Yi has a multivariate normal density
X
Yi N Xi ;
where is a vector of fixed effects and is the covariance matrix. Since the study
has two time points (rounds 1 and 2), we adopt an unstructured covariance matrix for
the covariance structure. The unstructured covariance matrix offers the most generalized structure that does not assume any prior knowledge of the relationship between
the variables of interest.
128
Appendix 3
Table 3 Mean IRT score gain based on student, teacher, and school characteristics
Category
Mean (SE)
% Lessons
Student characteristics
Have mathematics tuition
No
0.550 (0.023)
44.3
Yes
0.611** (0.020)
55.4
Student gender
Female
0.601 (0.022)
48
Wealth Index
Male
Level 1 (poor)
0.565 (0.021)
0.655 (0.033)
52
20.1
Level 2
0.627 (0.035)
20
Level 3
0.565** (0.034)
19.7
Level 4
0.600 (0.031)
20.2
Level 5 (poorest)
Number of times repeated grade
Never
Once
Twice
Three and above
Never
0.463*** (0.035)
20
0.612 (0.022)
52
0.571 (0.024)
36.6
0.499** (0.044)
0.381** (0.103)
9
2.4
0.481 (0.041)
13.4
Sometimes
0.595** (0.017)
79.6
All times
0.604** (0.062)
6.7
Top
0.645 (0.020)
57
Bottom
0.493*** (0.021)
43
Female
Male
0.560 (0.025)
0.553 (0.025)
32.2
35.6
0.644** (0.028)
32.2
Mixed
Head teacher supervision frequency
Often
0.805 (0.040)
15.4
0.498*** (0.028)
25.4
Rarely
0.518*** (0.037)
18.8
Never
0.581*** (0.023)
40.2
sometimes
Teachers gender
Teachers preparedness
No education
0.640 (0.038)
17.1
Certificate
Diploma
0.559** (0.018)
0.736* (0.049)
73.2
7.5
Degree
0.337** (0.081)
Female
0.647 (0.051)
47
Male
0.575 (0.016)
53
Inadequate
0.647 (0.051)
10.6
Adequate
0.575* (0.016)
89.5
2.2
129
Appendix 4
Table 4 Multiple linear regression model based on all schools and top and bottom performance schools
Variable
Category
Intercept
Teachers characteristics
Mathematical proficiency (ref: weak)
Cognitive demand (ref: weak)
All schools
Top schools
Estimate
Estimate
0.58
1.20**
Bottom
schools
Estimate
2.21
Moderate
0.10
0.97***
0.01
Strong
0.07
0.90***
0.06
Moderate
0.07
0.73***
0.09
0.64**
Strong
0.56**
0.58**
High
0.09
0.90**
0.08
Adequate
0.17*
0.80**
0.1
0.42**
0.74**
1.52*
0.02
0.05*
0.01
Certificate
Diploma
Degree
0.02
0.45**
0.24*
0.05
0.02
0.53**
0.43
0.6
0.02
Student characteristics
0.27***
IRT score_1
Group score for test 1
Have mathematics tuition (ref: no)
0
Yes
0.21***
0
0.02
0.36***
0.01**
0.01
0.02**
0.04**
Male
0.01
0.02
0.01
Once
Twice
0.05
0.11**
0.05
0.15*
0.05
0.08
0.25**
0.19
0.29**
Sometimes
0.11**
All times
0.12*
0.29**
Student age
0.01
0.04
0.15
Male
0.09*
0.05
0.15*
Mixed
0.07*
0.10*
0.06
0.22***
0.01**
0.25**
0.01**
0.16
0.03**
0.01**
0.01***
0.36
0.08
0.19
0.59
0.05
0.42
Sometimes
0.32**
0.25**
0.35
Rarely
Never
0.18
0.16*
0.19
0.01
0.39
0.27
Bottom
0.18**
0.01
Class size
0
0.82
2.22**
2.03**
130
Table 4 (continued)
Variable
Category
All schools
Top schools
Bottom
schools
Estimate
Estimate
Estimate
0.135
0.075
Student variability
0.589
0.598
0.546
Inter-class correlation
0.05
0.016
1693.7
1019.43
607.13
References
Akiba, M., LeTendre, G. K., & Scribner, J. P. (2007). Teacher quality, opportunity gap, and national achievement
in 46 countries. Educational Researcher, 36(7), 369387.
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., et al. (2010). Teachers mathematical
knowledge, cognitive activation in the classroom, and student progress. American Educational Research
Journal, 47(1), 371406.
Carnoy, M., Chisholm, L., Arends, F., Baloyi, H., Hoadley, U., Kivilu, M., et al. (2008). Towards understanding
student academic performance in South Africa: A pilot study of grade 6 mathematics lessons in South Africa.
Pretoria: HSRC. Retrieved from http://nicspaull.files.wordpress.com/2011/04/hsrc-stanford-2008-towardsunderstanding-student.pdf. Accessed January 19, 2015.
Chesterfield, R. (1997). Classroom observation tools. Retrieved 26 February 2015 from http://www.ieq.org/pdf/
Class_ObsTool.pdf
Darling-Hammond, L., & Bransford, J. (Eds.). (2005). Preparing teachers for a changing world: What teachers
should learn and be able to do. San Francisco: John Wiley and Sons.
Georges, A., Borman, K. M., & Lee, R. S. (2010). Mathematics reform and teacher quality in elementary grades:
Assessments, teacher licensure, and certification. Educational Policy Analysis Archives, 18(13). Retrieved
fromhttp://www.redalyc.org/pdf/2750/275019712013.pdf. Accessed January 19, 2015.
Gillies, J., & Collins, P. (2008). Opportunity to learn: A high impact strategy for improving educational outcomes
in developing countries. Washington: EQUIP2, AED.
Glewwe, P. (2002). Schools and skills in developing countries: education policies and socioeconomic outcomes.
Journal of Economic Literature, 90, 436482.
Goldschmidt, P., & Phelps, G. (2010). Does teacher professional development affect content and pedagogical
knowledge: How much and for how long? Economics of Education Review, 29, 432439.
Government of Kenya. (2005). Kenya education sector support programme 2005 - 2010. Nairobi, Kenya:
Ministry of Education, Science and Technology.
Harris, D. N., & Sass, T. R. (2007). Teacher training, teacher quality and student achievement (Working Paper
No. 3). Retrieved from CALDER website: http://www.caldercenter.org/sites/default/files/1001059_Teacher_
Training.pdf. Accessed January 19, 2015.
Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London:
Routledge, Taylor and Frances Group.
Hungi, N., & Thuku, F. W. (2010). Differences in student achievement in Kenya: Implications for policy and
practice. International Journal of Education Development, 30, 3343.
Kenya Institute of Education. (2002). Primary education syllabus (Vol. 2). Nairobi: KIE, Ministry of Education.
Kilpatrick, J., Swafford, J., & Findell, B. (2001). Adding it up: Helping children learn mathematics. Mathematics
Learning Study Committee. Washington: National Academy Press.
Kimani, G. (2005). A study to establish a national assessment system for monitoring of learner achievement in
Kenya. Unpublished report. Nairobi, Kenya: Ministry of Education.
Marshall, J. H. (2009). School quality and learning gains in rural Guatemala. Economics of Education Review,
28, 207216.
131
Morrison, F. J., Bachman, H. J., & Connor, C. M. (2005). Improving literacy in America: Guidelines from
research. New Haven, CT: Yale University Press.
National Research Council. (2001). The strands of mathematical proficiency. Adding it up: Helping children
learn mathematics. Washington: National Academy Press.
Ngware, M. W., Ciera, J., Musyoka, P. K., & Oketch, M. (2013). The influence of classroom seating position on
student learning gains in primary schools in Kenya. Creative Education, 4(11), 705712.
Ngware, M., Oketch, M., Mutisya, M., & Abuya, B. (2010). Classroom observation study. Nairobi, Kenya:
African Population and Health Research Center.
Ngware, M., Oketch, M., Mutisya, M., & Kodzi, I. (2010). Does teaching style explain differences in low and
high performing schools in Kenya? (APHRC Working Paper No. 44).
Ngware, M. W., Oketch, M., & Ezeh, A. C. (2011). Quality of primary education inputs in urban schools:
Evidence from Nairobi. Education and Urban Society, 43(1), 91116.
Nishimura, M., & Yamano, T. (2008). School choice between public and private primary schools under the Free
Primary Education policy in rural Kenya (Discussion Paper 0802). Retrieved from GRIPS Policy
Information Center website: http://www3.grips.ac.jp/~pinc/data/08-02.pdf. Accessed April 13, 2011.
Palardy, G. J., & Rumberger, W. R. (2008). Teacher effectiveness in first grade: The importance of background
qualifications, attitudes, and instructional practices for student learning. Educational Evaluation and Policy
Analysis, 30(2), 111140.
Phillips, K. J. R. (2010). What does BHighly qualified mean for student achievement?^ Evaluating the
relationships between teacher quality indicators and at-risk students mathematics and reading achievement
gains in first grade. The Elementary School Journal, 110(4), 464493.
Rice, J. K. (2003). Teacher quality: Understanding the effectiveness of teacher attributes. Retrieved from http://
www.epi.org/publication/books_teacher_quality_execsum_intro/. Accessed May 27, 2013.
Rockoff, J. E. (2004). The impact of individual teachers on student achievement: evidence from panel data.
American Economic Review, 94(2), 247252. Retrieved from https://www.aeaweb.org/atypon.php?return_
to=/doi/pdfplus/10.1257/0002828041302244. Accessed January 19, 2015.
Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects
on student achievement: Insights from the prospects study of elementary schools (Research Report Series
RR-051). Retrieved from Consortium for Policy Research in Education website:http://hostmaster.literacy.
org/sites/default/files/researchreport/791_rr51.pdf. Accessed January 19, 2015.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4
14.
Sorto, M. A., Marshall, J. H., Luschei, T. F., & Carnoy, M. (2009). Teacher knowledge and teaching in Panama
and Costa Rica: A comparative study in primary and secondary education. Revista Latinoamericana de
Investigacin en Matemtica Educativa, 12(2), 251290.
Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2009). Implementing standards-based
mathematics instruction: A casebook for professional development (2nd ed.). New York: Teachers
College, Columbia University.
Suh, J. M. (2007). Tying it all together: Classroom practices that promote mathematical proficiency for all
students. Teaching Children Mathematics, 14(3), 163169. Retrieved from http://mason.gmu.edu/~jsuh4/
tenure/part4thru8/papers/Tying_It_All_Together.pdf. Accessed January 19, 2015.
Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. New York: Springer.
Walshaw, M., & Anthony, G. (2008). The teachers role in classroom discourse: A review of recent research into
mathematics classroom. Review of Educational Research, 78(3), 516551.