Académique Documents
Professionnel Documents
Culture Documents
General Description
The KeyMath-3 Diagnostic Assessment (KeyMath-3 DA) is an American normreferenced mathematical assessment tool that is used to measure the abilities of students
in regards to their age/grade level in the mathematics curriculum. Dr. Austin J. Connolly
is the author of all three editions. The first edition of the KeyMath Diagnostic Arithmetic
Test (with Nachtman & Pritchett) was published in 1971 while Dr. Connolly was
completing his doctorate of education from Colorado State College. The updated version
titled KeyMath Revised: A Diagnostic Inventory of Essential Mathematics was published
in 1988 as a more user-friendly version for the examiner and examinee (Rosli, 2011).
The current version of the KeyMath 3 DA test was published in 2007. The KeyMath-3
Essential Resources (KeyMath-3 ER) was also released in 2007 to provide an
instructional program that corresponds to the KeyMath-3 DA. The previous versions of
KeyMath were extensively used in special-education classrooms (Pearson Assessments &
Information/Clinical Assessments Inc., 2012). However, KeyMath-3 DA, with the
addition of algebra content and progress-monitoring components, is relevant and useful in
a broad range of classroom settings (Pearson Assessments & Information/Clinical
Assessments Inc., 2012).
The KeyMath-3 Canadian Edition (KeyMath-3) was developed after the U.S.
Edition was published in 2007. An independent content review was conducted to ensure
that every item in KeyMath-3 was correlated to at least one curriculum outcome in each
of the four mathematical protocols in Canada: the Western & Northern Canadian Protocol
(WNCP), The Ontario Curriculum (TOC), The Atlantic Provinces Education Foundation
(APEF), and the Quebec Education Program (QC) (Connolly, A.J., 2008). The KeyMath3 is a comprehensive, norm-referenced diagnostic test that is individually administered to
students from ages 5.0 to 17.11 years who are functioning at these instructional levels
(Connolly, A.J., 2008).
The test is available in two parallel forms, Form A and Form B, to allow
monitoring of a students progress over time by retesting in an alternating sequence every
3 months. The two forms contain 372 full-colour test items grouped into 10 subtests that
represent three general math content areas that are recommended to be delivered in the
following order: Basic Concepts (conceptual knowledge), Operations (computational
skills), and Applications (problem solving) (Pearson Canada Assessments Inc., 2012).
The subtests are not timed but the duration depends on the age level of the student.
Examinees in lower elementary need approximately 30 to 40 minutes while older
examinees require 75 to 90 minutes (KeyMath-3 DA Publication Summary Form, 2012).
Each form contains a manual, two easels, a written computation examinee booklet, and a
score summary form for the examiner. The Canadian Edition of KeyMath-3 also has
accompanying Essential Resources lessons and exercises (ER) and the ASSIST software
application, which is a computer-scoring program for KeyMath-3. It is recommended
that examiners of this assessment must have a minimum of a bachelors degree program
that included (a) coursework in principles of measurement and in the interpretation of
tests and (b) formal training in mathematics (Connolly, A.J., 2008). These examiners
may include teachers, special education professionals, educational diagnosticians, and
school psychologists.
Subtest Description
The items in each of the 10 subtests increase in difficulty and the examiner has a
choice to administer the whole battery or individual subtests. If a student is being
examined on all 10 subtests, the Numeration subtest in the Basic Concepts content area
must be completed first in order to obtain a ceiling score for that student to use in each
subtest and category that follows (Connolly, A.J., 2008). The Basic Concepts category
contains the fundamental mathematical knowledge that is needed in order to understand
higher-level subtests in the subsequent categories. Basic Concepts contains five subtests:
Numeration, Algebra, Geometry, Measurement, and Data and Probability. These subtests
are administered using the dual-sided easel that contains the item content for the
examinee on one side and the question, item content, and answer for the examiner on the
other side. The examiner will read the question out loud while the student listens. The
examinee cannot use a paper, pencil, or calculator to help calculate their answer.
Numeration. Numeration measures a students basic understanding of the
classifications of numbers such as whole and rational numbers. Numeration also reviews
rounding one, two, and three-digit numbers, operations with fractions, decimals, and
percents, and advanced numeration concepts such as scientific notation, square roots, and
exponents (Connolly, A.J., 2008). The Numeration subtest must be administered first
since it tests the introductory concepts that are needed in order to understand the higherlevel content on the other 9 subtests.
Algebra. This is a new subtest that was introduced into KeyMath-3 since the
previous versions had little algebra content. The student is first evaluated on prealgebraic concepts such as sorting, classifying, and ordering of numbers using different
categories. Depending on the students grade level, algebraic concepts are also
introduced by looking at operations involving variables, equations, proportions, and
functions (Connolly, A.J., 2008).
Geometry. This subtest is the foundation of the shape and space curriculum
outcomes from elementary to high school math courses across Canada. In this subtest,
examinees must analyze, describe, compare, and classify two and three-dimensional
shapes (Connolly, A.J., 2008). Other topics such as spatial relationships and reasoning,
coordinates, symmetry, and geometric modeling are also included within the items
(Connolly, A.J., 2008). Coordinate geometry and angle relationships between parallel
lines and intersecting transversal lines are also evaluated.
Measurement. Many changes were made in this subtest in the Canadian version
to account for Canadian currency, the use of the metric system, and our use of time zones
across the Canadian provinces and territories. The items require examinees to measure
lengths in metric, select appropriate linear units of length and distances, and determine
volume using linear measurements of three-dimensional shapes (Connolly, A.J., 2008).
Data Analysis and Probability. This subtest measures an individuals ability to
collect, display, and interpret data as well as his or her understanding of the concepts
associated with chance and probability (Connolly, A.J., 2008). Examinees must be able
to read and interpret tables of data, estimate quantities, and interpret tally charts.
The Operations content area contains three subtests: Mental Computation and
Estimation, Addition and Subtraction, and Multiplication and Division. These subtests
measure an individuals written and computational skills using the four basic
mathematical operations of addition, subtraction, multiplication, and division.
Mental Computation and Estimation. This subtest is administered using an easel.
He or she must listen to the question and look at the item information given on the easel
and compute the answer mentally. The items contain addition, subtract, multiplication,
and division questions of one, two, and three-digit numbers, fractions, mixed numbers,
decimal values, and integers (Connolly, A.J., 2008). Emphasis is placed on the students
ability to answer these questions accurately and as quickly as possible.
three consecutive correct responses before the first incorrect response. The ceiling set is
four consecutive incorrect responses. The raw scores for each subtest are calculated by
recording the ceiling item and subtracting by the number of errors in the subtest.
The KeyMath-3 Manual contains Normative and Interpretive Tables of Scale and
Standard Scores, Confidences Interval-Values, Grade and Age Equivalents, Percentile
Rankings, and Area Comparisons on the standard normal distribution curve. These tables
correspond to the raw scores for each subtest and are categorized separately by norms
(age or grade) and by what time of the year the student is being tested (fall or spring).
When scoring, the administrator can select the appropriate norm (age or grade) and the
appropriate subtest confidence level (68% or 90%) as well as the area and total test
confidence level (90% or 95%). The Appendices A (for Form A) and B (for Form B) in
the manual contain the tables needed to locate the derived scale score from the raw scores
in each subtest (Tables A.1 A.3 or B.1 B.3). Once these scale scores are obtained, the
Confidence-Interval Value can be found under Tables A.4 A.6 or Tables B4. B.6. The
Confidence Interval can be calculated by adding and subtracting the Confidence-Interval
Value to the Scale Score. Tables A.7 A.8 and B.7 B.8 contain the grade/age
equivalent of the students that correspond to the raw scores he/she obtained on each
subtest. The grade/age equivalent number allows the examiner to see what level the
student is at for each subtest in comparison to the standardized population.
The Score Summary Form also contains a Score Profile page where the
administrator can plot the students scale score and corresponding confidence interval
(based on a 68% or 90% confidence interval) on ten number lines for each subtest. At the
top of the score profile page is a standard normal distribution curve that shows where the
students scores are located in regards to the mean score. The examiner can visually
determine how many standard deviations above or below the mean the student is in each
subtest category. A student who is between 2 and 3 standard deviations below the mean
is categorized as Well Below Average, 1 to 2 standard deviations below the mean is
Below Average, 1 standard deviation below and above the mean is Average, 1 and 2
standard deviations above the mean is Above Average, and 2 to 3 standard deviations
above the mean is Well Above Average. The Score Profile page provides
administrators and parents/guardians a visual diagram of their childs progress in each
strand in the math curriculum.
Behavioral observations are also collected during testing to interpret the students
focus, attitude, and work ethic throughout the test. Written observations are made and the
examiner can rate the four behavior categories in the Behavioral Observations During
Testing section in the Record Form (Connolly, A.J., 2008). This feedback may be useful
to administrators and parents about behavioral factors that may be relevant. Item and
functional-range analysis of item scores also allow parents to see what mathematical
concepts and skills their child has acquired and what range of grade-level curriculum they
are capable of comprehending. If the student is evaluated again in the future, a Growth
Scale Value (GSV) Chart can be constructed to show how much developmental progress
that student is making in relation to age/grade levels of the population norm-group.
The ASSIST scoring software for KeyMath-3 allows examiners the opportunity to
enter the students results for each item and it will automatically calculate the raw scores,
Confidence Intervals, and Grade/Age Equivalents for each subtest. GSV charts can also
be created on the computer using this software. ASSIST allows less chance for human
error in computation and interpretation of these results. The administrator is also saved
time from having to do all of the mathematical calculations and looking up results on the
10
tables.
Technical Adequacy
Standardization. A standardization process was developed by Psycan to ensure
that a proper cross section of students across Canada would be included in the norming
procedures based on the Canadian population by province and by urbanization (Connolly,
A.J., 2008). The proposed sample plan by Psycan was to test 1600 students by
Urbanization and Gender. Students across the ten provinces would be sampled with a
larger portion coming from Ontario and Quebec due to higher populations in those
provinces. A total of 41 examiners were trained and assigned to 71 schools selected to
participate in the norming study in 2008 (Connolly, A.J., 2012).
1638 students from kindergarten to grade 12 were sampled with 51 urban males,
48 urban females, 14 rural males, and 13 rural females for a total of 126 students per
school across the provinces (Connolly, A.J., 2008). Approximately half of the students
completed Form A and the other half completed Form B. English and French
assessments were required to account for Francophone students. Approximately 200
students were retested with the alternate form 4 weeks later (Connolly, A.J., 2008).
Canadian socio-economic and cultural diversity variables were not controlled in the
selection of schools to participate in this standardization study. Age was also not
controlled since a random sample of students was collected by grade (Connolley, A.J.,
2008). Pearsons Norms Development Team received all test materials for analyzing all
of the data and developing the normative and interpretive tables for Forms A and B for
KeyMath-3 (Connolly, A.J., 2008).
Reliability. The reliability testing information provided in the Canadian version
of the KeyMath-3 manual is from the American Edition of the KeyMath-3 DA. Internal-
11
consistency reliability of the KeyMath-3 was evaluated using the split-half method, in
which the items of a subtest are separated into two halves that are matched on item
content and difficulty (Connolly, A.J., 2008). The correlation between scores on the
halves is adjusted by the Spearman Brown prophecy formula to estimate the correlation
that would occur between two full-length forms (Connolly, A.J., 2008, p. 316).
Coefficient alpha could not be used for internal consistency with KeyMath-3 due to the
basal and ceiling rules allowing the examiner to adjust the KeyMath-3 subtests to each
examinees ability. With the split-half method, the examinees performance on each half
is converted to an ability score, using the item difficulties from the Rasch calibration of
the entire subtest, and it is these ability scores that are correlated and adjusted, using the
Spearman Brown formula, to produce the reliability estimate (Connolly, A.J., 2008, p.
316). The reliability estimates are based on actual item scores.
Internal-consistency reliabilities were computed for the norm samples for each
grade/season or year of age. Forms A and B were administered to approximately half of
this sample and their reliabilities were calculated separately. The reliability scores for the
subtests were mainly in the .80s, while several subtests had reliability in the .90s (grade 5
and higher in Numeration, Algebra, Mental Computation and Estimation, and
Multiplication and Division) and other reliabilities were in the .70s (subtests in the lowest
grades and ages and several in Grades 10 through 12 where there was a restriction in
score range due to content difficulty). Internal-consistency reliabilities of the area scores
are almost all in the upper .80s or .90s (Connolly, A.J., 2008). Some reliabilities were
shown between .70s to mid-.80s where there was a restriction in score range caused by
content difficulty.
Alternate-form reliability was utilized on KeyMath-3 by distributing Forms A and
12
13
325).
Validity. The Canadian developers of the KeyMath-3 assessment validated all of
the items to ensure that they matched the 4 Canadian math protocols, with suitable reallife contexts (Connolly, A.J., 2008). Canadian math consultants were used to review the
test items and connect it to their current curriculum. However, the validity testing
information provided in the Canadian version of the KeyMath-3 manual is from the
American Edition of the KeyMath-3 DA. In the United States, 400 educational
practitioners and several consultants were involved in designing and aligning test items
(Connolly, A.J., 2008).
The KeyMath-3 DA used several different forms of construct validity for their
assessment tool. An analysis of average performance across grades was provided in order
to determine whether the trend conforms to the expected outcomes (Connolly, A.J.,
2008). Statistical analysis of the relationships between both the instruments subtest and
composite scores on other tests that measure related content was used to reveal the
consistency of correlation patterns between concept and skill (Connolly, A.J., 2008).
Stability between these scores was found with correlations between .60 and .90. Content
area scores were .80 and total test was .90. The accuracy of decisions made based on
using test scores was also reviewed by comparing scores obtained by clinical groups with
those obtained in the overall population (Connolly, A.J., 2008).
Correlations with other tests were calculated with the KeyMath Revised, The
Kaufman Test of Educational Achievement, Second Edition, Iowa Tests of Basic Skills,
Measures of Academic Progress, and the Group Mathematics Assessment and Diagnostic
Evaluation. These correlation scores ranged between .60 to .90 (Connolly, A.J., 2008).
Studies were also conducted of the correlations among the special populations of students
14
15
English.
While the assessment tool has shown to be very effective in the Canadian school
population, there is no reliability or validity testing on the Canadian population with its
version of the KeyMath-3. The only testing information provided in the manual is from
the United States. No paper and pencil can be used on any of these assessments except
for the Written Computation and Applied Problem Solving subtests. This could result in
issues for students who have auditory issues or who learn better by writing. Students
who have trouble speaking English, French, or Spanish may also experience difficulty
understanding these questions, especially the Mental Computation subtest where speed an
accuracy is a factor. Proficiency in English is important in order to succeed at math (Alt,
M., Arizmendi, G.D., Beal, C.R., & Hurtado, J.S., 2013). Immigrant students may also
struggle on Canadian content questions that test their knowledge on our currency, time
zones, and the metric system. Also, an important accommodation for students who are
weak in math is the use of a calculator. Unfortunately the only subtest that allows the use
of a calculator is the pplied Problem Solving subtest. This restriction can be stressful on
a student who does not have strong mental math skills.
Summary
KeyMath-3 allows an educator and parents to see what possible deficiencies/
strengths a math student has in the different curriculum objectives in their grade/age
level. It also allows a comprehensive overview of where a child is at in terms of their
mathematical ability in comparison to a standardized Canadian sample for all of the grade
levels. Dr. Connolly developed this assessment to try and help students to improve on
areas of weakness in order to do well on standardized achievement tests. He believed
that in order to create a successful academic assessment, it needed to satisfy three
16
essential student needs: students must enjoy the experience, to apply real thought to a
situation, and all students need to experience ongoing success at their appropriate skill
level (Connolly, A.J., 2008). The KeyMath-3 has colourful pictures and a variety of
questions that cover a lot of different areas to keep a students interest throughout the test.
The subtests provide lots of real-life examples and application word problems that allow
a student to think logistically about that they would do in those situational-type examples.
KeyMath-3 ER also allows practice for students who are struggling at concepts to try and
improve on their results. The two forms of the assessment allow retesting, which can
provide a chance for students to improve on their score and obtain success. This allows
students to not give up on themselves and to try and succeed in mathematics rather than
to simply give up.
17
References
Alt, M., Arizmendi, G. D., Beal, C. R., & Hurtado, J. S. (2013). The effect of test
translation on the performance of second grade english learners on the KeyMath3.
Psychology in the Schools, 50(1), 27-36. doi:10.1002/pits.21656
Connolly, Austin J. (2008). KeyMath-3 Diagnostic Assessment Canadian Edition
Manual. Richmond Hill, Canada: Psycan Corporation.
KeyMath-3 DA Publication Summary Form. (2012). Retrieved 30 May 2013, from
http://www.pearsonassessments.com/hai/images/pa/products/keymath3_da/km3da-pub-summary.pdf
Maccow, Gloria. (2011). General Overview of KeyMath-3. In Pearson Assessments.
Retrieved 25 May 2013, from
http://www.pearsonassessments.com/hai/images/PDF/Keymath-3Handout.pdf
Pearson Assessments & Information/Clinical Assessments Inc. (2012). Retrieved 30
May 2013, from http://psychcorp.pearsonassessments.com/HAIWEB/Cultures/enus/Productdetail.htm?Pid=PAaKeymath3
Pearson Canada Assessments Inc. (2012). Keymath-3 diagnostic assessment: Canadian
edition. Retrieved 30 May 2013, from http://pearsonassess.ca/haiweb/Cultures/enCA/Products/Product+Detail.htm?CS_ProductID=KEYMATH-3-DIAGNOSTICASSESSMENT&CS_Category=educational-math&CS_Catalog=TPC-CACatalog
Rosli, R. (2011). Test review: A. J. Connolly KeyMath-3 diagnostic assessment:
Manual forms A and B. Minneapolis, MN: Pearson, 2007. Journal of
Psychoeducational Assessment, 29(1), 94-97. doi:10.1177/0734282910370138