PHYSICAL REVIEW SPECIAL TOPICS - PHYSICS EDUCATION RESEARCH 11, 020106 (2015)

Conceptual problem solving in high school physics

Jennifer L. Docktor,1 Natalie E. Strand,2,3 Jos P. Mestre,2,3,4,* and Brian H. Ross3,5
1
Department of Physics, University of WisconsinLa Crosse, La Crosse, Wisconsin 54601, USA
2
Department of Physics, University of Illinois, Urbana, Illinois 61801, USA
3
Beckman Institute for Advanced Science and Technology, University of Illinois,
Urbana, Illinois 61801, USA
4
Department of Educational Psychology, University of Illinois, Champaign, Illinois 61820, USA
5
Department of Psychology, University of Illinois, Champaign, Illinois 61820, USA
(Received 30 April 2015; published 1 September 2015)
Problem solving is a critical element of learning physics. However, traditional instruction often
emphasizes the quantitative aspects of problem solving such as equations and mathematical procedures
rather than qualitative analysis for selecting appropriate concepts and principles. This study describes the
development and evaluation of an instructional approach called Conceptual Problem Solving (CPS) which
guides students to identify principles, justify their use, and plan their solution in writing before solving a
problem. The CPS approach was implemented by high school physics teachers at three schools for major
theorems and conservation laws in mechanics and CPS-taught classes were compared to control classes
taught using traditional problem solving methods. Information about the teachers implementation of the
approach was gathered from classroom observations and interviews, and the effectiveness of the approach
was evaluated from a series of written assessments. Results indicated that teachers found CPS easy to
integrate into their curricula, students engaged in classroom discussions and produced problem solutions of
a higher quality than before, and students scored higher on conceptual and problem solving measures.

DOI: 10.1103/PhysRevSTPER.11.020106 PACS numbers: 01.40.ek, 01.40.Fk, 01.40.gb

I. INTRODUCTION conceptual knowledge following introductory physics

Physics teaching in both high school and college places
an emphasis on problem solving [18], and although
students demonstrate reasonable competence in traditional
assessments of problem solving skills, there is evidence that
understanding of fairly fundamental concepts is weak or
lacking following completion of introductory courses
[914]. Students in introductory physics courses solve
problems largely using a process termed means-ends
analysis, whereby they search for equations containing
the quantities in a problem and try to reduce the distance
between the goal state and their current state in the solution
process [5,8,15,16]. Students are not taught to solve
problems simply by manipulating equations since instruc-
tors typically mention the concepts and principles that they
are applying, but students rightly perceive the equations
as being central to obtaining quantitative answers and tend
to ignore conceptual information. This approach can be
effective at getting answers, but falls short in understanding
the conceptual underpinnings of the solution process. It is,
therefore, not surprising that students learn or retain little
*
mestre@illinois.edu
Published by the American Physical Society under the terms of
the Creative Commons Attribution 3.0 License. Further distri-
bution of this work must maintain attribution to the author(s) and
the published articles title, journal citation, and DOI.
courses.
Although physics instructors at all levels would agree that
integrating conceptual knowledge with problem solving is a
desirable goal in physics instruction, traditional materials
tend to promote, albeit inadvertently, equation manipulation
at the expense of conceptual understanding. Standard
physics textbooks present equations in terms of general
symbols and elaborate upon what those symbols stand for;
however, there is little guidance for students regarding when
it is useful to apply a particular relation to a problem [15].
Understanding the conditions of applicability for a prin-
ciple and the procedures for determining whether the
necessary conditions have been met are essential for
proficient problem solving, and these conceptual aspects
need to be made explicit during instruction [6,14,17].
The equation-centered approach favored by beginners is
in contrast to the more strategic approach favored by skilled
problem solvers. Skilled problem solvers organize their
solution strategies around major principle(s) or concept(s)
[2,5,18]. For skilled problem solvers, principles or concepts
in memory are also bundled with contexts or conditions in
which they can be applied and with procedures for applying
them [2,15,19]. This type of integration of major ideas,
contexts, and procedures provides skilled solvers with a
hierarchically structured, well-integrated knowledge base
that guides their problem solving [17,18,20]. Since build-
ing such a knowledge base takes considerable time and
effort, an important consideration is whether there is benefit

1554-9178=15=11(2)=020106(13) 020106-1 Published by the American Physical Society

DOCKTOR et al.
in attempting to help students in introductory courses
develop the type of knowledge needed for skilled problem
solving. Several studies suggest an affirmative answer.
One intervention in introductory mechanics with under-
graduates began with a conceptual overview of the material
in qualitative form [21]. Students learned to reason about
phenomena qualitatively, using various representations
(e.g., free-body diagrams and energy bar charts) for the
concepts learned. After students learned qualitative ways of
describing and reasoning about physical phenomena, they
then experienced a second pass at the same content but this
time with mathematical descriptions. They also worked on
multistep problems that required application of knowledge
from different parts of the course, thus integrating knowl-
edge across course components. Additional activities
included problem categorization, where students identified
the major principle(s) or concept(s) needed to solve prob-
lems without actually solving them. Students experiencing
this intervention outperformed students in a traditional
course in conceptual and problem solving assessments,
and were also better able to retain conceptual knowledge
long after the course was over. The same results were
replicated with at-risk students at a different university [22].
Another attempt to integrate conceptual knowledge within
problem solving used a computer-based tool that allowed
users to analyze problems in terms of the principles or
concepts needed to solve them [23,24]. To analyze problems
using the tool, students made selections from a series of
menus, with each subsequent menu becoming increasingly
specific and building on choices made in previous menus.
By selecting appropriate principle(s) or concept(s) (e.g.,
work-energy theorem or conservation of energy), and
specifying initial conditions and problem context (e.g.,
whether or not frictional forces were present, whether a
body possessed kinetic and/or potential energies in some
initial state), conceptual qualitative analyses of problems
could be performed, which in turn resulted in one or more
equations that the tool generated to fit the specific analysis
performed. Compared to students who experienced a more
traditional control treatment, students who used the analysis
tool were better able to categorize problems according to the
underlying principle [25] and to solve problems.
Yet another study [26] conducted in a large introductory
mechanics college course asked that students write
conceptual, qualitative strategies [descriptions of the
principle(s) needed to solve a problem, a justification for
why the principle(s) applied to the problem, and a procedure
for applying the principle(s)] for problems in homework and
exams prior to generating a solution. Course instructors
modeled writing quality strategies in lecture and discussion
sections, and posted solutions for the weekly homework
assignments that displayed both strategies and solutions for
all problems. When compared with students undergoing
traditional instruction, students who practiced strategy writ-
ing displayed benefits in conceptual measures, including
020106-2
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
better ability to categorize problems according to principles,
and better ability to identify the major ideas covered in the
course several months later.
At the high school level, studies of the impact of
integrating conceptual knowledge in problem solving on
students conceptual understanding are rare, although there
is a curriculum [27] that targets the development of physics
conceptual knowledge, and a program [28] specifically
aimed at helping students overcome misconceptions.
However, both are designed in a context outside of problem
solving and with no mathematics. One high school study
[29] did compare the impact of explicit problem solving
instruction to traditional problem solving instruction on
various conceptual and problem solving measures. Both
approaches included five steps:
Explicit problem solving: (a) focus the problem;
(b) describe the physics; (c) plan the solution;
(d) execute the plan; and (e) evaluate the solution.
Traditional problem solving: (a) Draw a sketch;
(b) define known and unknown quantities; (c) select
equations; (d) solve equations; and (e) check the
answer.
The step of focus the problem guides students to state a
general approach in terms of concepts and principles,
however, it does not emphasize the justification for the
appropriateness of those principles in the manner that the
Conceptual Problem Solving method does [30,31].
Although there were some advantages for the explicit
problem solving group, the two groups performed equiv-
alently in tasks requiring conceptual knowledge, such as
planning solutions and conceptual questions.
The current study investigated a question with dual
goals: Can pedagogical approaches that have been shown
to be effective at promoting CPS among college students be
adapted for use in a HS setting in ways that allow flexible
adoption by teachers, and that promote measurable positive
outcomes in students? The need for flexibility in adoption
among high school teachers is important. The college
studies reviewed above that promote conceptual develop-
ment within problem solving were both developed and
implemented by the same faculty member or researcher,
thus creating investment or ownership as well as insuring
fidelity of implementation. On the other hand, it is
unrealistic to have high school instructors adopt wholesale
an approach handed to them by university researchers.
High school teachers face constraints and challenges in
teaching their physics courses that may place limits on the
time they devote to implementing the new approach, as
well as to the style in which they implement it; for example,
the time that can be devoted to implement a new approach
as well as to administer assessments to measure outcomes
was limited. It was impossible, therefore, to design a
classroom-based high school study in which the teachers
adopted or delivered the approach in carefully controlled
ways that one could achieve in the researchers laboratory.
CONCEPTUAL PROBLEM SOLVING IN HIGH PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)

The intervention was designed to provide teachers with
ways of highlighting the role of conceptual knowledge in
problem solving while keeping the implementation of the
intervention flexible, thereby allowing teachers ways of
promoting conceptual problem solving in ways to fit their
curricular demands. Materials were developed to encourage
students to perform qualitative analyses of problems
(similar to the strategies described in Ref. [26]), and then
to use those strategies in formulating quantitative solutions
to problems. The intervention attempted to steer students
tendencies away from equation hunting and toward follow-
ing a general framework for solving problems that began
with identifying principles to apply, justifying why they
could be applied to the problem, and then generating a
solution plan to generate an answer.
The second goal was equally important, namely, meas-
uring whether the approach had an impact on student
outcomes. Assessments needed to be developed that
attempted to measure conceptual understanding as well
as problem solving. The assessments developed took more
time to administer than teachers typically had available to
devote to noncourse-related assessments. Teachers had the
flexibility to decide how much class time they would
devote to administering study assessments subject to their
curricular constraints, as well as to which particular assess-
ments would be administered.
There were additional challenges facing the study.
Whereas almost all the studies cited above evaluating
interventions to promote conceptual problem solving were
done with college students who had declared an interest in
pursuing a STEM major, the high school students in our
study had much more diverse interests and mathematical
preparations. In addition, unlike the college studies cited
earlier where those designing and implementing the inter-
vention were physicists, the high school teachers in this
study reflected the background of teachers nationwide
some were teaching out of field while others had degrees in
physics. Finally, the number of students participating in
each class in our study was small given physics enrollments
at the participating high schools and the high schools
were different enough that we felt it best to analyze each
separately, placing constraints on the interpretation of the
findings. Even though this limitation requires a focus on
trends across schools, we believe the trends are fairly
strong. In summary, the study reported here was challeng-
ing, looking for what were likely to be somewhat subtle
changes in problem solving and conceptual understanding
resulting from an intervention that was implemented in
different ways over different durations by teachers with
varying degrees of physics competence to students with
varying degrees of physics interest and mathematical
preparation.
The next section describes the materials, participants,
and experiment design in more detail. It is then followed by
the study results and a general discussion of the findings.
II. METHOD
A. Materials
1. Learning materials
Conceptual Problem Solving (CPS) is not a curriculum,
but rather is a framework for solving physics problems that
can be easily adapted to existing course materials. The
features of CPS are adapted from Ref. [26] and modified

Problem: At point A on a roller-coaster, a 150 kg car is traveling at 13 m/s and

is 3 m above the ground. Calculate the speed of the car at point B, when it is
5 m above the ground.

Principle:
Conservation of energy: the total mechanical energy of an isolated system (sum of
kinetic and potential energies) is the same in the initial and final states.

Justification:
There are no non-conservative forces doing work on the roller coaster car as it
travels along the track (we neglect air drag and friction). Therefore no energy is
gained or lost and the energy of the car at point A is equal to the energy of the car
at point B.

Plan:
1. Draw a picture and assign symbols for quantities in the problem. Choose a
coordinate system.
2. Write an equation for conservation of mechanical energy. Expand this
equation to include the initial and final kinetic and potential energy terms.
3. Solve for the final speed of the roller coaster car. Substitute values and
calculate a numerical answer.

FIG. 1. Sample problem and strategy (principle-justification-plan).

020106-3
DOCKTOR et al. PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
Plan Step Equation(s) used in step
1. Draw a picture and assign symbols for
quantities in the problem. Choose a
coordinate system.

m = 150 kg mass of car

vi = 13 m/s initial speed of car (at A)
hi = 3 m initial height of car
hf = 5 m final height of car (at B)
vf = ? final speed of car (at B)
2. Write an equation for conservation of E 0 Ei Ef
mechanical energy. Expand this
equation to include the initial and final KE i PEi KE f PE f
kinetic and potential energy terms.
1 2 1 2
mvi mghi mv f mgh f
2 2
3. Solve for the final speed of the roller 1 2 1 2
coaster car. Substitute values and mv f mvi mghi mgh f
calculate a numerical answer. 2 2
1 2
mvi mghi mgh f
vf
2 2
1
m
2
2
vf vi 2 g hi hf
2
vf 13 m / s 2 9.8 m / s 2 3m 5m
vf 11 m / s

FIG. 2. Sample two-column solution.

to be more suitable to a high school student population.
The strategies used in Ref. [26] for the study with university
STEM majors were intended to be prose descriptions of the
principle(s), justification(s), and procedure(s) needed to
solve problems, but no specific guidance was provided for
how to write strategies other than telling them that a good
strategy had to be a logical discussion of how to solve a
problem that contained the three pieces above. For the high
school student population we broke down this process into
more specific stages to provide more guidance to students
on what was expected of them. Further, because writing
qualitative descriptions for solving problems is a high level,
difficult task, we deemed it important to provide as much
structure as possible to help students succeed in writing
strategies, while at the same time not being overly
burdensome in our requirements. The CPS approach that
we finally decided upon contained three separate parts:
Principle (the principle or concept applicable to the
problem), justification (explanation of why the concept
or principle is appropriate), and plan (numbered steps that
provide the recipe for solving the problem and the
equations that do with each step in the plan. Students
carry out their plan for solving the problem by formatting
their solution as two columns: A left column describing the
plan step, and a right column consisting of equations or
mathematics that go along with the step [32]. When the
method is first introduced, scaffolding is provided in the
form of headings or blank worksheet templates that prompt
students to complete each part. A sample strategy is
presented in Fig. 1 and its accompanying two-column
solution is presented in Fig. 2.
It is important to note that students were not required to
write the principle, justification, and plan before carrying
out the quantitative solution on the right-hand side of the
two-column solution, although most students did, as did
instructors when illustrating the approach to students. Here,
as in Ref. [26], the aim was to highlight the role of
conceptual knowledge in problem solving, and this could
be accomplished by writing the principle, justification,
and plan either before or after generating a solution with
equations.
A sample set of problems, strategies, two-column
solutions, and blank worksheet templates were written
by the authors prior to classroom implementation of

020106-4
CONCEPTUAL PROBLEM SOLVING IN HIGH
CPS. Typically 10 or more problems were developed for
each physics topic, exhibiting a range in problem
complexity and difficulty. The topics included motion in
one and two dimensions, Newtons laws, momentum
(Impulse-Momentum Theorem and Conservation of
Momentum), and energy (Work-Energy Theorem and
Conservation of Energy). Upon request from participating
teachers, a few problems were also written for minor topics
including vectors and torque. The files for these problems
and solutions were provided to teachers participating in the
study, who were given the option to use these problems as
written, modify these problems, or to use their own problems
but format them to look similar to the samples.
2. Assessments
Students from the traditionally taught classes and the
CPS classes completed a series of written assessments to
evaluate their understanding of physics concepts and their
problem solving skills (see Supplemental Material [33] for
sample items from the assessments). There were five
different tests administered to assess different aspects of
the students understanding, which we briefly explain here.
Three of the tests (problem solving, conceptual questions,
and categorization) examined important aspects of what
might be learned from CPS, another test examined whether
there was any influence on a different complex skill
(finding errors), and the final test examined students
understanding of the variables in the final equation (equa-
tion instantiation). Examples of each of the tests are
provided in the Supplemental Material [33] and should
be examined when reading these explanations. We return to
these assessments again in the results section.
Problem solving.Items in this test are standard free
response problems, similar to traditional problem solving
measures used in physics courses, that students were asked
to solve. Most physics courses use problem solving as their
primary assessment, so it was deemed critical to examine
problem solving. CPS is meant to help the students to
proceed from a problem to a principle, so should improve
problem solving. Total time: 15 min for 3 items.
Conceptual questions.Items in this test prompt stu-
dents to write free-response explanations for a realistic
physical situation (school A) or answer conceptual ques-
tions in a multiple choice format (schools BC). This test
evaluated whether students could link physical situations to
principles and conceptsa basic goal of CPS. Total time:
15 min for 6 items.
Problem categorization.Items in this test are similar to
the 3-problem categorization tasks used in Ref. [25] but
modified to be grade appropriate. Students must select one
of two problems that would be solved most like a third
model problem. Problems are specifically designed to
vary their match on superficial features (objects or context)
or match on concepts and principles used to solve them.
This test evaluated ability to determine if problems were
020106-5
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
solved with similar principles and whether students were
distracted by the presence of similar surface attributes in
problems. Total time: 25 min for 9 items.
Finding errors.Items in this test contain a problem
statement and worked-out solution that includes a con-
ceptual physics error. Students are asked to identify and
describe the error in writing, and are specifically informed
that there are no mathematical errors in the solutions. This
test measured if students could detect solutions that were
lacking the application of a needed principle or solutions
that applied concepts incorrectly. Total time: 15 min for
3 items.
Equation instantiation.Items in this test include a
problem and worked solution in symbolic form. Students
are asked to assign or match the appropriate values for each
quantity in the final expression. Because the final expres-
sion from the principle is given, it does not require any
decision about the correct principle or justification for it.
Instead, the test requires students to be able to match
particular quantities in the problem to variables in the final
expression. This is important knowledge for getting the
correct answer, but it is not what would be learned by CPS,
and we did not expect this would lead to much difference
between the groups. It is possible that there will be some
small effect because a better understanding of the principles
could also lead to better understanding of what the principle
variables map onto in a problem. Total time: 15 min for
5 items.
B. Participants
Physics teachers from four high schools agreed to
participate in the study, however due to unforeseen circum-
stances that affected the implementation and assessment
procedures, one of the schools was excluded from the data
analysis.1 The remaining schools and CPS teachers will be
referred to as A, B, and C. These teachers were selected in
part because they were teaching nonadvanced placement
classes at their school and had some flexibility in designing
classroom activities.
The particular experiment design at each school was
determined by situational constraints, such as the number
of physics teachers and number of classes offered there. At
every school there was at least one CPS class and one class
taught using traditional, equation-focused problem solving
methods. At school A there was one teacher who taught the
same course over multiple years, at school B there was a
single teacher teaching two sections of the same class using
different methods, and at school C there were two different
teachers. Because of such vast differences in the design, the
1
Despite a previous verbal agreement, the control teacher at this
school was unwilling to administer the assessments during class
time. This restriction of CPS students to ones willing to come to
an out-of-class testing session means the sample was susceptible
to a potential bias, so we excluded the data.
DOCKTOR et al.
TABLE I. Profile for each high school participating in the study.
School Design Teacher experience
School A One teacher, multiple years In field, Experienced
School B One teacher, multiple sections Out-of-field, Inexperienced
School C Two teachers Both in field, Both experienced
a
Proportion of students at the school classified as low income.
b
Proportion of students who meet or exceed federally mandated standards for adequate yearly progress (AYP) in reading and
mathematics.
student populations at each school, and the teachers
knowledge and experience teaching physics, each school
will be treated separately in the data analysis.
A brief profile for each of the four schools is provided
in the Table I below. Information regarding the proportion
of students who are classified as low income, and the
proportion of students who meet federally mandated
standards for AYP (adequate yearly progress) in reading
and mathematics are taken from public reports by the
Illinois State Board of Education [34].
School A is a small suburban school with an experienced
physics teacher and an affluent, high-achieving student
population. This was the teachers second year participating
in the project, with the first year used as a control class and
taught without CPS. All students were in their senior year.
School B is a rural high school with a student population
that is primarily low income. The teacher taught two
sections of the same course, one using the CPS approach
and one using his usual way of teaching problem solving
that emphasizes equations and mathematical procedures.
The teachers area of expertise is chemistry, and this was
only his second time teaching physics. Students enrolled in
this physics course at school B are in their junior year.
School C is a high school in a small city with a diverse
student population. There were two teachers both teaching
different sections of the same course (one used CPS and
one taught traditionally). Although both teachers have a
strong background in physics the CPS section teacher
typically teaches math courses and the control teacher has
far more experience in terms of years teaching physics.
All participants whose data are reported signed consent
forms allowing their results to be used (students under
18 years of age signed assent forms and their parents signed
consent forms).
C. Procedure
1. Training procedure
Teachers met for a half day with researchers during the
summer prior to using the CPS approach to receive minimal
training and guidelines for implementing Conceptual
Problem Solving. The teachers at each of the three schools
were given guidance on key elements of the CPS approach,
such as the goal being to focus on the role of concepts
in solving problems. They were shown specific examples
020106-6
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
Location Low income1a AYP2b
Suburban 12.0% 68.8%
Rural 53.2% 47.2%
Small city 31.5% 60.0%
of several problems solved using the CPS approach
(see Figs. 1 and 2). Teachers were also told that they
would be given detailed CPS solutions to problems (as in
Figs. 1 and 2) during the semester as they covered new
material to use in their teaching. We pointed out to them
that they had substantial freedom to adapt the CPS
approach to their particular class and student population.
This was intentional since no curricular intervention will be
implemented unless teachers have the flexibility to use it in
ways that fill their instructional needs. Prior to this short
training session, the teachers were not familiar with the
approach, other than previously agreeing to participate in
the study prior to the training. Our observations suggested
that the teachers bought into the CPS approach to the extent
that they were presented with previous research evidence
favoring conceptual approaches to problem solving (as
reviewed in the introduction), and to the extent that they
found the approach reasonable and useful for classroom
implementation.
2. Observations and debriefing meetings
Each CPS teachers classroom was observed at least
once for each major physics topic (four times) during the
implementation. Each teacher met with the researchers for a
debriefing session at the conclusion of the study; these
semistructured interviews were audio recorded, which
provided self-reported descriptions of how they imple-
mented CPS, what they thought about teaching this way,
how frequently they used materials, and teachers percep-
tions of students reactions to the approach.
3. Implementation of the intervention
The conceptual problem solving approach was designed
to be implemented during classroom activities that per-
tained to problem solving instruction. Since the emphasis
placed on problem solving varied for each course and
student population, the time spent on CPS also varied.
Other course activities such as lecturing, demonstrations,
and laboratories remained unchanged.
There was substantial flexibility in how teachers could
implement the approach, and as a result the fidelity of
implementation varied across teachers. There were varia-
tions across teachers in how much time they spent on the
approach during class, how classroom activities were
CONCEPTUAL PROBLEM SOLVING IN HIGH
structured, and how they assessed student understanding of
the approach. Despite these differences, there were some
common elements across all of the schools. Each teacher
devoted at least two class periods to the approach for each
23 week long topic unit studied. They were all accus-
tomed to having students work together in cooperative
groups of between two and four students during class, and
CPS was no exception to this procedure. All of the teachers
chose to grade students on their use of the approach in some
fashion, and typically this occurred on an exam. How
teachers implemented the approach was in fact one of the
factors of interest and so we report below on our obser-
vations regarding implementation as well as teachers
descriptions of their own implementation of the CPS
approach.
Implementation at school A. Teacher A was an experi-
enced physics teacher who preferred to adapt the approach
to his own established set of homework problems. He
reported implementing the conceptual framework primarily
for more advanced problems, which represented between
one-third and one-fourth of students homework (problems
with multiple steps, multiple principles, or more advanced
mathematical procedures). Students were not graded on
homework, however, at least one problem on each exam
required students to use the approach. He was observed to
model the strategy-writing aspects for students when doing
example problems on the board during class (principle-
justification-plan), but only used the two-column solution
in an informal way.
For example, late in the semester teacher A was observed
to use the approach for a complex problem in which a block
slides down a frictionless curved ramp, collides with a
second block, and the blocks fall off of a table onto the floor
below. He prompted students to decompose the problem
into three segments and identify a relevant principle for
each part (conservation of energy, conservation of momen-
tum, and motion with constant acceleration), which he
wrote on the board. The Justifications for these principles
were brief statements: frictionless ramp, elastic colli-
sion, and constant acceleration. He then guided students
to construct a plan of 3 steps and wrote them out in words
on the board. Then he took each step individually and (with
verbal prompts from the students) completed the math-
ematical solution for each step sequentially. Although the
solution was not presented side-by-side in the two-column
format, all of the components were present.
Teacher A also devoted some class time to synthesize
the big ideas that had been covered during the fall semester.
He did this by showing a diagram that listed the major
theorems and conservation laws in boxes, and arrows that
connected them to corresponding equations. He empha-
sized to students that if they are uncertain how to begin
solving a problem, they should refer back to the big ideas
from the class and select one of these concept(s) or
principle(s).
020106-7
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
Implementation at school B. In contrast, teacher B
primarily teaches chemistry and physical science courses,
and was teaching this physics course out of field for only
the second time. As a result, he described his instructional
plans as somewhat fluid and subject to change on a weekly
basis. Teacher B taught one section of the course using the
conceptual problem solving approach, and taught a second
(control) section using a more equation-centered approach
(a framework called GUESSwrite what you are Given,
the Unknown quantity, an Equation, Substitute values, and
Solve.) He almost exclusively used sample problems
provided by the researchers for both of his classes, but
formatted them in a different way for the control class.
When introducing the conceptual approach for a new topic,
he was observed to write the strategy on the board word
for word from the researchers examples and refer to the
example on paper as he did so. Students in the CPS section
were given blank worksheets that prompted them for each
step in the strategy and to format their solutions as two
columns, and they often worked in cooperative groups.
Students homework was graded and at least one problem
on each exam (for the CPS class) required them to complete
a strategy and two-column solution. Teacher B described
his classroom implementation of CPS in the following way:
Initially it was more my guidance, and then while they
were still learning it, then they were in pairs or maybe
even three to help each other out getting through it while
Id be going around. And then toward the end it was
more individual. Itd be, Ok, start on it individually,
and then now lets compare and see where we are. And
then go big group and Id have somebody talk about
what they thought (teacher B).
In addition, teacher B later adjusted the way worksheet
templates were used in response to student feedback. When
some students complained about writing their plan steps
and then recopying them into the left column of the solution
template, he modified the approach to have students write
their plan steps directly into the left column of the two-
column solution template rather than writing it as part of the
strategy.
Implementation at school C. Teacher C has a strong
background in physics, but was hired primarily to teach
math classes at his school and only taught physics classes
on an as needed basis. He used many of the problems
provided by researchers but also incorporated his own set of
problems. He admitted that he did not use the approach
very much for beginning topics like one-dimensional
motion, but increased his use of strategy writing substan-
tially for later topics like energy and momentum (approx-
imately six times in a 34 week period). Teacher C felt that
the approach was more relevant later in the semester:
I didnt focus a lot on it during the first half of
the semester because the entire first half is constant
DOCKTOR et al.
accelerationI was kind of holding back on it until we
had a few more things in our toolbox before we could
start to use it to differentiate between the different
concepts. So as we started to get into momentum and
energy and torque and things like that I started to use it
more frequently (teacher C).
At the introduction of a new topic, he would write a
sample strategy and the initial plan steps on the board with
verbal guidance from students. The plan step usually
involved a general framework of drawing a picture, listing
what you know, writing equations (separating the x and y
directions when appropriate), and solving for the unknown
quantity. The solve step was not very specific; he told
students that as they got started the details would become
more obvious. Students followed the two-column solution
format on homework, but were not required to do so
on exams.
Teacher C made extensive use of cooperative group
problem solving and categorization with the conceptual
approach. During a group problem solving session, he
handed out worksheet templates for four to six problems
that prompted students to complete only the first steps of
principle and justification. Then, each student group was
required to write out a plan and solution to just one or two
of those problems, which they presented to the class.
4. Assessment procedures
As described previously, there were five different written
assessments (see sample questions provided in the
Appendix). However, due to time constraints not every
school gave every test. The researchers allocated time per
test and all the tests were gauged to take 85 min (25 min for
the categorization test and 15 min for each of the four
remaining tests). However, teachers had different limita-
tions on how much class time they could give. The total
time allotted for the assessments ranged from 45 to 85 min
(administered in either one session or split over two class
periods). Teachers did not have access to the assessments
ahead of time. The assessments completed by participating
students were scored by researchers and did not contribute
to students grades.
TABLE II. Performance on each assessment by condition and school. The Conceptual Problem Solving treatment group typically
scored higher than the Equation Problem Solving control group. Values represent a percent score  the standard error of the mean and
stars indicate significant differences p < 0.05.
School A
Assessment
CPS (N 14) EPS (N 23)
Problem solving 0.48  0.10 0.38  0.05
Conceptual questionsa 0.32  0.04 0.22  0.02
Categorization 0.47  0.03 0.45  0.03
Finding errors 0.06  0.03 0.14  0.03
Equation instantiation 0.54  0.07 0.42  0.05
a
At school A the conceptual questions were free response explanations and at schools B and C they were multiple choice questions.
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
Questions that required written explanations or problem
solutions from students were scored independently by two
researchers using agreed-upon rubrics (this pertained to the
free-response conceptual questions used as school A, and
problem solving questions used at all schools). These
scores were further discussed by the researchers to reach
a single consensus score. The problem solving questions
were scored according to a rubric modeled after Ref. [35],
which scores solutions on the categories of useful descrip-
tion, selecting a relevant physics concept or principle,
applying the concepts to the specific conditions in the
problem, executing mathematical procedures, and the over-
all communication of a logical reasoning pattern. The
finding errors test was scored for both identifying a mistake
in the sample solution and giving an appropriate explan-
ation for the mistake.
III. RESULTS AND DISCUSSION
A. Assessment scores
1. Description of assessment scores
The average scores on each assessment are provided in
Table II. Schools that did not administer a particular test
are designated with not applicable (NA). In general, the
problem solving and conceptual measures administered to
students showed a consistent advantage to the Conceptual
Problem Solving sections over the classes taught more
traditionally.
The problem solving test scores were significantly
different at school C, differing by 16% [t28 3.17,
p < 0.005], with the CPS class having stronger skills at
both selecting and applying an appropriate principle. The
differences in problem solving scores were large but did not
reach significance at the other two schools; school A (10%)
and school B (15%). Each problem on the test was scored
according to 67 subparts as described in the assessment
procedures section (useful description or diagram, physics
principle, specific application of physics, mathematical
procedures, final units, and communicating reasoning.)
Each part received a score of 0, 0.25, 0.5, 0.75, or 1.
The interrater reliability for the two scorers was a Cohens
School B School C
CPS (N 8) EPS (N 9) CPS (N 12) EPS (N 18)
0.41  0.08 0.26  0.06 0.55  0.04 0.39  0.03
0.35  0.06 0.15  0.02 0.46  0.09 0.35  0.05
0.46  0.08 0.35  0.04 NA NA
0.06  0.03 0.05  0.03 NA NA
0.52  0.05 0.51  0.07 0.57  0.08 0.52  0.06
020106-8
CONCEPTUAL PROBLEM SOLVING IN HIGH
weighted kappa (quadratic squares) value of 0.72  0.02
indicating substantial agreement [36]. Each subpart was
weighted to obtain a total score out of five points (with
more weight placed on the physics principle and applica-
tion). Scores reported in Table II represent consensus scores
reached by the two scorers.
The conceptual questions test showed a significant
difference at two of the schools, a 10% difference at school
A [t35 2.54, p < 0.05] and a substantial 20% differ-
ence at school B [t15 3.53, p < 0.005]. Although not
significant, school C also showed a difference of 11%
favoring the CPS class. For the free-response version of the
conceptual questions test administered at school A, each of
the six questions was scored according to 2 or 3 subparts as
0, 0.5, or 1 point. The interrater reliability for two scorers
was a Cohens weighted kappa (quadratic squares) value of
0.48  0.06 indicating moderate agreement.
The categorization test, equation instantiation test, and
finding errors tests did not show any significant differences
between groups. However, there was an 11% difference in
categorization at one of the two schools that administered
the test (2% at the other school), and a 12% difference in
equation instantiation at one school (with 1% and 5% at the
other schools), all differences favoring CPS. The finding
errors test was extremely difficult for all sections at both
schools that administered it, with almost all students
receiving no credit for their answers.
2. Interpretation of assessment scores
Overall, the CPS classes performed better, but the
differences for tests and schools varied. The assessment
scores require a bit of interpretation, but we stress two
points in evaluating the results. First, there were not large
numbers of students in these classes, so the standard errors
are sometimes large. Given the large differences in schools
and implementation, we decided that the individual school
data are more informative than some way of combining it
across the schools. Second, there was great variation in the
fidelity of the implementation. Finally, the CPS part of the
course was quite short (less than 7 h during a semester). We
begin by addressing each test.
The traditional measure in physics classes, problem
solving, showed a clear benefit for CPS across the schools
but with much variation between students as well. The
mean differences were 10%, 15%, and 16%, though only
the last reach conventional levels of significance. Given the
short amount of time used for CPS, these are consistent and
important increases.
The conceptual questions also showed consistent and
important increases, with CPS advantages of 10%, 20%,
and 11%, with the first two having a statistically significant
difference at the 0.05 level. Note that between these two
tests, all schools showed one statistically significant differ-
ence and one sizable, if not quite significant, difference.
The results of the problem solving and conceptual
020106-9
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
questions provide strong evidence for the effectiveness
of the CPS intervention.
The categorization test was administered at two schools
and did not show consistent or significant differences (2%
and 11%). We expected that this test would be sensitive to
the CPS intervention and were surprised the overall effects
were not larger. We mention two issues that may have
contributed to smaller than expected effects. First, these are
difficult tests. Remember, the student was given a choice of
two problems to match to the model, one of which was very
similar superficially but was solved by a different principle
and one of which was very dissimilar superficially but was
solved by the same principle (see Appendix). As Table II
shows, all groups chose the superficially similar one
slightly more often than the similar principle one (all
average scores are below 50%). Second, after we conducted
this study, we have done additional work on problem
categorization [37]. That study showed that even once
people appreciated that principles should be used to
categorize, they found it very difficult to determine the
underlying principles. This test required students to deter-
mine at least two underlying principles (the model and one
other) and to overcome a strong tendency for novices to
choose superficially similar problems.
The finding errors test was simply too difficult to find
differencesalmost all students received no credit for their
answers. We knew that such a test requires rather different
skills than the CPS focus, but thought it would be
interesting to examine if there was transfer. However,
given the low performance, it is difficult to know whether
the problem is the low transfer or difficulty of the particular
problems.
The equation instantiation test showed small nonsignifi-
cant advantages for the CPS intervention (12%, 1%, and
5%). We expected any effect of CPS here would be small,
since it tests a more specific understanding (objects
matched to variables) than the CPS focus, although the
trend favors the CPS classes. It is possible that a better
understanding of why a concept is applicable could also
help to understand how to instantiate the particular vari-
ables in the equation of that principle.
Overall, there was a clear advantage for CPS, particu-
larly, for the problem solving and conceptual questions,
despite the little time used for the intervention and the small
number of students in each comparison. We do recognize
that there were unexpected difficulties with the categori-
zation and finding error tests, but we believe there is a clear
pattern of performance that favors CPS classes over tradi-
tional classes.
3. Additional assessment issues
We also wish to address two issues brought up in
discussions and reviews. First, although this covered a full
semester of a course, it would be helpful to know if CPS
understanding is not just specific to the assessments used in
DOCKTOR et al.
the study. One school (school C) gives all students district-
wide tests in various subjects and the teacher was able to
give us the summaries from this CPS year and the previous
year for his class for each problem. For the 12 conceptual
questions asked in the test, his class for the year using the
CPS performed 9% better across the problems [marginally
significant, t11 1.84, p < 0.10; the CPS class also did
better on 9 of the 12 problems, also p < 0.10 by a sign
test]. Thus, even after the course was over, the CPS
intervention led to some gain in district-wide tests devel-
oped independently of this study.
Second, is there any evidence that the CPS and tradi-
tional classes were of equal ability at the beginning of the
study? This was the students first physics course so we
have no way to examine prior physics ability. We chose not
to include access to various test scores as part of the assent
or consent forms to insure greater participation, so we
cannot look at those (and are not sure of the correlation with
physics performance). We think ability differences are very
unlikely to explain the CPS advantage. Recall that each
school had different arrangements for choosing the class
that was CPS and the one that was traditional. In school A,
with only one physics class per year, one year was tradi-
tional and the following year was CPS (this was with a very
experienced teacher so the extra year was not likely to
improve teaching performance). In school B, the teacher
had two classes and randomly chose which would be CPS
and which traditional. In school C, the teacher had one class
and used the class of another teacher (with far more
experience) as the traditional class. Thus, it seems very
unlikely that the CPS advantages were due to differences in
prior ability.
B. Debriefing meetings
Each teacher met with researchers at the conclusion of
the study for a semistructured interview referred to as a
debriefing meeting. They were asked to describe how they
implemented conceptual problem solving in their class and
reflect on what they thought about teaching in this way.
1. Comments from teacher A
Teacher A said the approach is good in that it forces
students to think about what they are doing and why, but
said that students at this level struggle with the math and it
is difficult to introduce high-level concepts right from the
start. He also felt that students had trouble developing a
plan before actually trying to solve the problem because
they might need to do a little trial and error before knowing
what principle to use. His students expressed some resis-
tance to the additional writing required for a strategy, but
appreciated that it helped them make sense of the equations.
2. Comments from teacher B
Teacher B observed that students had the most difficulty
with the justification component of strategy writing.
020106-10
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
Identifying a relevant principle was not as difficult for
these students because they recognized what chapter or unit
was currently being covered, and they were never tested on
multiple concepts at the same time. For more complicated
problems that included multiple steps, some students also
struggled with planning the solution.
Teacher B stated that early in the semester he found that it
took longer to go through problems with strategy writing and
he had some concerns that this might hinder students in the
treatment section if they didnt see as many problems during
class. Despite his initial difficulties to manage classroom
time, he later came to the decision that the additional depth
of coverage for the CPS section was merited:
I think a lot of the style I use is more equation based. So
I think to me it probably is a pretty good thing to slow
down a little bit and concentrate more on the concepts.
Because it seems like it merits that, that they would get
more out of that and maybe even doing less of the
problems and focus a little bit more on the conceptual.
I think also getting them to produce more of the
conceptual too rather than me talking to them about
it, that theyre going through and producing it on their
own (teacher B).
3. Comments from teacher C
Teacher C indicated that prior to using this conceptual
approach some students would just solve problems by
writing down some random stuff, because they werent
sure how to start, and with this framework he did not see
this happening. He elaborated on this when describing how
students were assessed on their use of the approach for
several problems on a unit exam:
They had to write out the principle and justify their
answer as part A, and then part B was solve it. So it was
sort of adapting the method. Because we had done
enough practice on it they almost all got that. And once
they knew the principle, solving it was easy. And that was
one of the themes that, at least I took away from it, was
that the actual algebra was really easy, its just kind of
knowing, looking at this problem, how do we even start it.
Having them focus on the principle first gave them the
confidence to know how to solve it (teacher C).
He observed that students were producing a higher quality
of work and engaging in richer discussions than in the past,
and they performed better on a district-wide physics assess-
ment than in past years. Teacher C felt encouraged by
improved student performance in the course, especially on a
unit test near the end of the semester:
So I thought, well, best way to try this out is to give
them momentum, energy, and torque all in one test and
I was really really happy. I think it was one of the best
CONCEPTUAL PROBLEM SOLVING IN HIGH
results Ive ever seen with those concepts all mixed
together, all jumbled upThats something that Ive
observed in virtually every math class, is that the
students can always do any individual skill, but when
they see all the skills all at once on like the final, or on
an AP test, they just freeze.
Teacher C stated that he agrees with the philosophy of
the approach, because concepts and principles are essential
for learning math and science. He was observed to make
this goal explicit to his class through statements such as,
I want to change your thinking from Which equation
should I use? to What is the concept? In the near future included both conceptual and problem solving measures
he hopes to apply the conceptual approach to classes he
teaches in mathematics.
C. Summary of debriefings
Each teacher had a slightly different interpretation of the
best use for CPS in their classes. Teacher A found it helpful
for students to use this framework to organize their thinking
on challenging, multistep problems after they had some
practice with simpler (traditional) problems. Despite this
partial implementation his students saw modest gains on
the assessments, particularly, the free-response conceptual
questions. Teacher B used a full-fledged implementation of
CPS during all problem solving activities in his class. He
exclusively utilized the sample materials designed by the
researchers, including the worksheet templates for students.
This resulted in widespread improvement over traditional
methods; his students showed differences of 10% or higher
on three of the five assessments: categorization, conceptual
questions, and problem solving. Last, teacher C viewed
CPS as a tool to help students distinguish among multiple
principles that could be applied to solve problems (e.g.,
energy and momentum), and as a starting point for
problem solving. Once students could identify the princi-
ples needed to solve problems teacher C felt that problem
solving became more straightforward for students. His
students made significant gains in problem solving, and
performed higher on a district-wide assessment.
All teachers acknowledged that it takes more time to go
through a conceptual problem in depth, and, consequently,
implemented CPS in ways to suit their own instructional
needs as well as perceived pressures (e.g., to cover more
content). Teacher A only used the full approach for one
fourth to one third of all problems, teacher B ended up
assigning more problems as homework for the CPS class or
going through fewer problems, and teacher C had students
just practice the first two steps (principle and justification)
for several problems and only solve one or two of them
completely. The teachers also indicated student resistance
to having to write more, which prompted them to grade
their use of the approach on homework or exams. Teacher
B also modified the two-column solutions to reduce the
amount of writing.
020106-11
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
IV. GENERAL DISCUSSION
The motivation behind the Conceptual Problem Solving
approach is to shift students away from their focus on
equations and mathematical procedures and toward prob-
lem solving approaches based on physics concepts and
principles. We found positive effects of CPS even after
relatively short exposures to the approach (on average, less
than 7 h of CPS classroom instruction over a 4-month
period). Except for one of our measures (finding errors)
where students from all schools performed near floor, gains
favoring the CPS-taught classes in a battery of tests that
consistently emerged. Gains varied across schools and
measures, likely due to differing student populations and
different implementation of the approach by the three
teachers.
CPS is an approach that is flexible, because it does not
require major restructuring of the way instruction is carried
out in high schools and because it allows teachers to
implement it in different ways. This was confirmed by
observing the ways in which each teacher adapted CPS to
align with their personal teaching style and the perceived
needs of their students. In general, the teachers who were
more experienced (teachers A and C) used the approach
for a subset of problems and placed greater emphasis on
choosing among multiple principles or combining princi-
ples for problems, whereas inexperienced teacher B relied
upon researchers materials for all problem solving activ-
ities and as a result implemented the approach with a high
level of fidelity. More rigorous training beforehand and/or
during the semester may have led to a closer alignment with
researchers intentions for the materials.
The realities and constraints of working in real class-
rooms with real teachers created implementation chal-
lenges. For example, the participating teachers physics
expertise varied, from two teachers who had physics
undergraduate degrees to one teacher trained in chemistry
who was teaching physics out of field for only his second
time. Pressure to cover content and time constraints also
limited the amount of time spent on CPS and on assessing
its impact. Finally, students tended to resist the approach
since it required them to do significantly more prose writing
about physics and to carry out more conceptual discus-
sions, which were both difficult for students to do and time
consuming.
In light of these difficulties, our experiences suggest
possible improvements to the approach. Implementing
CPS using technology (e.g., via a web-based program that
presents problems and students perform analyses by
answering questions posed by the program and receiving
conceptual feedback) would create several advantages. For
example, students could use CPS both for in-class problem
solving and for homework, thereby spending more time
thinking about the application of concepts in problem
solving. Designing the technology-based program in ways
DOCKTOR et al.
where students are queried about what concepts they could
apply and allowing them to make selections from choices
provided would streamline the analysis and reduce student
frustration during times where they were stuck with the
paper-and-pencil version. This would also add flexibility
to teachers implementation of the approach. Additional
improvements to the approach include greater emphasis on
categorization practice, as was done at school C (identify-
ing appropriate concepts and principles for problems
without solving them) and prompting students to synthesize
across all topics, as was done at school A.
Looking toward the future, there is a need for curricular
materials that help students achieve better conceptual
understanding of physics in high school, and indications
are that CPS is one possible way of achieving this goal.
[1] E. Bagno and B.-S. Eylon, From problem solving to a
knowledge structure: An example from the domain of
electromagnetism, Am. J. Phys. 65, 726 (1997).
[2] J. H. Larkin, Processing information for effective problem
solving, Eng. Educ. 70, 285 (1979).
[3] J. H. Larkin, Enriching formal knowledge: A model for
learning to solve textbook physics problems, in Cognitive
Skills and Their Acquisition, edited by J. R. Anderson
(Lawrence Erlbaum, Hillsdale, NJ, 1981), p. 311.
[4] J. H. Larkin, The role of problem representation in physics,
in Mental Models, edited by D. Gentner and A. L. Stevens
(Lawrence Erlbaum, Hillsdale, NJ, 1983), p. 75.
[5] J. Larkin and F. Reif, Understanding and teaching problem
solving in physics, Int. J. Sci. Educ. 1, 191 (1979).
[6] F. Reif, J. H. Larkin, and G. C. Brackett, Teaching general
learning and problem-solving skills, Am. J. Phys. 44, 212
(1976).
[7] J. Tuminaro and E. F. Redish, Elements of a cognitive
model of physics problem solving: Epistemic games, Phys.
Rev. ST Phys. Educ. Res. 3, 020101 (2007).
[8] L. N. Walsh, R. G. Howard, and B. Bowe, Phenomeno-
graphic study of students problem solving approaches in
physics, Phys. Rev. ST Phys. Educ. Res. 3, 020108 (2007).
[9] J. Bowden, G. DallAlba, E. Martin, D. Laurillard, F.
Marton, P. Ramsden, A. Stephanou, and E. Walsh, Dis-
placement, velocity, and frames of reference: Phenomeno-
graphic studies of students understanding and some
implications for teaching and assessment, Am. J. Phys.
60, 262 (1992).
[10] D. Hestenes, M. Wells, and G. Swackhamer, Force
Concept Inventory, Phys. Teach. 30, 141 (1992).
[11] E. Kim and S.-J. Pak, Students do not overcome conceptual
difficulties after solving 1000 traditional problems, Am. J.
Phys. 70, 759 (2002).
[12] D. Maloney, T. OKuma, C. J. Hieggelke, and A. Van
Heuvelen, Surveying students conceptual knowledge of
electricity and magnetism, Am. J. Phys. 69, S12 (2001).
020106-12
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
Further study of Conceptual Problem Solving is warranted,
in order to determine which specific elements of a teachers
implementation are linked to student performance. The
findings from this study support the goal of emphasizing
concepts used to solve problems, so that students do not
treat physics problem solving as a hunt for equations that
are forgotten soon after a course is over.
ACKNOWLEDGMENTS
The authors wish to thank the high school physics
teachers who helped to facilitate this research. Work in
part supported by the Institute of Education Sciences of
the U.S. Department of Education under Grant No. DE
R305B070085.
[13] E. Mazur, Peer Instruction: A Users Manual (Prentice
Hall, Upper Saddle River, NJ, 1997).
[14] L. C. McDermott, Guest comment: How we teach and how
students learnA mismatch?, Am. J. Phys. 61, 295
(1993).
[15] J. H. Larkin, Cognition of learning physics, Am. J. Phys.
49, 534 (1981).
[16] A. Newell and H. A. Simon, Human Problem Solving
(Prentice-Hall, Inc., Englewood Cliffs, NJ, 1972).
[17] F. Reif and J. I. Heller, Knowledge structure and problem
solving in physics, Educ. Psychol. 17, 102 (1982).
[18] M. T. H. Chi, P. J. Feltovich, and R. Glaser, Categorization
and representation of physics problems by experts and
novices, Cogn. Sci. 5, 121 (1981).
[19] R. Glaser, Education and thinking: The role of knowledge,
Am. Psychol. 39, 93 (1984).
[20] B.-S. Eylon and F. Reif, Effects of knowledge organization
on task performance, Cognit. Instr. 1, 5 (1984).
[21] A. Van Heuvelen, Learning to think like a physicist: A
review of research-based instructional strategies, Am. J.
Phys. 59, 891 (1991).
[22] R. Gautreau and L. Novemsky, Concepts firstA small
group approach to physics learning, Am. J. Phys. 65, 418
(1997).
[23] R. J. Dufresne, W. J. Gerace, P. T. Hardiman, and J. P.
Mestre, Constraining novices to perform expertlike prob-
lem analyses: Effects on schema acquisition, J. Learn. Sci.
2, 307 (1992).
[24] J. P. Mestre, R. J. Dufresne, W. J. Gerace, P. T. Hardiman,
and J. Touger, Promoting skilled problem-solving behavior
among beginning physics students, J. Res. Sci. Teach. 30,
303 (1993).
[25] P. T. Hardiman, R. Dufresne, and J. P. Mestre, The relation
between problem categorization and problem solving
among experts and novices, Mem. Cogn. 17, 627 (1989).
[26] W. J. Leonard, R. J. Dufresne, and J. P. Mestre, Using
qualitative problem-solving strategies to highlight the role
CONCEPTUAL PROBLEM SOLVING IN HIGH
of conceptual knowledge in solving problems, Am. J. Phys.
64, 1495 (1996).
[27] P. G. Hewitt, Conceptual Physics, 9th ed. (Addison Wesley,
San Francisco, CA, 2002).
[28] C. W. Camp and J. J. Clement, Preconceptions in Mechan-
ics: Lessons Dealing with Students Conceptual Difficul-
ties (Kendall Hunt Publishing Co., Dubuque, IA, 1994).
[29] D. Huffman, Effect of explicit problem solving instruction
on high school students problem-solving performance and
conceptual understanding of physics, J. Res. Sci. Teach.
34, 551 (1997).
[30] K. Heller and P. Heller, The Competent Problem Solver for
Introductory Physics (McGraw-Hill, Boston, 2000).
[31] P. Heller, R. Keith, and S. Anderson, Teaching problem
solving through cooperative grouping. Part 1: Group versus
individual problem solving, Am. J. Phys. 60, 627 (1992).
020106-13
PHYS. REV. ST PHYS. EDUC. RES 11, 020106 (2015)
[32] A. D. Smith, J. P. Mestre, and B. H. Ross, Eye-gaze
patterns as students study worked-out examples in me-
chanics, Phys. Rev. ST Phys. Educ. Res. 6, 020118 (2010).
[33] See Supplemental Material at http://link.aps.org/
supplemental/10.1103/PhysRevSTPER.11.020106 for
separate auxiliary material for sample assessment items.
[34] URL: http://illinoisreportcard.com/.
[35] J. L. Docktor, Development and validation of a
physics problem solving assessment rubric, Ph.D. thesis,
University of Minnesota, 2009.
[36] J. Cohen, Weighted kappa: Nominal scale agreement
provision for scaled disagreement or partial credit, Psychol.
Bull. 70, 213 (1968).
[37] J. L. Docktor, J. P. Mestre, and B. H. Ross, Impact of a
short intervention on novices categorization criteria,
Phys. Rev. ST Phys. Educ. Res. 8, 020102 (2012).