DAVID K.

PUGALEE

A COMPARISON OF VERBAL AND WRITTEN DESCRIPTIONS OF

STUDENTS’ PROBLEM SOLVING PROCESSES

ABSTRACT. The purpose of this exploratory study was to investigate the impact of writ-
ing during mathematical problem solving. The study involved an analysis of ninth grade
algebra students’ written and verbal descriptions of their mathematical problem solving
processes. Through this comparison, a better understanding of the connection between
problem solving and writing is realized. The written and verbal data show a relationship
between the number of problem solving strategies tried by students and their success. The
majority of problem solving behaviors involve execution actions such as carrying out goals
and performing calculations. Students who construct global plans are more successful
problem solvers. Students engage in verification behaviors at various stages of problem
solving though the majority of students do not verify their final answers. While both oral
and written descriptions serve as a tool for understanding students’ thinking processes, a
comparison of the two modes of reporting, using a metacognitive framework as the lens
of analysis, reveals some important variations. Students who wrote descriptions of their
thinking were significantly more successful in the problem solving tasks (p < 0.05) than
students who verbalized their thinking. Differences in metacognitive behaviors also support
the premise that writing can be an effective tool in supporting metacognitive behaviors.

KEY WORDS: communication, discourse, metacognition, problem solving, writing

1. I NTRODUCTION

Recommendations emphasizing communication as an essential element of

mathematics teaching and learning have been central tenets of standards
for more than a decade (NCTM, 1989; NCTM, 2000). Students who have
opportunities to engage in mathematical communication including speak-
ing, reading, writing, and listening receive a dual benefit of communic-
ating to learn mathematics and learning to communicate mathematically
(NCTM, 2000).
Recent interest in writing as a communicative tool posits that this form
of communication has the potential to promote mathematical understand-
ing (Pugalee, 2000; Sierpinska, 1998; Morgan, 1998; Brown, 1997). Des-
pite this interest, the relationship between writing and mathematics has
been largely neglected in educational research with most literature describ-
ing writing in mathematics in general terms with little analysis of the text
themselves (Morgan, 1998).

Educational Studies in Mathematics 55: 27–47, 2004.

© 2004 Kluwer Academic Publishers. Printed in the Netherlands.
28 DAVID K. PUGALEE

In this study, research involving written texts will be extended through a

comparison of written and verbal (talk-aloud) protocols of students’ math-
ematical problem solving. More specifically, this comparison will (a)
provide a context for understanding basic differences in the oral and writ-
ten communication of students working individually to solve mathematical
problems and (b) provide a characterization of these processes.

2. BACKGROUND

Writing in mathematics has been identified as integral in sustaining the de-

velopment of reasoning, communication, and connections (Connolly and
Vilardi, 1989; Countryman, 1992; Sierpinska, 1998). The ability to artic-
ulate one’s ideas is seen as a benchmark of deep understanding, requiring
reflection to identify and describe critical elements and concepts (Car-
penter and Lehrer, 1999). This level of cognitive engagement substantiates
writing as a generative act in the process of constructing meaning (Apple-
bee, 1981; Emig, 1977; Flower, 1989; Miller and England, 1989; Morgan,
1998; Brown, 1997). Though the literature asserts the power of writing as
a learning tool advancing the writing across the disciplines movement, it
has not gained wide acceptance in mathematics classrooms.
Vygotsky (1987) viewed writing as involving deliberate analytical ac-
tion on the part of the writer requiring the writer to maximally compact
inner speech so that it is fully understandable; thus, written words require
a deliberate structuring of a web of meaning. He also viewed writing as im-
portant in forming associations between current and new knowledge, help-
ing the writer tie down ideas in order to make connections between prior
and new concepts. Meier and Rishel (1998) hold that writing, speaking and
thinking are entwined such that narrative writing and speaking assists stu-
dents in better understanding mathematical ideas. Written words provide
a vehicle for students to communicate with self and others. These writ-
ten forms support deeper mathematical understanding developed through
students’ critical thinking (Masingila and Prus-Wisniowska, 1996).
Research also shows that writing supports metacognition (Pugalee, 2001;
Powell, 1997; Artzt and Armour-Thomas, 1992; Carr and Biddlecomb,
1998). In these studies, writing is posited as providing a level of reflec-
tion that promotes students’ attending to their thinking about mathematical
processes. This awareness and self-regulation appears to play an import-
ant role in students’ selection of appropriate information and strategies
while solving a wide range of mathematical problems. Metacognition in-
cludes such behaviors as predicting, planning, revising, selecting, classi-
fying and checking (Allen, 1991) allowing individuals to be successful in
 COMPARISON OF PROBLEM SOLVING PROCESSES 29

problem solving situations by contributing to their ability to identify and

work strategically (Davidson and Sternberg, 1998). These behaviors also
allow them to make connections between their problem solving work and
their knowledge of mathematical content and procedures (Schurter, 2002).
Garofalo and Lester (1985) identified a metacognitive framework con-
sisting of four categories of activities or behaviors involved in perform-
ing a mathematical task: orientation, organization, execution, and veri-
fication. Each phase includes several metacognitive behaviors related to
problem solving. Orientation includes comprehension strategies, analysis
of information and conditions, assessment of familiarity with a task, ini-
tial and subsequent representation, and assessment of problem difficulty
and chance of success. Organization includes identification of goals and
subgoals, global planning, and the local planning necessary to carry out
global plans. Execution includes performance of local actions, monitoring
progress of local and global plans, and trade-off decisions. Verification in-
cludes evaluation of decisions and results of executed plans. The research-
ers determined that these four categories of behaviors impacted perform-
ance on a wide variety of mathematical tasks. This framework provides a
useful context for analysis of students’ written and oral protocols generated
while engaged in mathematical problem solving.
Think-aloud protocols are generally accepted as a means of assessing
the mental processes of an individual. These protocols require the subject
to ‘think aloud’ concerning his/her thoughts as s/he works. Such informa-
tion is valued as a tool in process tracing research. Smagorinsky (1989)
argues that internal thought processes of subjects are not altered when
engaging in verbalizations that do not require the reporting of inform-
ation not normally attended to while performing the task. He points to
studies that show no differences in the specific steps taken by groups who
either solved a problem silently or while talking aloud. Such verbalizations
should not be confused with various genres of classroom discourse that
involve a complex amalgamation of mathematical, social, and experiential
factors providing cognitive advantages when students work together or in-
teract with teachers (see O’Connor, 1998; Gravemeijer, Cobb, Bowers, and
Whitenack, 2000). Writing has been viewed as “thinking aloud on paper”
(Rose, 1989). These process-tracing methods (Flower and Hayes, 1983)
provide more information than input-output methods, they yield rich data,
and provide a means of observing important processes difficult to identify
using other methods. Verbal and written protocols are used in this study as
a tool in comparing students’ mathematical problem solving processes.
30 DAVID K. PUGALEE

3. DATA AND ANALYSIS

3.1. Sample
Twenty subjects enrolled in an introductory high school algebra class par-
ticipated in this study. These ninth grade subjects included eleven females
and nine males. Seven were African American and thirteen were Caucasian.
The subjects were assigned to one of two groups by alternating from a
ranked list based on a composite score from language arts and mathematics
results obtained from the previous year’s state mandated tests in these two
subjects. This process was utilized so that both groups would be com-
parable in mathematical and verbal abilities. Since the groups would be
alternating in providing written and verbal descriptions of their problem
solving processes, this procedure controlled for potential differences in the
language and mathematics abilities of the two groups.

3.2. Problem selection

The researcher and a university mathematics educator identified twenty
problems from curricular materials, including print resources and text-
books, as appropriate for the age and academic level of the students. Aca-
demic level was considered so that the problems would likely provide
students with an entry point for obtaining a solution. Additionally, these
open-ended problems were deemed likely to produce descriptive problem
solving processes. A panel of ten experts in mathematics education, the
majority of them classroom teachers with an average 12.5 years of ex-
perience, then rated the problems for the level of difficulty. The ratings
reflected the experts’ judgments about the relative success of students in
introductory algebra to arrive at a correct solution: low level problems
likely to be answered correctly by the majority of students, medium level
problems likely to be answered correctly by less that three-fourths of stu-
dents, and high level problems likely to be answered correctly by less than
one-third of students. The panel also indicated the strategy which they
felt was most likely to be used by the students. Six problems were then
chosen so that two problems from each of three levels of difficulty (low,
moderate, high) were included. This selection process also considered the
strategies the panel thought would most likely be used by the students.
This range of difficulty level and predicted strategy use was intended to
provide opportunities to observe students’ responses in diverse problem
situations. A task analysis of the six problems provides information about
the problem structure giving additional indicators of problem complexity
and variation (see Kulm, 1979; Barnett, 1979; Webb, 1979).
 COMPARISON OF PROBLEM SOLVING PROCESSES 31

The task analysis of these six selected problems includes descriptions of

syntax, content, context, and structure variables. Syntax, content, and con-
text variables are discussed in the following paragraphs. Structure variables
will be limited to a discussion of the problem-solving strategies intrinsic
to the problem. This discussion occurs as part of the analysis of error and
strategy types later in this paper.
Syntax variables describe arrangement of and relationship among words
and symbols in a problem including problem length, grammatical com-
plexity, and data sequence. Syntactic variables can be used as indicators
of the time and difficulty involved in language processing. Problem length
may be measured by considering the total number of words per problem
and the average number of words per sentence. For the six problems, the
number of words ranged from 14 for problem 3 to 50 for problem 2.
Average sentence length is considered as an indicator of grammatical com-
plexity. Two problems had average sentence lengths of less than ten words
per sentence (problem 2 with 8.3; problem 3 with 7.0). The other four
problems had average sentence lengths of more than ten words (problem
1 with 11.3; problem 4 with 13.3; problem 5 with 14.0, and problem 6
with 18.3). Location of the question stem in the problem is another feature
in considering syntax. All six problems contain the question stem at the
end of the problem. Data sequence, or the arrangement of information in
the problem used in the solution, is also a surface factor of a problem.
For all the problems, with exception of number 2, the data in the problem
was presented in the order needed to solve the task. Question 2 requires
subtraction from the total distance which is given at the end of the problem.
Content and context variables refer to the meanings conveyed by a prob-
lem. Content variables extend beyond mathematical topic to include lin-
guistic content variables such as the use of mathematical words or phrases.
The six problems contain some mathematical terms that might impact the
meaning. Problem 1 requires distinction between digit and numeral. Prob-
lem 2 and 6 use distance units such as miles for problem 2 and meters (and
meters per second) for problem 6. Problem 3 uses the term triangular num-
ber. Problem 4 requires understanding of average and relation terms such
as lowest and least. Problem 5 uses ‘adding’ as a mathematical operation.
Context involves nonmathematical meanings conveyed in the problem.
Problems 2 and 5 involve the use of diagrams that are essential in solving
the problem. This context provides an additional component in interpreting
the problem situation. None of the problems contain extraneous informa-
tion or numbers, and all numbers given in the problems are necessary to
obtain correct solutions. Whether the problem is a real world application
32 DAVID K. PUGALEE

is also a context variable. All problems, except 2 and 5, present real-world

contexts.
An analysis of problem characteristics such as those described through
consideration of content, context, and structure variables allows for the
formulation of inferences about the complexity of the tasks and sources of
possible variation in mathematical problem solving. This focus on those
variables that are intrinsic to the problem itself offers support for the selec-
tion of the problems based on varying levels of linguistic complexity and
anticipated strategy employment. The rationale for this process is based on
the premise that these elements interact to determine the complexity of the
problem situation and that such problem situations should be cognitively
engaging while also at a level that would not be overly demanding for the
student to carrying out, monitor, and describe solution processes.

3.3. Procedure
Students were involved in a two-week enrichment period where they en-
gaged in journal writing, focusing on describing their thought processes
while solving mathematics problems. At the beginning of ten class ses-
sions, students were given a mathematics problem to solve. Students were
instructed to write every thought that came to mind while solving the
problems. Students were given approximately ten minutes for the exer-
cises, though students who needed more time were allowed to complete
their work. The writing samples were collected each day and read by the
teacher who provided comments and questions related to the depth of
the details provided in the written descriptions of the students’ problem
solving processes. Students were encouraged to expand and elaborate on
steps where descriptive information was vague or nonexistent. In general,
the focus was to encourage students to extend both the quantity and qual-
ity of the descriptions of their mathematical thinking: providing details
about processes, justifying and reasoning about the steps taken in the prob-
lems. The problems along with the teacher’s feedback were returned during
the next class. Students were encouraged to read the comments, build on
strengths during subsequent writing sessions while also addressing the
teacher’s questions and comments intended to extend the students’ de-
scriptions. Since students routinely discussed problem solving approaches
and strategies in the classroom, this emphasis on writing exposed students
to the use of writing as a mode to describe their mathematical thinking.
The intensive writing focus was intended to facilitate students’ ability to
describe their problem solving processes orally as well as in writing. Ad-
ditionally, all students were given two trial sessions providing oral de-
scriptions of their problem solving processes while being videotaped. The
 COMPARISON OF PROBLEM SOLVING PROCESSES 33

purpose of these sessions was to help students feel at ease with a cam-
era recording their oral descriptions and to provide some basic comments
related to how their verbal descriptions could be extended.
Data was collected over a six-day period with students divided into
groups as described. Students alternated in providing oral descriptions or
written descriptions as they worked the problems. For example, students
in Group 1 solved one of the difficult problems and provided oral descrip-
tions of their problem solving processes while Group 2 provided written
descriptions for the problem. The groups then switched the type of descrip-
tions provided for the second problem, also rated as difficult. This process
continued for the two medium level and two easy level problems. For each
problem there was a total of twenty student responses, ten from the group
providing verbal descriptions and ten from the group providing written
descriptions. Each student gave both an oral and written description of a
problem with a low, medium, and high level of difficulty.
Students who provided written descriptions on a given day worked the
problems individually in the classroom while those students who provided
oral descriptions were supervised in the library. This prevented students
from having any prior knowledge of the prompt for that day. For all prob-
lems, students were provided paper, calculators, and pencils without erasers
to discourage erasing of work. In the writing sessions, students were presen-
ted with the problem and read the following statement:

You have been given a mathematics problem to solve. Please record any work
for the problem on the paper provided just as you did in the practice sessions.
Remember to write everything which comes to mind during the solving of the
problem.

If any students were observed not writing for fifteen seconds, they were
prompted, “Please write what you are thinking.” The problems were col-
lected at the end of the session and labeled using codes for identification
purposes so that all responses from an individual student would be identi-
fiable along with the student’s race and gender, but protecting the personal
identity of the student.
Subjects in the think-aloud groups were taken one at a time from the
library to an adjoining media room where their problem solving sessions
were videotaped. The student was given the problem and materials. The
following instructions were read by the researcher:

You have been given a mathematics problem to solve. You may do any necessary
computations and work on the paper provided. While you are solving the problem,
please think out loud by telling everything that comes to mind while you are
solving the problem.
34 DAVID K. PUGALEE

The student was videotaped while solving the problem with the camera
focusing on the student’s work. If the student was silent for fifteen seconds,
s/he was prompted, “Please tell me what you are thinking.” When the
student finished, s/he was sent back to class with care taken that s/he did
not communicate with any students who were waiting to complete their
videotaped problem solving session. All work was collected and coded for
identification.

3.4. Analysis of the data

The videotaped sessions were transcribed and categorized by student and
problem. The data also included any samples of work generated by the
student during the problem solving session. These data were analyzed at
two levels: identification of error patterns and strategies employed by the
students, and categorization of data components using qualitative method-
ologies to identify similarities and differences in the students’ responses.
Additionally, the frequency of correct and incorrect final answers for each
of the problems was recorded.
The researcher and a trained rater reviewed all problems to determ-
ine the error patterns and the strategies used by the students. The rater
was a college graduate with extensive experience working with a private
company specializing in scoring tests and inventories. For practice, the
rater worked through five sample problems, under the direction of the
researcher, obtained from the practice sessions preceding the collection
of data. The analysis involved a content analysis of students’ work fo-
cusing on conceptual and procedural errors (Porter and Masingila, 2000).
Three categories were utilized: procedural, computational, and algebraic.
Computational errors were a subset of procedural errors separated for de-
scriptive purposes. Procedural errors primarily consisted of using an incor-
rect mathematical operation in executing a plan. Algebraic errors included
conceptual and algorithmic problems in selecting an appropriate model,
representing information, and application of algebraic principles. The re-
searcher and rater agreed on 91.2% of the ratings. The rater also used the
strategies identified by the problem selection panel as likely to be used
by students in solving the mathematics problems. The categories included
using a diagram, table, list, or other visual; guess and check; working back-
wards; searching for a pattern; logical reasoning; and other. The researcher
and rater agreed at a rate of approximately 98% in categorizing the primary
strategy used in a problem.
The second level of analysis employed a qualitative process of identify-
ing pieces of data based on their homogeneity and heterogeneity (Patton,
2001). Determining which pieces of data were similar led to categories
 COMPARISON OF PROBLEM SOLVING PROCESSES 35

of data where the differences in the categories were separate and evident
while also demonstrating categories where the data interlocked in a mean-
ingful way (Patton, 2001). This process of grouping the data into themes
provided a link to patterns that could be induced from the categories (Gay
and Airasian, 2003). Five categories emerged with four corresponding to
the metacognitive framework used by Garofalo and Lester (1985) to de-
scribe mathematical problem solving processes: orientation, organization,
execution, and verification. The fifth category consisted of responses that
represented either students’ affective statements or filler phrases.
This initial data analysis of the transcribed verbal and written responses
from the problem solving sessions provided data components in the four
broad categories already mentioned. Data in each of these four categor-
ies were further classified into sub-categories using Garofalo and Lester’s
(1985) metacognitive framework. The fifth category which included filler
statements such as uhhh, ok, ummm and similar terms was not included
in this subsequent analysis. It might be noted, however, that students giv-
ing verbal descriptions of their problem solving processes used more such
fillers, 62 instances versus 41 occurrences in the written data. The categor-
ized data pieces varied in length from two sentences to sentence fragments
of a few words. The pieces were color-coded based on the type of metacog-
nitive behavior represented. Totals for the categories and sub-categories
were tabulated.
A university-level mathematics educator reviewed a random sample of
30% of the students’ work categorizing data segments using the same
qualitative methods. This process included categorizing data items into
one of the five broader categories, discussing discrepancies and reaching a
consensus on this initial classification, further classifying the data into sub-
categories from the metacognitive framework, discussing discrepancies in
this subsequent categorization and reaching a final consensus. An equal
number of problems in this random sample came from the writing group
and the ‘think-aloud’ group. Initial agreement was at 88% and all discrep-
ancies were satisfactorily resolved through reanalysis and discussion.

4. F INDINGS AND DISCUSSION

4.1. Strategies and error types

The strategies used by students did not vary greatly between those who
provided written or verbal descriptions of their problem solving processes.
For four of the six problems, more than 90 percent of the students used
the same strategy for that particular problem. For two of the problems, 1
 36

TABLE I
Frequencies of problem solving strategies

Diagram, Guess and Work Search for Logical Other

table, list, check backwards pattern reasoning
other visual

Problem 1 7 – Verbal none none 2 – Verbal 2 – Verbal 1 – Verbal

5 – Written 3 – Written 1 – Written 2 – Written
Problem 2 2 – Verbal none 1 – Verbal none 9 – Verbal none
2 – Written 2 – Written 9 – Written
Problem 3 4 – Verbal none none 9- Verbal 1 – Verbal 2 – Verbal
4 – Written 10 – Written 1 – Written 0 – Written
Problem 4 none 9 – Verbal none none 1 – Verbal none
DAVID K. PUGALEE

10 – Written 0 – Written
Problem 5 none 8 – Verbal none 2 – Verbal 1 – Verbal none
9 – Written 1 – Written 1 – Written
Problem 6 0 – Verbal 3 – Verbal none none 4 – Verbal none
1 – Written 0 – Written 3 – Written
Totals 13 – Verbal 20 – Verbal 1 – Verbal 13 -Verbal 18 -Verbal 3 -Verbal
12 – Written 19 – Written 2 – Written 14 – Written 15 – Written 2 – Written
 COMPARISON OF PROBLEM SOLVING PROCESSES 37

TABLE II
Frequency of error types by group

Algebraic Computational Procedural∗∗ None∗∗

Think-Aloud 4 10 30 20
Written 4 7 19 32

∗∗ Test of significance between proportions, significant at p < .05.

and 6, the percentage of students using the same strategy was 78 and 70.
These two problems also had the lowest rate of success. The strategy used
the least by the students was ‘working backwards’ which was only used
for problem number 2. The raters selected working backwards as the least
likely strategy to be used. The other strategies ranged from twenty-five,
approximately 19% of total, for using diagrams, tables, lists, and other
visuals to thirty-nine, approximately 30%, for guess and check. In general,
strategies actually used by students were consistent with the raters’ pre-
dictions about strategy use with some differences in the rankings between
predicted and actual strategy used. The raters selected making diagrams /
tables / other visuals, guess and check, and logical reasoning (in that order)
as the three most likely strategies to be used by students. Students actually
used guess and check most frequently followed by logical reasoning and
diagrams / tables / other visuals. Table I shows the primary strategy em-
ployed for each problem across the two reporting methods: think-aloud
and writing. A two proportion z test was computed for each strategy to
determine if there were significant differences in the strategies employed
based on whether the think-aloud or written reporting method was used.
There were no significant differences in the strategies used: the resulting
one tailed p values were all greater than .28.
An analysis of the types of errors made by the students indicated that the
majority of errors was procedural (66.2% of all errors) followed by compu-
tation (23%) and algebraic (10.8%). Table II shows total frequency across
all problems by type of error and reporting method. A test for differences
between proportions revealed a significant difference in procedural errors
based on students’ process reporting method. Students written descriptions
resulted in significantly fewer procedural errors when compared to think-
aloud descriptions (z = 2.222, p < 0.05) (see Bruning and Kintz, 1987).
There were no statistical differences between the two reporting methods
on algebraic and computation errors. All problems were scored for the
correct final answer. Those students who wrote about their problem solving
processes produced 32 correct solutions out of the total of 60 solutions,
38 DAVID K. PUGALEE

TABLE III
Frequency of metacognitive behaviors

Think-aloud Written
process process

ORIENTATION
Reading/Rereading 12 25
Initial/Subsequent Representation 9 10
Analysis of Information & Conditions 9 20
Assessment of Difficulty 11 16
Total Orientation∗∗ 38 71
ORGANIZATION
Identification Goals/Subgoals 11 22
Making/Implementing Global Plans 60 35
Data Organization 6 15
Total Organization 77 72
EXECUTION
Performing Local Goals 37 64
Monitoring Goals 14 31
Calculations 75 73
Redirecting 1 4
Total Execution∗∗ 127 172
VERIFICATION
Evaluating Decisions 42 18
Checking Calculations 2 3
Total Verification∗∗ 44 21

∗∗ Significant at p < 0.05.

whereas the think aloud students produced 20 correct solutions. A test for
significance between two proportions was conducted resulting in a z score
of 2.2126, significant at the .05 level.

4.1.1. Describing the data using the metacognitive framework

The four major categories of the framework include sub-categories that
further delineate the metacognitive behaviors indicative of each of the four
phases. Each of these is described in the following paragraphs using repres-
entative data to illustrate the types of student responses that are indicative
of each of these metacognitive behaviors. This descriptive process also
includes a comparison of the frequencies of these behaviors depending on
the type of reporting method (verbal or written) employed when the student
was engaged in solving the problem. Table III shows the frequencies for
the sub-categories and four phases for the two reporting methods. Due to
the small frequencies in some of the cells for the sub-categories, a test for
 COMPARISON OF PROBLEM SOLVING PROCESSES 39

significance between proportions was only conducted for each of the four
phases: orientation, organization, execution, and verification.

4.2. Orientation phase

The orientation phase represents initial attempts of the student to become
familiar with the problem situation. Metacognitive behaviors associated
with this category include reading/rereading, initial and subsequent rep-
resentations, analysis of information and conditions, and assessment of
problem difficulty. The total number of orientation behaviors was signi-
ficantly higher for students providing written descriptions (z = 4.470, p <
0.05).
Reading and rereading behaviors were more evident in students who
described their problem solving processing through writing, indicating that
the problem was reread more than twice as often as the ‘think-aloud’
group. The initial reading of the problem was not coded. The data indicates
that students placed high importance on rereading. A student in the think-
aloud group stated, “Well, first of all I do not understand the problem so I
will reread it.” Another student in the writing group disclosed, “I read this
problem three times.”
Analysis of information and conditions statements was more prevalent
for the writing group. Comments from both groups for the digit problem
(number 1) show similarity in the content of such statements. A student
using ‘think-aloud’ added, “For 1 through 9 that would be single digits”
while another student wrote, “There is 1 of every number including zero
every 10 numbers.” Such statements suggest that metacognitive behavi-
ors of this type press students toward an initial understanding of how
information in the problem can be applied in developing a solution plan.
Initial/subsequent representations continued the pattern of being more
than twice as prevalent among the written responses. For both groups,
initial and subsequent representations involved restating information from
the problem. “The apartments are numbered from 1 through 99” was vo-
calized by one student. Another student wrote, “I said if she falls 54 meters
per second without the parashoot and 6 meters per second with it.” Such
statements illustrate students’ attending to information in the problem. As-
sessing problem difficulty was evident at relatively equivalent levels for
both groups. Behaviors in this category also represented the fewest num-
ber of responses in the orientation category. There were few statements
about the level of difficulty of the problem. Statements directly related
to problem difficulty were restricted to the two problems with the lowest
success rate, problems one and six. Statements across both groups were
comparable including short comments about the problem being “hard”.
40 DAVID K. PUGALEE

Behaviors in these two subcategories, analysis of information and condi-

tions and assessment of difficulty, seem to be indicative of students sorting
through the information and focusing on how to initially represent key
information from the problem.

4.2.1. Organization phase

The organization phase includes metacognitive behaviors such as identific-
ation of goals and subgoals, making and implementing global plans, and
data organization. In general such behaviors assist the student in under-
standing how information in the problem relates to the problem-solving
task including the formulation of goals and plans. There was no signific-
ant difference in the total number of responses between the two reporting
methods for the organization phase (z = .579, p > .28).
For identification of goals and subgoals, protocols from the writing
group were twice as frequent as those from the think-aloud group. Goals
for the apartment numbers problem included such statements as “How
many numbers does she have to buy,” “So, I can count every number,”
and “You go up a number each time in the pattern.” Statements in this sub-
category denoted an overall plan that would lead to a successful solution
for the problem.
The subcategory of making and implementing global plans involved
more specific information than identification of goals and subgoals. Be-
haviors in this category reflected students’ attention to more specific ideas
about how to carry out the broader goals that reflected how to best approach
solving the problem. This category shows a higher frequency of statements
for the think-aloud group. Further investigation shows that some of these
verbalized statements reflected the restatement of plans. For example, a
student for problem five stated that she was “going to select random num-
bers to equal 14.” Later, she states that she is “lost” and reiterates that she
is going to “try another number.” There were four such occurrences in the
verbal protocols of similar restatements while there were no observations
noted for the writing group. The robust performance of a student verbaliz-
ing his solution to number six also contributed to a high frequency in this
category as he changed his local plan several times in an attempt to solve
the problem.

Let’s say she falls 54 meters a second before her chute opens for 10 seconds and
she falls 90 seconds after her chute opens. . . . I’m going to try something else. I
am going to try 20 seconds before her chute opens. . . . I’m going to try 19. She
dropped 19 seconds before her chute opened. . . . So, I’m going to do 16. . . . I was
going to see if you could say he was like 17.5 seconds. . . . I’m going to try since I
need to go up a little bit I’m going to try 163 /4 before her chute opens and 821 /4
seconds after chute open. . .
 COMPARISON OF PROBLEM SOLVING PROCESSES 41

This student’s performance accounted for approximately ten percent of

the coded segments for making/implementing global plans by think-aloud
students. The extent to which this student evaluated and reevaluated his
plan was not matched by any other student in either group. Repetitious
statements and this robust performance appear to account for the difference
in frequencies between the two groups.
Responses labeled as data organization were more evident in the stu-
dents’ written responses. Data organization actions included listing in-
formation about the number of digits or numbers for problem one and
making dots for the triangular numbers in problem three. Though differ-
ing in quantity, these processes were similar regardless of the reporting
method. Two responses (for problem two) from the writing group included
diagrams with labels showing the distances between the cities. There were
no diagrams constructed by any of the students who used think-aloud to
describe their problem solving processes.

4.2.2. Execution phase

The execution phase includes metacognitive behaviors such as performing
local goals, doing calculations, monitoring goals and redirecting plans.
This category accounted for the highest number of coded data statements.
While the writing group generated more statements classified as perform-
ing local goals, monitoring goals, and redirecting, the think-aloud group
had more instances of performing calculations. There was a significant
difference in the proportion of execution behaviors with approximately
35% more for the written descriptions (z = 3.680, p < 0.05).
Performance of local goals and calculations are intrinsically related in
the students’ problem solving descriptions; therefore, they are discussed
together. For example, a student verbalized, “So, I’ll try 7 + 7. Then 7 +
16. 16 is in the lower left-hand circle. That is 23.” This movement back and
forth between executing local goals and performing calculations is further
elucidated in the following student’s response to the digit problem:
So there are 9 sets of 10 up to 90 + 9 more numbers to 99. So I multiply 1 ×
10 sets of 1s and that give me 10 ones plus 10 more counting the numbers in the
teens. That gives me 20 ones. That process is repeated for the following numbers
1–9. 0 is an exception. There are only 10 0s because it only used for numbers 0,
10, 20, 30, 40, 50, 60, 70, 80, 90. So she should buy 20 of each digit except zero
where she should buy only 10 of those.
Problem solving actions classified as redirecting plans result from an ef-
fective monitoring of goals. There were few redirecting behaviors for stu-
dents using either process. Several such examples are found in the written
descriptions for problem 1. In one of these occurrences, the student had
begun counting the number of digits needed for the apartment numbering
42 DAVID K. PUGALEE

by considering all ones and then all twos. He realizes the output is the same
and changes his approach to finding a pattern. “By looking at the numbers
– all the numbers, they each need twenty digits but zero needs nine digits.”
The other example of redirecting for problem 1 also involved a student
moving beyond a counting method for the apartment numbers to that of
finding a pattern.
Students who wrote descriptions while solving the problems produced
more than twice as many examples of monitoring goals than students who
engaged in think-aloud. One student wrote of his approach for finding the
average in problem 4, “Then what I did was pick some number and average
till I got the lowest he could make.” Monitoring goals was also evident
in more global statements such as the following comment from a student
using think-aloud. “By using the strategy guess and check, I found my
answer and the answer is 96.” The student displays confidence that the
selected strategy was viable in producing an acceptable response. Such
instances demonstrate the metacognitive nature of monitoring behaviors.
While it is difficult to infer the level of monitoring required in the types of
actions described by students using either process, such statements show
the importance of goals in directing students’ thinking about their problem
solving behaviors.

4.2.3. Verification phase

The verification phase consists of metacognitive behaviors such as eval-
uating decisions and results. Because of the non-linear nature of problem
solving, these behaviors were not necessarily found at the conclusion of the
problem solving sessions. Some verification processes are by nature part
of the process of monitoring global and local plans and were categorized
with the related behaviors in other parts of the framework. The behaviors
in this category deal with those segments demonstrating that students were
checking final answers and computations. Verification behaviors accoun-
ted for the smallest number of data cells among the four phases. There was
a significant difference between the two methods with twice as many veri-
fication behaviors for students using the think-aloud description method
(z = 4.034, p < 0.05)
In many cases, students simply indicated that they had engaged in such
a process. “I checked to make sure that I was right. It was right, so I am
finished with this problem” and “I have checked my problem very carefully
and my conclusion is right.” There were few instances of verifying com-
putations. Verification of computations included using subtraction to check
for addition, labeling work as “mistake” and recalculating, and checkmarks
beside computation results on papers from both groups. Statements such
 COMPARISON OF PROBLEM SOLVING PROCESSES 43

as “Might go a little bit higher” and “In order to check you would have
to draw everything out” represent cases where students’ perceptions about
their solutions appeared to indicate some dissonance but did not result in
any follow-up actions that would have changed their incorrect solutions.
The frequency of verification behaviors was not related to success with the
problem solving task.

5. C ONCLUSIONS AND IMPLICATIONS

This study involved analysis of written and think-aloud descriptions of

the problem solving processes of ninth grade algebra students as they
solved mathematical problems. Qualitative analysis showed that both re-
porting methods provide a useful lens for studying the problem solving
behaviors of students. A comparison of the data from the two modes (writ-
ing and think-aloud) reveals some notable differences. Before focusing on
the comparison, there are some pertinent conclusions related to students’
problem solving behaviors regardless of the method used to describe their
thinking. First, there was a relationship between the number of different
strategies attempted by students for a particular problem and their success
in solving that problem. Problems with a high rate of success showed sim-
ilar strategy use among the students; likewise, there was evident variability
in strategies for those problems with low success rates. Second, execution
behaviors (performing local goals, monitoring goals, performing calcula-
tions, and redirecting) comprised the largest number of problem solving
actions. Third, students who constructed global plans were more likely
to be successful at the problem solving tasks whether those plans were
stated or implied in the descriptions of their problem solving processes.
Fourth, students assessed the difficulty at the beginning and end of the
problem solving process suggesting that this behavior is both formative
and summative. Fifth, students did not generally verify the accuracy of
their final answers.
Students who wrote about their problem solving processes produced
correct solutions at a statistically higher rate than when using think-aloud
processes. Students who wrote descriptions of their processes produced
significantly more orientation and execution statements than students who
verbalized their responses. While there was no significant difference be-
tween verbalized and written organization statements, closer inspection
revealed that this difference is largely accounted for in repetitious state-
ments and one student’s robust verbalization for one problem as he worked
through multiple redirects in attempting to reach a solution. In two of the
three subcategories, identification of goals and data organization, written
44 DAVID K. PUGALEE

responses were at least twice as frequent as verbal responses. While there

was a significant difference favoring the verbalization mode for verifica-
tion behaviors, the greater frequency did not have an impact on students’
success with the problem. While students may have stated that they verified
their answer, there is little evidence that such behaviors occurred and few
examples of students redirecting their work.
These findings provide support for the premise that the interaction of
students with metacognitive processes along with an understanding of math-
ematical concepts distinguishes them as successful problem solvers (Schur-
ter, 2002). This study demonstrates that writing can be a tool for supporting
a metacognitive framework and that this process is more effective than the
use of think-aloud processes. Further research is necessary to investigate
those factors that provide for successful implementation of a writing pro-
gram designed to develop metacognitive behaviors. Additional research
is also necessary to determine how and to what extent verbalized pro-
cesses might be used in positively affecting students’ problem solving
performance.

A PPENDIX A

1. Mrs. Maples buys numerals to put on the doors of each apartment in

a 99-unit apartment building. The apartments are numbered 1 through
99. How many of each digit should Mrs. Maples buy?
2. Claire drove from Philadelphia to Brooklyn, then to Manhattan, then
to Scarsdale. She returned over the same route. From Philadelphia to
Brooklyn is 93 miles. From Brooklyn to Manhattan is 11 miles. The
trip from Scarsdale to Philadelphia was 131 miles. What is the distance
between Manhattan and Scarsdale?
3. These are the first four triangular numbers. What is the tenth triangular
number?

4. Kevin’s scores on 4 tests were 86, 72, 94, and 77. He wants his average
on five tests to be at least 85. What is the lowest score he can get on
the fifth test to achieve this average?
 COMPARISON OF PROBLEM SOLVING PROCESSES 45

5. The numbers in the big circles are found by adding the numbers in
the two smaller circles on each side. Find the numbers for the small
circles.

6. Skydivers fall at 54 meters per second before their chutes open. They
fall at 6 meters per second after their chutes open. If a skydiver jumped
from a plane 1400 meters high and reached the ground in 100 seconds,
how high was she when she opened her chute?

R EFERENCES

Allen, B.R.: 1991, A Study of Metacognitive Skill as Influenced by Expressive Writing

in College Introductory Algebra Classes, Unpublished doctoral dissertation, Louisiana
State University and Agricultural And Mechanical College.
Applebee, A.N.: 1981’, Writing in the Secondary School, National Council of Teachers of
English, Urbana, IL.
Artz, A.F. and Armour-Thomas, E.: 1992, ‘Development of a cognitive-metacognitive
framework for protocol analysis of mathematical problem solving in small groups’,
Cognition and Instruction 9(2), 137–175.
Barnett, J.: 1979, ‘The study of syntax variables’, in G.A. Goldin and C.E. McClintock
(eds.), Task Variables in Mathematical Problem solving, ERIC Clearinghouse for Sci-
ence, Mathematics, and Environmental Education, Columbus, OH, pp. 23–68. (ERIC
Document Reproduction Service No. ED178366) Retrieved July 22, 2003, from the
ERIC database.
Brown, T.: 1997, Mathematics Education and Language: Interpreting Hermeneutics and
Post-Structuralism, Kluwer Academic Publishers, Dordrecht, The Netherlands.
Bruning, J.L. and Kintz, B.L.: 1987, Computational Handbook of Statistics, Scott,
Foresman and Company, Glenview, IL.
Carpenter, T.P. and Lehrer, R.: 1999, ‘Teaching and learning mathematics with under-
standing’, in E. Fennema and T. Romberg (eds.), Mathematics Classrooms that Promote
Understanding, Lawrence Erlbaum Associates, Mahweh, NJ, pp. 19–32.
Carr, M. and Biddlecombe, B.: 1998, ‘Metacognition in mathematics: From a constructivist
perspective’, in D.J. Hacker, J. Dunlosky and A.C. Graesser (eds.), Metacognition in
Educational Theory and Practice, Lawrence Erlbaum Associates, Mahweh, NJ.
Davidson, J.E. and Sternberg, R.J.: 1998, ‘Verbalization and problem solving’, in D.J.
Hacker, J. Dunlosky and A.C. Graesser (eds.), Metacognition in Educational Theory
and Practice, Lawrence Erlbaum Associates, Mahweh, NJ, pp. 47–68.
Emig, J.: 1977, ‘Writing as a mode of learning’, College Composition and Communication
28, 122–127.
46 DAVID K. PUGALEE

Flower, L.: 1989, ‘Taking thought: The role of conscious processing in the making of
meaning’, in E.P. Maimon, B.F. Nodine and F.W. O’Connor (eds.), Thinking, Reasoning
and Writing, Longman, New York, pp. 185–212.
Flower, L. and Hayes, J.R.: 1983, ‘Uncovering cognitive processes in writing: An introduc-
tion to protocol analysis’, in P. Mosenthal, L. Tamor and S.A. Walmsley (eds.), Research
on Writing, Longman, New York.
Gay, L.R. and Airasian, P.: 2003, Educational Research: Competencies for Analysis and
Applications, Pearson Education, Inc., Upper Saddle River, NJ.
Garofalo, J. and Lester, F.: 1985, ‘Metacognition, cognitive monitoring and mathematical
performance’, Journal for Research in Mathematics Education 16(3), 163–176.
Gravemeijer, K., Cobb, P., Bowers, J. and Whitenack, J.: 2000, ‘Symbolizing, modeling,
and instructional design’, in P. Cobb, E. Yackel and K. McClain (eds.), Symbolizing and
Communicating in Mathematics Classrooms, Lawrence Erlbaum Associates, Mahwah,
NJ, pp. 225–273.
Kulm, G.: 1979, ‘The classification of problem-solving research variables’, in G.A. Goldin
and C.E. McClintock (eds.), Task Variables in Mathematical Problem Solving, ERIC
Clearinghouse for Science, Mathematics and Environmental Education, Columbus, OH,
pp. 1–22. (ERIC Document Reproduction Service No. ED178366) Retrieved July 22,
2003, from the ERIC database.
Masingila, J.O. and Prus-Wisniowska, E.: 1996, ‘Developing and assessing mathemat-
ical understanding in calculus through writing’, in P.C. Elliott and M.J. Kenney (eds.),
Communication in Mathematics, K-12 and Beyond, National Council of Teachers of
Mathematics, Reston, VA, pp. 95–104.
Meir, J. and Rishel, T.: 1998, Writing in the Teaching and Learning of Mathematics, The
Mathematical Association of America, Washington, DC.
Miller, L.D. and England, D.A.: 1989, ‘Writing to learn algebra’, School Science and
Mathematics 89(4), 299–312.
Morgan, C.: 1998, Writing Mathematically: The Discourse of Investigation, Falmer Press,
London.
National Council of Teachers of Mathematics: 2000, Principles and Standards for School
Mathematics, author, Reston, VA.
National Council of Teachers of Mathematics: 1989, Curriculum and Evaluation Standards
for School Mathematics, author, Reston, VA.
O’Connor, M.C.: 1998, ‘Language socialization in the mathematics classroom: Discourse
practices and mathematical thinking’, in M. Lampert and M.L. Blunk (eds.), Talking
Mathematics in School: Studies of Teaching and Learning, Cambridge University Press,
Cambridge, UK, pp. 17–55.
Patton, M.Q.: 2001, Qualitative Research and Evaluation Methods, Sage, Thousand Oaks,
CA.
Porter, M.K. and Masinglila, J.O.: 2000, ‘Examining the effects of writing on conceptual
and procedural knowledge in calculus’, Educational Studies in Mathematics 42(2), 165–
77.
Powell, A.B.: 1997, ‘Capturing, examining and responding to mathematical thinking
through writing’, The Clearing House 71(1), 21–25.
Pugalee, D.K.: 2001, ‘Writing, mathematics and metacognition: Looking for connec-
tions through students’ work in mathematical problem solving’, School Science and
Mathematics 101(5), 236–245.
 COMPARISON OF PROBLEM SOLVING PROCESSES 47

Rose, B.J.: 1989, ‘Writing and math: Theory and practice’, in P. Connolly and T. Vilardi
(eds.), Writing to Learn Mathematics and Science, Teachers College Press, New York,
pp. 15–30.
Schurter, W.A.: 2002, ‘Comprehension monitoring: An aid to mathematical problem
solving’, Journal of Developmental Education 26(2), 22–33.
Sierpinska, A.: 1998, ‘Three epistemologies, three views of classroom communication:
Constructivism, sociocultural approaches, interactionism’, in H. Steinbring, M.G. Bar-
tolini Bussi and A. Sierpinska (eds.), Language and Communication in the Mathematics
Classroom, National Council of Teachers of Mathematics, Reston, VA, pp. 30–62.
Smagorinsky, P.: 1989, ‘The reliability and validity of protocol analysis’, Written Commu-
nication 6(4), 463–479.
Vygotsky, L.S.: 1987, ‘Thinking and speech’, in R.W. Rieber and A.S. Carton (eds.), The
Collected Works of L.S. Vygotsky, Plenum Press, New York, pp. 39–243.
Webb, N.: 1979, ‘Content and context variables in problem tasks’, in G.A. Goldin and
C.E. McClintock (eds.), Task Variables in Mathematical Problem Solving, ERIC Clear-
inghouse for Science, Mathematics and Environmental Education, Columbus, OH, pp.
69–102. (ERIC Document Reproduction Service No. ED178366) Retrieved July 22,
2003, from the ERIC database.

University of North Carolina Charlotte,

9201 University City Blvd. Charlotte, NC 28223-0001,
Telephone 704-593-1283, Fax 704-687-3410,
E-mail dkpugale@email.uncc.edu