Académique Documents
Professionnel Documents
Culture Documents
ANALYSIS
by
Lauren A. McGuire
Doctor of Philosophy
Capella University
May 2010
UMI Number: 3408479
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
UMI 3408479
Copyright 2010 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106-1346
© Lauren A. McGuire, 2010
Abstract
This study investigated the effect of direct instruction in rhetorical analysis on students’
critical thinking abilities, including knowledge, skills, and dispositions. The researcher
Guides, based on Paul’s model of critical thinking; and Socratic questioning. The explicit
goals of the study were to promote critical thinking knowledge, skills, and dispositions
skills, and dispositions. A mixed methodology teacher action approach was used in this
one-group pretest-posttest design research study which consisted of a small sample size
(N = 15) and pertained to a 1-semester critical thinking college course that was taught by
the researcher in a naturalistic setting. Changes in critical thinking were evaluated using
the pretest and posttest scores on the California Critical Thinking Skills Test 2000 and the
Pre- and Post- Critical Thinking Surveys. Data from the research findings indicated there
were no significant differences between the pretest and posttest scores for induction,
analysis, and evaluation; however, there was a slight difference between the pretest and
posttest scores for inference and deduction with a negligible significance level of (p =
0.2). Findings from the Pre- and Post- Critical Thinking Surveys indicated a 2.2 mean
gain in critical thinking knowledge, a 1.8 mean gain in critical thinking skills, and a 1.8
mean gain in critical thinking dispositions. These data suggest that purposeful
mentor during the dissertation process and who has been truly inspirational. I will be
forever grateful for her knowledge, expertise, insightful guidance, and patience. She has
had her as my mentor. I also wish to thank the other members of my committee, Dr.
Maureen McGlynn and Dr. Howard Jacobs, for their positive feedback, assistance, and
counsel. Finally, I wish to thank my husband, Terry, for his love and support and for his
unremitting encouragement and tolerance throughout this long, arduous, yet gratifying
educational journey.
iv
Table of Contents
Acknowledgments iv
List of Figures ix
CHAPTER 1. INTRODUCTION 1
Statement of Problem 7
Research Questions 8
Definition of Terms 9
Theoretical Framework 14
Introduction 20
Specific Strategies 38
v
Conclusion 49
CHAPTER 3. METHODOLOGY 52
Introduction 52
Statement of Problem 52
Research Questions 53
Instrumentation 57
Field Test 60
Data Collection 60
Data Analysis 63
Ethical Considerations 64
Summary 66
Introduction 68
Research Questions 69
Demographic Characteristics 72
Results 75
Summary 96
vi
CHAPTER 5. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS 98
Introduction 98
Overview of Methodology 99
Summary of Findings 99
Limitations 119
Recommendations 120
Conclusions 123
REFERENCES 125
vii
List of Tables
viii
List of Figures
ix
CHAPTER 1. INTRODUCTION
abilities is of utmost importance and that critical thinking “is central to both personal
success and national needs” (Paul, 2004, p. 2). In his article, Paul (1995f) cited
effectively develop student critical thinking knowledge, skills, and dispositions across the
pedagogy on a consistent basis could enable instructors with the ability to encourage in
their students the transfer of those critical thinking skills learned in the academic
thinking abilities, however, educators must possess a clear definition of what critical
thinking is. As the concept of critical thinking is highly complex, a variety of definitions
exist, so it is difficult to pinpoint the exact meaning of the skills involved in this intricate
1
process. P. A. Facione (2000), using the two-sentence definition of critical thinking
generated from the Delphi Report, identified critical thinking as “the process of
evidence, context, conceptualizations, methods, and criteria” (p. 5). P. A. Facione argued
that a true definition of critical thinking involves both skills and habits of mind or
dispositions. Similar to P. A. Facione and his concern with the necessity for an
inclination toward critical thinking, Paul (1993) asserted that critical thinking is an
analyzing, synthesizing or evaluating information” (p. 3). Halpern (1999) believed that
critical thinking is “purposeful, reasoned, and goal-directed” (p. 70), while maintaining
that “it is the kind of thinking involved in solving problems, formulating inferences,
calculating likelihoods, and making decisions” (p. 70). In a statement in 1997 for the
National Council for Excellence in Critical Thinking Instruction, Scriven and Paul (1987)
reason at the highest level of quality in a fair-minded way” (p. 1). As with P. A. Facione,
Scriven and Paul contended that “critical thinking varies according to the motivation
underlying it” (p. 1), it is “never universal in any individual” (p. 1), and “the development
of critical thinking skills and dispositions is a life-long endeavor” (p. 1). Critical thinking
is not something that can be learned immediately; however, educators can encourage
students to be aware of their thinking so that it becomes intentional rather than accidental.
Critical thinking is a complex skill that takes a lifetime to master and that involves
2
The greatest contribution of the community college, according to Elder (2000),
has been its emphasis on teaching students those abilities needed for competency in the
workplace, yet Elder claimed that “training students for job performance in narrowly
defined skill areas no longer serves students well” (p. 1). Rather, current instructional
practices need to begin to place more emphasis on those critical thinking skills necessary
for survival in the new economy. What follows is the necessity for educators to begin to
teach those intellectual skills that encourage mental flexibility and intellectual discipline
struggle with concepts, find meaning, distinguish bias, and use logic in arguments so they
may gain a deeper understanding of the world in which they live. The question is what
thinking abilities while attempting to include critical thinking curriculum into a variety of
academic disciplines. Although colleges and universities offer critical thinking courses,
and faculty can engineer their course focus so that it is more thinking-skills-based
(Halpern, 1999). Mendelman (2007) believed critical thinking should be taught in every
In a day and age in which more and more children grow up engaged with
primarily passive activities like television, video games, and the Internet, teaching
critical reading is one of the most important, and most difficult, burdens of the
classroom. (p. 300)
3
Critical thinking abilities are necessary for success in college and for responsible
literacy and cognitive skills required for academic success or for successful living in a
global economy. National assessments indicate that “although students can perform basic
skills pretty well, in international comparisons, American students are not doing well at
p. 3). Van Gelder (2005) maintained that “a majority of people cannot, even when
prompted, reliably exhibit basic skills of general reasoning and argumentation” (p. 2).
Students are, or seem to be, unable to provide the essential evidence needed to support
their reasoning. Jackson (2008) discussed the need for educators to provide opportunities
for students to grapple with complexities so they may cultivate their problem-solving
Technology (NJIT). Elliot had launched a vigorous campaign at NJIT for information
literacy. His chief concern with students was their seeming inability to “sift, use, and
make meaning out of the information around them, in other words to read their world” (as
observed that
4
The writing portfolios of NJIT’s seniors scored an average of 6.14 on a scale of
12, or just below “satisfactory.” Students could find and cite sources better than
they were able to judge their relevance and authority, and were even less able to
use information to support their argument. (as cited in Jackson, 2008, p. 165)
College students, when confronted with controversial issues, are all too often prone to
disciplined thought and to teach students that “uncertainty is an ally of good thinking and
skills, and dispositions may improve students’ academic success while encouraging those
abilities needed for transfer and for competency in the workplace. Elder (2000)
contended, however, that traditional education is not nurturing the intellectual capabilities
needed for personal and academic success. Often, students are merely asked to write
down facts rather than to question or reflect on their reading, and, as a result, they are
literature (both fiction and nonfiction prose) they read. Elder suggested further that “as
the economic structure of the world becomes more complex” (p. 1) and “as we become
increasingly more interdependent both at home and abroad, ‘training’ students for job
performance in narrowly defined skill areas no longer serves students well” (p. 1).
Willsen (1997) discussed the increasing complexities of the world and suggested
these new global realities “are rapidly working their way into the deepest structures of
implications for teaching and learning” (p. 1). The question is whether educational
institutions are preparing students to adapt to and accommodate for these complexities.
5
Cultivating students’ abilities to develop higher order thinking skills and problem-solving
strategies could better prepare students for survival in this complex world. As Willsen
(1997) reasoned
We must sooner or later abandon the traditional attempt to teach our fellow
citizens what to think. Such efforts cannot prepare us for the real world we must,
in fact, face. We must concentrate instead on teaching ourselves how to think,
thus freeing us to think for ourselves, critically, fairmindedly, and deeply. We
have no choice, not in the long haul, not in the face of the irrepressible logic of
accelerating change and increasing complexity. (p. 16)
critical thinking abilities. Sternberg (2003), in his discussion of the future of education in
the United States, argued that educational institutions far too often emphasize rote
memorization; while “rote memorization requires recital and repetition” (p. 1), critical
thinking “requires skillful analysis, evaluation, and interpretation” (p. 1). Although all
individuals need a “knowledge base” (Sternberg, 2003, p. 1). or store of information, this
knowledge base must prove useful for living, and instead of emphasizing the
that students can transfer into their lives. Rather than being taught how to think critically,
students are far too often being taught to do little more than recall and recognize; they
2003, p. 1) and, as a result, they may tend to “act on their prejudices and their fears”
6
Statement of Problem
learners’ critical thinking knowledge, skills, and dispositions. This study explored
course could make a difference in students’ critical thinking skills. The interventions
included argument mapping; Thinker’s Guides, based on Paul’s (1995f) model of critical
thinking; and Socratic questioning. These interventions were supplemented with course
The purpose of this study was to determine whether there would be improvement
in students’ critical thinking abilities, including knowledge, skills, and dispositions, after
argument mapping; Thinker’s Guides, based on Paul’s (1995f) model of critical thinking;
and Socratic questioning. The goal of the study was, on the one hand, to promote critical
thinking knowledge, skills, and dispositions, and, on the other hand, to determine
dispositions so that they may transfer those skills across the academic disciplines and into
7
Research Questions
1. What were the students’ perceptions of critical thinking before and after direct
instruction in argument mapping?
2. What were the students’ perceptions of critical thinking before and after direct
instruction in Paul’s Thinker’s Guides?
3. What were the students’ perceptions of critical thinking before and after direct
instruction in Socratic questioning?
4. What did the students report about the changes in their knowledge, skills, and
dispositions relating to critical thinking after the direct instruction in rhetorical
analysis?
critical mind. The American educational system has been a target for reform to teach
students to think critically, and the educational arena is beginning to take the issue of
increasing demands and accelerating changes occurring in the 21st century, educators are
practices would be most effective for the instruction and assessment of critical thinking.
Paul (1995a) believed that “only through an explicit shift to a critical conception
education, can we achieve significant reform” (p. 278). As national assessment of student
achievement far too often focuses on lower order thinking and learning, Paul (1995a)
8
argued that assessment must focus on higher order thinking, reasoning, and authentic
performance. Paul (1995c) further contended that “critical thinking is the heart of well-
conceived educational reform and restructuring because it is at the heart of the changes of
the 21st Century” (pp. 97–98). At the request of the U.S. Department of Education, Paul
(1995f) developed A Model for the National Assessment of Higher Order Thinking. This
paper outlined the problem of lower order learning; explained 21 criteria for higher order
thinking assessment; defended the view that a clear, substantial concept of critical
thinking meets those criteria; articulated four domains of critical thinking; offered
recommendations as to how to assess the domains of critical thinking; and discussed the
engagement in the learning process could encourage student critical thinking abilities and
the transfer of those abilities necessary for academic achievement, personal success, and
success in the work force. Although critical thinking abilities can be encouraged in all
thinking. Using direct instruction in rhetorical analysis, this study adds to the knowledge
in critical thinking and that may alter student perception of critical thinking and promote
Definition of Terms
9
idea that is true or false). The proposition is expressed in some claim, and the evidence is
expressed in other claims” (van Gelder, 2005, p. 4). Paul (1995d) asserted that
informal logic. It is a map that “makes the logical structure of the argument completely
explicit” (van Gelder, 2005, p. 4). These maps are often used in the teaching of reasoning
and critical thinking, and can support the analysis of pros and cons when deliberating
over problems.
Critical thinking. In 1990, a panel of experts, who included 46 men and women
from throughout the United States and Canada and who represented a variety of scholarly
(2006), and the final Delphi Report generated a consensus definition of critical thinking
(P. A. Facione, 2006). The final Delphi Report defined critical thinking as “purposeful,
contextual considerations upon which that judgment is based” (P. A. Facione, 1990, p. 2).
develop the student’s critical thinking, reading, and writing skills beyond the level
achieved in a basic English course. It focuses primarily on the analysis and evaluation of
10
expository and argumentative discourse and on writing analytical and argumentative
essays.
model for teaching that emphasizes well-developed, highly scripted, and carefully
planned lessons designed around small learning increments and clearly defined and
prescribed teaching tasks. Through the use of carefully prescribed instructional practices,
Thinking can be divided into seven components, including purpose; point of view;
interpretations; concepts; and the question at issue. Moreover, thinking can be measured
against standards, such as clarity, accuracy, precision, relevance, depth, breadth, logic,
significance, and fairness (Paul, 1995g). Paul (1995g) maintained that all thinkers should
given case the available means of persuasion” (as cited in Shea, Scanlon, & Aufses, 2008,
exchange of opposing views. Effective rhetoric has a context and a purpose, and it offers
support [a] position, or to move others to take action” (Shea et al., 2008, p. 1). Rhetorical
11
analysis is the act of carefully scrutinizing the tools and techniques the author, speaker, or
visual artist has used to appeal to the audience. It is, according to Shea et al. (2008),
nurturing the student’s ability to discover the “nutritional value in the books, stories,
with challenging tasks. Many theorists believe that scaffolding may improve problem-
solving performance. Ways educators can assist students with difficult problems include
simplifying or dividing a task into smaller units, modeling problem solutions, asking
probing questions, pointing out errors, keeping students focused and motivated, and
18). Socratic questioning encourages rigorous, thoughtful dialogue, and involves the
practice of thoughtful, probing questioning, which enables students to examine the logic
research sample. Although the findings should prove useful, it is doubtful they can be
generalized to a target population in other locations unless the sample sizes share similar
enrolled in courses elsewhere; however, as Gall, Gall, and Borg (2003) maintained,
“Generalizations to other cases can be done, but it must be done on a case-by-case basis”
12
(p. 10). Therefore, generalization may not be possible, nor is it necessary as it is truly up
to the reader to find similarities in an action research study to a different site or location.
Possible limitations exist due to the modest sample size of 15 students and the
that critical thinking is a lifelong process and that successful interventions may need to
occur over an academic year or over several years. Ideally, a longitudinal study would be
dispositions as a result of the interventions. The researcher was the teacher of an intact
class, and no attempt at randomization occurred; therefore, the study was limited to a
teacher action research studies, the data are collected and analyzed to assess and improve
educational practice. Since the researcher was the instructor in the critical thinking
course, there was potential for researcher bias as the researcher may have authority over
the participants. The researcher understood this and attempted to approach the study with
caution and without preconceived assumptions of possible outcomes or results. Care was
taken to maintain clarity and objectivity through sharing with research participants and
further consideration is the Hawthorne effect which refers research situations where
“experimental conditions are such that the mere fact that individuals are aware of
attention improves their performance” (Gall et al., 2003, p. 376). The researcher
accommodated for this concern by providing equal guidance and attention to all students
in the course, and research participants were not given special attention. Finally, this
13
study was limited to examination of the interventions employed in a critical thinking
course and did not attempt to measure the influence of out-of-classroom experiences in
Theoretical Framework
recognizing a problem without bias, and distinguishing the main idea from subordinate
ideas. Analysis is when the learner identifies the intended and inferential relationships
among statements, and includes the examination of ideas, the analysis of arguments and
the ability to examine alternative approaches to a problem and identify any unstated
Inference refers to a person’s ability to construct meaning and to identify the implications
Facione’s method of critical thinking involves two systems that are considered essential
14
characteristics, readily associated ideas, and vivid memories to arrive quickly and
confidently at a judgment” (p. 12). This system is most helpful when decisions must be
made quickly and when immediate action is required. System 2 thinking is more
reflective and is useful when individuals are in “unfamiliar situations and have more time
to figure things out” (P. A. Facione, 2006, p. 13). It allows deliberation, planning ahead,
and considering options; “it is reasoning based on what we have learned through careful
and fair-mindedness” (P. A. Facione, 2006, p. 13). P. A. Facione asserted that individuals
who have a disposition toward critical thinking are inquisitive, systematic, judicious,
Finally, P. A. Facione (2006) asserted that good critical thinkers can misuse those
superior skills in immoral and unethical ways through manipulation and exploitation.
Ultimately, being a good critical thinker is dependent on the character and integrity of the
individual. For this reason, “knowledge and skills are not sufficient. We must look to a
broader set of outcomes including habits of mind and dispositions, such as civic
engagement, concern for the common good, and social responsibility” (P. A. Facione,
2006, p. 11).
15
P. A. Facione’s (2006) definition of critical thinking, generated by the scholars
involved in the Delphi project, was the working definition for the study. The Delphi
Education’s Education Goals: 2000 mandate (P. A. Facione, 2000). The California
Critical Thinking Skills Test–Form 2000 (CCTST–2000), which was used in this study,
derived its construct validity from this critical thinking conceptualization. However,
although the CCTST–2000 targets the core critical thinking skills of analysis,
interpretation, inference, evaluation, and explanation, it does not measure critical thinking
critical thinking; however, the CCTST–2000 proved a valid and reliable instrument for
the research purposes as “high scores on the CCTST are often correlated with a strong
disposition toward critical thinking (high scores on the CCTDI). Individuals with high
scores on the CCTST–2000 and low scores on the CCTDI are relatively rare” (P. A.
Paul and Elder (2006a), leaders in the critical thinking movement, developed a
model for critical thinking that provides a practical approach for developing students’
critical thinking abilities. Paul and Elder applied the core elements of critical thinking and
the characteristics of good critical thinkers from the Delphi Report, generated by scholars
in critical thinking, to address the needs of educators and students (P. A. Facione, 2006).
Paul’s (2004) model insists there are universal standards for critical thinking, which
include clarity, accuracy, precision, relevance, consistency, depth, and breadth. Paul
maintained that critical thinkers analyze, assess, and upgrade their thinking, while
16
advocating that instructors design curriculum so that students understand the sense and
logic of what they learn, and that this learning will increase comprehension and insight.
discover and process information, and provides occasions for students to think through to
Van Gelder (2005) drew on cognitive science for teaching critical thinking, and
reasoned that because critical thinking is a difficult, lifelong process, it requires repeated
engagement, practice, and some theoretical knowledge for transfer. As arguments are
presented in “streams of words, whether written or spoken” (van Gelder, 2005, p. 4), van
out for better, more explicit understanding. Argument maps follow a particular set of
conventions in which the main point is put at the top of the argument tree. Arrows then
indicate that a claim is evidence. The use of the color green and the word reason indicate
they are supporting evidence (van Gelder, 2005). Van Gelder claimed argument maps
clearly and visibly make the logic of arguments more straightforward; as a result,
students’ critical thinking skills and abilities improve when they practice this skill.
In his article, Twardy (2003) claimed that “argument mapping greatly enhances
student critical thinking, more than tripling absolute gains made by other methods” (p. 1).
The results of his study, conducted at the School of Computer Science and Software
Engineering at Monash University in Australia, indicated that students who used van
17
posttest scores using the CCTST. The CCTST is the most widely used measurement of
critical thinking skills as it targets “those core critical thinking skills regarded to be
essential elements in college education” (P. A. Facione, 1992, p. 2). Argument maps
could prove to be useful tools for the analysis of formal arguments as this practice has the
potential to improve students’ ability to closely diagnose and map out rationale while
The mixed-methodology approach was used in this teacher action research study.
The study pertained to one semester of a college critical thinking course that examined
knowledge, skills, and dispositions. The study occurred in a naturalistic setting, and the
sample was comprised of students enrolled in the critical thinking course. A variety of
works were used to encourage analysis of authors’ rhetorical strategies and stylistic
choices. To reflect the increasing importance of graphics and visual images in texts
published in print and electronic media, students were asked to analyze how such images
both relate to written texts and served as alternative forms of texts themselves. The
interventions were supplemented with course work that involved reader/writer workshops
A Pre- Critical Thinking Survey was given at the beginning of the course and a
Post- Critical Thinking Survey was administered at the end of the course. The qualitative
survey was implemented for analysis of patterns of students’ responses; the observations
made by the survey results offered insights into student perceptions of the most effective
18
instructional interventions. The CCTST–2000 was administered as a pretest at the
beginning of the course and measured the students’ critical thinking skills. The same test
was given as a posttest and measured growth in critical thinking abilities at the end of the
course. Statistical analysis was done on the CCTST–2000 pretest and posttest scored by
Insight Assessment.
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
The remainder of the study is organized into four chapters, in the following
manner: Chapter 2 presents a discussion, evaluation and critique of prior research dealing
with the trends in critical thinking, direct instruction in rhetorical analysis, argument
chapter 3 explains how the study was conducted; chapter 4 includes data analysis and
results and presents a report of the data supported by tables and figures and an
recommendations, evaluates the work, and provides insight into and interpretation of the
study’s results.
19
CHAPTER 2. LITERATURE REVIEW
Introduction
Ideally, individuals possess the ability to fairly view issues and their world with
an open mind and from a variety of perspectives. It is unknown whether direct instruction
in critical thinking can meet the challenges educators encounter in their attempts to
encourage and develop those essential abilities for reflective, informed thinking. This
need for promoting students’ critical thinking knowledge, skills, and dispositions, and the
variety of scholars and philosophers (P. A. Facione, 2006; Halpern, 2006; Paul, 1995c;
Perkins, 1993; van Gelder, 2005). This chapter explores critical thinking and is divided
into the following sections: an overview of the critical thinking movement, definitions for
critical thinking, the importance of critical thinking, teaching critical thinking, specific
strategies for teaching critical thinking, and related studies of critical thinking. This is
followed by a brief summary that emphasizes the need for teaching or embedding critical
whether there would be improvement in student critical thinking knowledge, skills, and
dispositions on pretest and posttest scores before and after direct instruction in argument
20
The Critical Thinking Movement
lurked beneath smooth but largely empty rhetoric” (Paul, Elder, & Bartell, 1997a, p. 1).
Socrates questioned the assumptions and beliefs of those in authority and established
probing questions and seeking evidence to examine rhetoric. Socrates’s search for the
influenced the writings of Plato, Aristotle, Thomas Aquinas, Erasmus, Francis Bacon,
Descartes, Sir Thomas Moore, Hobbes and Lock, Robert Boyle, and Sir Isaac Newton
critical thinking theory and education are William Graham Sumner, John Dewey, Edward
Glaser, Jean Piaget, and Lev Vygotsky. Dewey’s pragmatic approach to critical thinking
genuine education comes about through experience; however, “the quality of any
disagreeableness, and there is its influence upon later experiences” (Brookfield, Tennant,
& Pogson, 2005, p. 326). Education and life are interrelated, and educators must design
and carefully monitor positive educational experiences. For Dewey, “knowledge is the
21
product of an interaction between the experiencing subject and the external objective
world” (Brookfield et al., 2005, p. 333). Students learn best by doing, and continuity of
teaching critical thinking and introduced the Watson–Glaser Critical Thinking Appraisal.
[a] an attitude of being disposed to consider in a thoughtful way the problems and
subjects that come within the range of one’s experiences, [b] knowledge of the
methods of logical inquiry and reasoning, and [c] some skill in applying those
methods. (as cited in Scriven & Paul, 1987, p. 2)
the study of child maturation and development (Atherton, 2005). Piaget’s theory is
such diverse topics as language; logical reasoning; moral judgments; and conceptions of
time” (Ormrod, 2004, p. 144). Piaget proposed the idea of scheme and believed that
Piagetian view of cognitive development espouses that there are four cognitive structures
that correspond to the stages of child development, and that “children progress through
different types of thinking as they develop toward mature adult thought” (Brookfield et
al., 2005, p. 27). Piaget’s view emphasized individual thought and autonomy. He
believed “the individual will interpret and act accordingly to conceptual categories or
schemas that are developed in interaction with the environment” (Oxford, 1997, p. 39),
that people are intrinsically motivated and actively involved in the learning process, and
22
that cognitive development results from the interactions that individuals have with their
physical environments.
stressed the importance of past experiences, prior knowledge, society, and culture on
promoting cognitive growth (Dahms et al., 2008). Whereas Piaget was concerned with
interaction, that learning occurs through language and shared experiences, and adults
manner” (Ormrod, 2004, p. 150). Individuals react to, alter, and adapt to their
Proximal Development (ZPD), which assumes that learning is social and human potential
is limitless, “but the practical limits of human potential depend upon quality social
interactions” (Dahms et al., 2008, p. 3). In Vygotsky’s view, cognitive processes begin to
occurs as children discuss and internalize these processes, and “through both informal
conversations and formal schooling, adults convey to children the ways in which their
culture interprets and responds to the world” (Ormrod, 2004, p. 151). Although children
are unable to perform certain tasks independently, their ZPD changes through the
assistance and support of others who are more knowledgeable, and children begin to
reach optimum performance. As students’ abilities develop and certain tasks are
mastered, they begin to acquire the readiness for other more complex skills and to solve
23
peers, the teachers who model the processes in a collaborative classroom environment,
and “through both informal conversations and formal schooling, adults convey to
children the ways in which their culture interprets and responds to the world” (Ormrod,
2004, p. 151).
critical thinking (Chance, 1986; Ennis, 1992; P. A. Facione, 2000; McPeck, 1981; Paul,
1995f; Paul et al., 1997a; Scriven & Paul, 1987; van Gelder, 2005). Chance focused on
argument and defined critical thinking as “the ability to analyze facts, generate and
organize ideas, defend opinions, make comparisons, draw inferences, evaluate arguments,
and solve problems” (p. 6). Chance reasoned that critical thinking involves the “ability to
analyze facts, generate and organize ideas, defend opinions, make comparisons, draw
on deciding what to believe or do” (p. 1). Ennis viewed an ideal critical thinker as one
argument, including the acceptability of its reasons, assumptions, and evidence; can
appropriate for the context; draws conclusions when warranted, but with caution; and
integrates all items in this list when deciding what to believe or do.
24
McPeck (as cited in Hatcher, 2000) defined critical thinking as “the skill and
propensity to engage in an activity with reflective skepticism” (p. 2). Hatcher, on the
other hand, believed reflective skepticism is a “vague notion [that has] negative
connotation[s]” (p. 2) and that critical thinking is constructive and its purpose “is not
always negative, but is often to arrive at a positive judgment on an issue” (p. 2). Hatcher
viewed critical thinking as open-ended and that focuses on areas where “evidence and
arguments are appropriate” (p. 3). He believed critical thinking is “thinking that attempts
habits of mind the individual has toward critical thinking. He maintained the cognitive
and correcting one’s own reasoning are at the heart of critical thinking” (p. 2). Individuals
must possess skill in critical thinking, but they must also have the internal motivation to
think. Critical thinking skills can be developed through practice and guidance as good
critical thinking is a complex, purposeful process. Although humans may possess critical
thinking abilities, they may not always choose to act on reasoned judgment for a variety
argument mapping and analysis can provide useful ways to understand and interpret the
that “once the argument has been mapped and analyzed, the reasoning it manifests can
25
P. A. Facione served as the lead investigator to coordinate an international effort
to determine the extent to which experts agreed on the definition of critical thinking for
the purposes of college-level teaching and assessment. The result was the 1990 APA
Delphi Report. An international group of experts, who included 46 men and women
throughout the United States and represented a variety of scholarly disciplines, was asked
to generate a consensus definition of critical thinking, including its core cognitive skills.
The report included a description of the Delphi research methodology, addressed the skill
assessment. The research project lasted 2 years, was conducted on behalf of the American
and assessment (P. A. Facione, 2006). The panel’s consensus statement regarding critical
thinking and the ideal critical thinker is as follows: “We understand critical thinking to be
Facione, 1990, p. 2). The Delphi panel further concluded that the ideal critical thinker is
26
Scriven and Paul (1987), like P. A. Facione, asserted that individuals may be
subject to undisciplined thought and that critical thinking varies upon the motivation
underlying it. Paul (1995c) was concerned with the metacognitive aspects of critical
thinking and characterized critical thinking as “thinking about your thinking while you’re
thinking in order to make your thinking better” (p. 91). In line with the metacognitive
aspects of critical thinking, Paul et al. (1997a) maintained that critical thinking requires
“the systematic monitoring of thought; that thinking, to be critical, must not be accepted
at face value but must be analyzed and assessed for its clarity, accuracy, relevance, depth,
breadth, and logicalness” (p. 5). Finally, Paul (1995f) defined critical thinking as “the
promoting critical reflection, metacognition, and literary analysis. Their contention was
that few readers have the abilities to skillfully read and translate the author’s intended
meaning in the text, that how students read should be determined by what they read, and
that “skilled readers do not read blindly; they read purposely. They have an agenda, goal,
or objective” (p. 36). Paul and Elder emphasized the importance of students
understanding the purpose of their reading and the author’s purpose in writing, perceiving
ideas in a text as being interconnected, connecting with a text while reading, and
formulating questions and seeking answers to those questions while reading. Further,
27
they encouraged explicating the thesis of a paragraph, analyzing the logic of what is
applying, analyzing, synthesizing, and/or evaluating information” (Scriven & Paul, 1987,
p. 1). People who think critically can analyze their own thinking and realize they can
improve their own reasoning; a highly cultivated critical thinker raises vital questions and
communicates effectively (Elder, 2007; Scriven & Paul, 1987). Carroll (2007) suggested
that “one of the key elements of critical thinking is the recognition that one’s worldview
can be a major hindrance to being fair-minded” (p. 4); hence, students must be willing to
negotiate previously held positions and beliefs while considering opposing viewpoints.
Teaching critical thinking through direct instruction in rhetorical analysis could improve
students’ critical thinking ability, for in order to teach students to think critically, “we
must teach them to try to understand how one’s worldview is likely to be embedded with
Van Gelder (2005) contended that critical thinking is a complex skill that involves
van Gelder proposed that the effective core to critical thinking is in learning how to
handle arguments. Argument maps make reasoning more easily understandable, students
can see the reasoning and can identify important issues, students can follow extended
critical thinking procedures because the arguments are presented in diagrammatic form,
and instructors can see the student’s line of reasoning when it is laid out in diagrams. Van
28
Gelder’s contention was that student critical thinking abilities improve when instruction
is based on argument mapping, and that “one semester of instruction based on argument
mapping can yield reasoning skill gains of the same magnitude as would normally be
discover their own reasoning through a series of questioning. The Socratic questioner
“acts as the logical equivalent of the inner critical voice which the mind develops when it
develops critical thinking abilities” (Paul & Elder, 1997, p. 1). Through a series of
questions, students are forced to think in a reasonable fashion. The Socratic questioner, or
the instructor, attempts to stimulate and encourage discussion while exploring students’
reasoning with probing questions. At the same time, the instructor/questioner should
ensure the discussion remains focused and intellectually responsible and summarize what
has or has not been accomplished (Paul & Elder, 1997). Socratic questioning has the
monitor their thinking, to become more rational, and to pay more attention to that “inner
exploring complex ideas, getting to the truth of some concept, uncovering assumptions,
analyzing concepts, and distinguishing “what we know from what we don’t know”
(Elder, 2006, p. 5). Elder encouraged educators to use Socratic questioning so they can
use it in their everyday lives when confronted with complex issues, “understand the
thinking of others [and] trace the implications of what they and others think and do” (p.
5).
29
Importance of Teaching Critical Thinking
Studies indicate that the United States has not been performing as well as other
industrialized nations, in terms of critical thinking skills (National Center for Education
Statistics, 2007). Students must learn to think skillfully and independently so they may
cultivate their own problem-solving abilities and come to reliable conclusions. The goals
of this study were to provide students with the analytical, problem-solving skills needed
in a variety of academic settings and in their everyday lives. Specifically, the researcher
promoting student critical thinking knowledge, skills, and dispositions, and whether there
important to personal satisfaction and national achievement” (p. 434). Teaching students
to think reflectively and critically should be a primary goal of educational institutions, for
although students may be knowledgeable, they may not have been taught how to think
analytically; hence, they could become highly vulnerable to the fallacious reasoning
exhibited by political leaders and within the media in its various forms (Sternberg, 2003).
The ultimate goal for educators is to promote lifelong learning by enhancing students’
problem-solving abilities so they may “apply these steps not just in school problems, but
Brookfield (2003) referred to the transfer of those skills learned in the classroom
and lifelong learning as “the organizing concept for adult education” (p. 2). Brookfield
claimed “there are forms of learning we engage in that are visible in a much more
heightened form in adulthood as compared to childhood and adolescence” (p. 2). Adult
30
learning is distinct to childhood and adolescent learning in that it includes the capacity to
think dialectically, to employ practical logic, to know how one knows what one knows,
and to think reflectively. Brookfield defined critical reflection as “the process by which
adults become critically reflective regarding the assumptions, beliefs and values which
they have assimilated during childhood and adolescence” (p. 10). This critical reflection
experiences.
critical thought. When discussing the importance of critical thinking, Brookfield et al.
Without the capacity to think and act critically, we would never move beyond
those assumptions we assimilated uncritically in childhood. We would believe
totally in the myths, folk wisdom, and values we encountered in authority figures
in our early lives. We would make no attempt to change social structures or to
press for the collective social action. (p. 46)
Under these circumstances, individuals might believe they are victims of circumstances
that seem beyond their control, unaware that they have the capacity to prompt action
teaching and to help students confront their world or environment with compassion,
understanding, and fairness; nevertheless, the sincerity of intentions “does not guarantee
the purity of practice” (p. 1). When teachers reflect critically on their practice and model
the critical thinking process, students begin to understand what is involved in a critical
analysis of assumptions. For educators, the reflective process involves questioning their
31
practice and discovering assumptions about their practice and their students. This
reflective habit will lead to informed action, add meaning to instruction, and encourage
Paul (1995a) believed critical thinking is the “essential foundation for education
because it is the essential foundation for adaptation to the everyday personal, social, and
professional demands of the 21st Century and thereafter” (p. xi). In view of the rapidly
changing world and the new global realities, there is a vital need for individuals to
develop those skills and abilities that enable them to respond and adapt to these changes.
Although students can perform basic skills pretty well, they are not doing well on
thinking and reasoning. American students can compute, but they cannot reason.
. . . They can write complete and correct sentences, but they cannot prepare
arguments. . . . Moreover, in international comparisons, American students are
falling behind . . . particularly in those areas that require higher order thinking. (as
cited in Paul, 1995g, p. 19)
In order to adapt to the complexities and demands of the 21st century, Paul
(1995b) recommended that educators “cultivate minds that habitually probe the logic of
the systems of the status quo as well as the logic of the possible variations and alternative
systems” (p. xii), and that rather than memorizing the conclusions of others, “students
should reason to those conclusions on the basis of their own disciplined thought” (p. xii).
However, not only are many of the educational institutions ill-prepared for the rapid
changes occurring in society, but educators and students, seemingly, fail to have a clear
idea of what critical thinking is. Students have an obligation to be responsible for their
thoughts, their conduct, and their lives, and educators can attempt to encourage this
32
accountability and improve students’ approach to problem solving through direct
Elder (2000) discussed the emphasis of teaching students the skills needed to
become competent employees at the community college level. As society becomes more
complex, and as a rapid change in technology occurs, “training students for job
performance in narrowly defined skill areas no longer serves students well” (Elder, 2000,
p. 1). Elder’s contention was that students are not prepared for the challenges of the
current job market. Therefore, educators should encourage in their students the
intellectual tools that “will render them mentally flexible and intellectually disciplined”
(Elder, 2000, p. 2). Successful employees must be able to utilize disciplined reasoning
and the metacognitive process so they can direct and redirect their thinking. Rather than
their thinking and to reason, analyze, judge, and interpret that information.
P. A. Facione (2006) discussed the value of critical thinking and the need for an
informed citizenry who can make good judgments while offering practices for nurturing
the habits of mind or dispositions for critical thinking. He suggested the ideal critical
thinker “can be characterized not merely by her or his cognitive skills but also by how
she or he approaches life and living in general” (p. 9). P. A. Facione related a study of
over 1,100 college students that indicated “scores on a college level critical thinking
skills test significantly correlated with college GPA” (pp. 17–18). His contention was that
(p. 18); however, this goal may be somewhat limited. Apart from the college experience,
students must learn to stand on their own, think for themselves, and make their own
33
contributions to society; hence, “learning critical thinking, cultivating the critical spirit, is
not just a means to this end, it is part of the goal itself” (P. A. Facione, 2006, p. 18).
To date, there are a number of researchers who have examined the effect of direct
instruction in critical thinking. This study, however, examined this issue by determining
whether there was improvement in students’ critical thinking knowledge, skills, and
dispositions on pretest and posttest scores after direct instruction in argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning. Of added importance was the
instructional interventions and of their own growth in critical thinking abilities at the end
of the course.
An assumption among many educators is that students who attend college will
develop the necessary critical thinking skills simply by attendance in class and
universities have been offering courses designed to improve students’ critical thinking
improving students’ critical thinking abilities has become a primary goal for education,
there is continual debate over the most appropriate or effective pedagogical strategies
identified seven components of intelligences that he believed are distinct from each other.
34
Although Gardner valued individuals who can who can think critically about literature
and world events, he cautioned that “one must be careful not to assume that it is a
particular, dissociable variety of human cognition” (p. 44), while suggesting that
“particular domains of human competence seem to require their own brand of critical
thinking” (p. 44). The kind of critical thinking required for a musician might be quite
different from the critical thinking required for a historian or biologist because “each
domain exhibits its own particular logic of implications” (Gardner, 1993, p. 14); hence,
training for one domain does not necessarily provide transfer to other domains. Further,
instructors should not assume that critical thinking skills taught in a standalone critical
thinking course will transfer to those skills needed for a history course; rather, Gardner
believed that
Only if the lessons of critical thinking are deliberately revisited in each of the
relevant classes or exercises is there any possibility that a more general virtue like
reflectiveness or taking the perspective of the other has any chance of emerging.
(p. 44)
intelligence and intelligence tests do not necessarily reflect talent or wisdom. Similar to
intelligence refers to “internal abilities [or] mental mechanisms” (Sternberg, 2008, p. 24)
35
environmental selection, and environmental shaping” (Sternberg, 2008, p. 24). His theory
takes into account the strengths and differences of individuals while considering the
sociocultural context in which they live. Sternberg believed intelligence can be increased
through study and practice, and encouraged educators to assist students so they can use
and develop all of their skills and perform well in all areas.
Sternberg (2003) defined the ideal critical thinker as a good problem solver;
however, students must be taught to transfer the problem-solving skills they learn in
school to their everyday lives. Successful intelligent thinkers have “the creative skills to
generate new ideas, analytical abilities to know whether they are good ideas, and the
practical abilities to know how to implement the ideas and to persuade others of the value
of their ideas” (Sternberg, 2003, p. 4). Yet, although Sternberg believed these to be
valuable skills, he advocated that schools should include wisdom-related skills in the
school curriculum.
toward the attainment of a common good” (p. 7). The dilemma exists when students use
extrapersonal interests so their knowledge is used for good purposes. Sternberg’s research
experimental group learn to understand things from different points of view, whereas
students in the control group learn the historical material in a standard way. The
definitive goal of the study was to teach students that “resolution of difficult life
36
problems requires people to want to understand each other and to reach a resolution,
whenever possible, with which all of those people can live” (Sternberg, 2003, p. 7).
taught within a variety of contexts and for transfer so that students can apply the
knowledge and skills into other domains. In agreement with Paul (2004) and P. A.
Facione (2006), Halpern affirmed that critical thinking instruction must address student
dispositions, while acknowledging “it is not enough to teach college students the skills of
critical thinking if they are not inclined to use them” (p. 72). Further, students should not
only have a disposition toward critical thinking, they should also possess the willingness
to apply it. It is, therefore, crucial that students be taught the value of critical, reflective
apply the knowledge and skills learned in one context to other situations. Students should
the educational environment to their personal and professional lives. Perkins (1993)
offered a variety of strategies to accomplish this complex task. He viewed the teacher as a
guide and mentor and suggested the three basic tools that can elicit thoughtful learning
37
performances through practice, self-assessment, and informative feedback. Perkins
asserted that “knowledge and skill in themselves do not guarantee understanding” (p. 2),
connections between their lives and the subject matter they are being taught. Perkins
thought-demanding ways with the topic, for instance to: explain, muster evidence, find
examples, generalize, apply concepts, analogize, represent in a new way” (p. 4). Perkins
outlined six priorities for educators who teach for understanding: making learning a long-
Specific Strategies
Argument Mapping
Van Gelder (2005) drew on lessons from cognitive science while discussing the
importance of developing students’ critical thinking abilities, and, as with Paul (1995c),
emphasized the need to improve teaching and educational institutions. He indicated that
individuals, as a rule, are not naturally critical and that critical thinking is a difficult, yet
not impossible, skill that takes time and deliberate practice to master. Students do not
become better thinkers merely through exposure to good critical thinking; they must
engage in critical thinking itself. He further contended that knowledge of the critical
thinking theory “is the basis for self-monitoring and correction” (p. 4) and that this
38
Van Gelder (2005) believed a central part of improving this skill is in handling
arguments, that “arguments are presented or expressed in streams of words” (p. 4), and
that the structure of the arguments is expressed in “sequences of words or sentences” (p.
4). Arguments are, generally, linear and can be mapped out so that students can view the
logical structure of the rhetoric. Van Gelder claimed that “critical thinking skills improve
faster when instruction is based on argument mapping” (p. 5) and that “students in classes
based heavily on argument mapping consistently improve their skills much faster than
students in conventional classes” (p. 5). The advantages of argument maps are that they
make reasoning more easily understandable; students can see the reasoning, so they can
more easily identify important issues; arguments presented in diagrammatic form allow
students to follow the extended critical thinking procedures; and arguments laid out in
diagrams allow the teacher to see what the student is thinking (van Gelder, 2005).
Argument maps offer a straightforward, effective tool for improving students’ critical
thinking abilities.
Paul (2004) stated studies of higher education demonstrate that most college
faculty lack a substantive concept of critical thinking, most college faculty do not realize
they lack this substantive concept, and that lecture and memorization are still the norm in
college instruction. Paul’s (2004) model of critical thinking was created in an attempt to
provide a concept of critical thinking and a common model for instructional design. His
purpose for designing this model was to encourage educators to place emphasis on higher
order learning as “it is precisely these higher order thinking skills that are routinely
39
sacrificed when coverage and lower order recall dominate the classroom” (Paul, 1995f, p.
105).
thinking, while emphasizing that students’ critical thinking abilities cannot be refined
without reference to the “values, traits of mind, and dispositions that underlie those
skills” (p. 8). Assessment must be specific and consistent, an accurate concept of critical
thinking must be fostered, and appropriate assessment strategies must align with a true
concept of critical thinking so it is incorporated into testing and teaching. A real danger
intellectual standards. Elements of thought refer to the kind of thinking that is conceptual
and inferential, and they are the basic building blocks of thinking; abilities include
thinking about complex issues; affective dimensions are the essential attitudes,
dispositions, and traits of mind needed for critical thinking; assessment of affective
dimensions is an important part of the assessment of higher order thinking and refers to
the validity and reliability in terms of the elements of thought; and intellectual standards
Paul and Elder took the essential elements of good critical thinkers and developed
The Thinker’s Guide to the Art of Socratic Questioning: Based on Critical Thinking
Concepts & Tools (Paul & Elder, 2006b) and The Thinker’s Guide to Analytic Thinking:
How to Take Thinking Apart and What to Look for When You Do (Elder & Paul, 2007).
40
The researcher used both of these Thinker’s Guides in the study. The Thinker’s Guides
address the following: critical thinking concepts and tools, how to study and learn a
discipline, the art of asking essential questions, active and cooperative learning, how to
improve student learning, how to write a paragraph, fallacies: the art of mental trickery
and manipulation, the human mind, critical and creative thinking, analytic thinking,
scientific thinking, how to read a paragraph, and ethical reasoning (Bessick, 2008).
Socratic Questioning
Paul and Elder (2000) asserted that educators concentrate far too much on
“coverage” (p. 1) of course material over “engaged thinking” (p. 1) partly because they
believe “answers can be taught separate from questions” (p. 1). Conversely, Paul and
Elder (2000) argued that questioning encourages thinking and learning because all
assertions and statements are, ultimately, answers to questions. They reasoned that
problems and delineate issues” (p. 1). Paul and Elder (1997) referred to Socratic
powerful, teaching tactic for fostering critical thinking” (p. 1). Along with allowing
educators a more focused approach to their students and to their instruction, Paul (1995e)
acknowledged that “Socratic questioning is at the heart of critical teaching” (p. 335). as it
presents opportunities for students to develop and evaluate their ideas. Socratic
questioning
Raises basic issues; probes beneath the surface of things; pursues problematic
areas of thought; helps students discover the structure of their own thought; helps
students develop sensitivity to clarity, accuracy, and relevance; helps students
arrive at judgment through their own reasoning; and helps students note claims,
evidence, conclusions, questions-at-issue, assumptions, implications,
41
consequences, concepts, interpretations, points of view—the elements of thought.
(Paul, 1995e, p. 336)
the social and physical world” (p. 289) and that educators, through higher cognitive
manner. While suggesting educators are in need of concrete strategies for improving
critical thinking, Newton (1978) contended that educators should stimulate rather than
indoctrinate learners. In line with Piaget and Dewey, she perceived the learner as an
active participant in the learning process and presented a theoretical basis for higher
knowledge in the light of the grounds to justify it” (p. 287), while encouraging educators
to teach “that knowledge and those skills through discussion of controversial social
the extent to which they prepare candidates to teach critical thinking and problem-solving
skills. Using both quantitative and qualitative data, the researchers’ objectives were to
assess current teaching practices and knowledge of critical thinking among faculty,
identify representative teaching practices that improve critical thinking, and develop
policy recommendations based on the results of the study. The results of their research
42
were based on a study of randomly selected California professors from 38 public and 28
private colleges and universities. In-depth interviews were utilized to determine the
extent to which students in teacher preparation programs were being taught to encourage
and teach critical thinking instruction. The researchers also sought to determine the
faculty’s conception of critical thinking. Findings of the research study indicated that
although the majority of the faculty claimed critical thinking is a primary objective of
instruction, only a small minority could clearly explain what critical thinking is. Paul et
al. maintained that although teacher educators claim to be committed to teaching critical
thinking, “few have had any in-depth exposure to research on the concept and most have
into instruction” (p. 10). Paul et al. claimed educators must have a clear concept of what
critical thinking is and must be taught “ways that facilitate skill in critical thinking and
the ability to teach it to others” (p. 2) so that students will graduate and enter the work
force with the problem-solving abilities needed for clarity of thought and intellectual
discipline.
thinking for New Mexico State University (NMSU) College of Business Administration
thinking and approaches to teaching critical thinking. Meinecke used surveys, interviews,
The study revealed that “business college faculty did not have a conceptual definition of
critical thinking and often failed to teach critical thinking in their classrooms” (Meinecke,
1997, Abstract). Meinecke’s conceptual definition of critical thinking was “an ability to
43
engage in a process of objectively analyzing a problem with an open mind and
intellectual honesty” (Abstract). She believed critical thinking is a recursive process that
includes reason assessment, a critical spirit, and metacognition, and maintained that
critical thinking can be developed through a reflective pedagogical paradigm and that
students should be “placed at the center of the classroom environment with autonomous
taught with a common language that includes their unified conceptual definition of
critical thinking” (p. 212). Further, she encouraged educators at NMSU to create
classroom environments that promote critical and creative thinking through the
application of the six dimensions of thinking, as developed by Tishman, Perkins, and Jay,
instructional pedagogy that will promote students’ critical thinking skills. Several studies
have focused on direct instruction of critical thinking through the use of argument
mapping, Paul’s Thinker’s Guides, and Socratic questioning (Bessick, 2008; Reed, 1998;
Scanlan, 2006; Tsui, 1998; Van Erp, 2008; Yang, 2008), and their findings suggested that
although these instructional strategies prove useful for promoting critical thinking, further
educational institutions in improving higher order thinking skills. Her contention was that
44
students entering colleges and universities are deficient in reading, writing, mathematics,
and critical thinking skills; however, “because of the lack of preparedness in the basics,
schools have become so focused on general academics that efforts to enhance the
development of critical thinking skills in their students is minimal” (p. 33). Bessick
believed direct instruction in critical thinking and reasoning can improve student learning
outcomes; however, the ultimate educational goal should be to enable students to transfer
those skills learned in the classroom to their everyday lives. In her study, Bessick
examined the importance of teaching critical thinking skills and their relationship to
freshmen students being tutored in repeat courses in which she examined the effect of
direct instruction in critical thinking on the students’ critical thinking ability and
academic achievement using van Gelder’s rational argument mapping and Paul’s
Thinker’s Guides. Data analysis revealed the intervention programs using the Thinker’s
Guides based on Paul’s model of critical thinking and van Gelder’s rational argument
skills. However, the Thinker’s Guides group and the control group demonstrated
improve the critical thinking skills and that “instruction in critical thinking, whether
the improvement of students’ academic achievement” (p. 153). Her findings suggested
that further research is needed using a larger sample size to determine the extent to which
direct instruction using the Thinker’s Guides and the rational argument mapping program
45
Scanlan’s (2006) study was conducted in order to explore strategies to improve
reasoning into all standards-based curriculum. The project researched the effect of
focused critical thinking training on the composition skills of 12th graders, specifically
and assessment was measured through a series of rubrics. The results of his research
indicated the value of incorporating Paul’s elements and standards of reasoning is in the
gains students made in terms of clarity of writing, level of analysis, use of supporting
she investigated the effect of integrating Paul’s model of critical thinking into a U.S.
history course on community college students’ abilities to think critically about U.S.
history and about everyday issues, dispositions toward thinking critically, and knowledge
of history content. Reed found that students’ critical thinking abilities improved in a
single course, the students’ end-of-term knowledge of history content did not suffer when
training in critical thinking abilities was integrated into course material, and age and
gender did not play significant roles in developing college students’ critical thinking
abilities. Reed’s study revealed that students’ abilities to think critically can improve in a
single course when provided with deliberate instruction and that Paul’s model “can be
benefits to students’ abilities to think critically within a domain and to their general
46
Yang (2008) conducted a study designed to examine the effectiveness of teaching
critical thinking skills “through asynchronous discussion forums (ADFs) with the
facilitation of teaching assistants, to investigate student interaction patterns and the depth
of their critical thinking demonstrated via the asynchronous discussion forum” (p. 241).
The major goal was to investigate whether students’ critical thinking skills would
improve after they participated in Socratic dialogues as taught by the instructor and then
as modeled and facilitated by the teaching assistants. The main task of the teaching
assistants was to focus on the quality of the student interaction, including the
assumptions, reasoning, and evidence provided by the students. The qualitative analysis
provided a detailed description of how students’ discussions moved from the lower to the
higher phases of critical thinking. Results from the CCTST indicated that instructors who
use Socratic dialogues during small-group online discussions can successfully improve
students’ critical thinking skills in a large university class. This improvement in learning
is accomplished as “students interact with their peers or instructor and benefit from
combining their levels of expertise, offering support, distributing the thinking load, and
confronting alternative points of view” (Yang, 2008, p. 261). The results also indicated
that “learning is also accomplished as students elaborate, clarify, and justify their
the instructor and a group of learners in three online graduate courses in which critical
thinking skills were emphasized alongside course content. Students completed a pre- and
courses included strategies that promote critical thinking skills, including asynchronous
47
discussion, instructor modeling and coaching, explicit instruction about critical thinking,
self-assessment writing, Socratic questioning, and scoring rubrics that included critical
thinking criteria. Van Erp used P. A. Facione’s definition in the study while investigating
the correlation between the definitions of critical thinking found in literature and the
themes identified in learners’ definitions; the terms “‘reflective’ and ‘reflection’ were
themes in both the pre and the post course self-assessment reflections” (p. 97). Van Erp
believed the strongest themes that emerged in the precourse self-assessment were open-
interview process, the strongest theme to emerge from the student interviews was
instructor feedback. Learners’ critical thinking ability was enhanced when they were
Further, the conversations and the systematic, carefully crafted questions designed by the
instructor challenged the learners’ assumptions and improved student learning and the
entire program while supporting van Gelder’s (2005) premise that critical thinking is a
difficult skill to master and that instructors should scaffold learning and be systematic
and intentional in their approach through carefully designed discussion questions. Quality
questions that scaffold learning can impact learning because they challenge learners to
Tsui (1998) suggested that although developing student critical thinking skills is a
primary goal of education, educators lack sufficient knowledge as to how this educational
aim can be achieved. Tsui combined quantitative and qualitative data to investigate the
impact of college on the development of student critical thinking skills and to examine
48
factors that lie inside as well as outside of the classroom. The sample included 13,216
students from more than 200 four-year colleges and universities. Quantitative data were
Tsui (1998) found that critical thinking abilities can be developed positively in
inquiry, emphasis on analysis over recall, and the constructivist philosophy to learning;
negative factors for promoting critical thinking include lectures and multiple-choice
examinations. Further, Tsui discovered that infusion of critical thinking into the
curriculum is more successful when “faculty have enough confidence in their students to
challenge them [and are] enthused enough to partake in collegial exchanges about
peers.
Conclusion
educators. There are a variety of definitions for critical thinking; however, the researcher
49
Paul (1995b) viewed critical thinking as the essential foundation for education as
individuals must have the ability to adapt to the demands of the 21st century. Frequently,
instructional practices emphasize the content rather than the quality of instruction.
Although students can perform basic skills, pedagogical practices that encourage higher
order thinking skills could enhance student critical thinking. Another key obstacle to
promoting critical thinking is that teachers and students may not have a clear idea of what
thinking should be taught within a variety of contexts, and teachers should encourage
students to transfer the knowledge and skills learned in the classroom into their everyday
lives. What is essential is that educators develop the appropriate skills and strategies
While there is continual debate regarding the best methods for improving
students’ critical thinking abilities, critical thinking, although a complex skill, may well
using a combination of van Gelder’s argument mapping and Paul’s Thinker’s Guides.
Scanlan (2006) and Reed (1998) discovered that student learning and critical thinking can
be enhanced by using Paul’s model of critical thinking. Van Erp (2008) and Yang (2008)
concluded that Socratic dialogues can improve students’ critical thinking abilities;
however, Van Erp and Yang stressed the importance of timely feedback, coaching, and
frequent modeling. Finally, Tsui (1998) found that critical thinking abilities can be
50
developed positively in association with substantive writing, critical discussion, class
primary goal of educational institutions so students acquire the abilities to transfer those
skills into their everyday lives. Paul (1995b) avowed that critical thinking should be the
essential foundation for education, while arguing that educational institutions are ill-
prepared for the rapid changes occurring in society, but educators and students do not
have a clear idea of what critical thinking is. Elder (2000) commented on students being
ill-prepared for the current job market, while suggesting that educators encourage
thinking and the need for an informed citizenry. Direct instruction in critical thinking
using a variety of instructional strategies could promote those critical thinking skills and
51
CHAPTER 3. METHODOLOGY
Introduction
opportunities that afford students the knowledge and skills necessary for success in life is
those pedagogical practices that could effectively develop higher order thinking skills.
could positively impact the range and quality of student critical thinking skills’
critical thinking course would improve students’ critical thinking abilities. This chapter
provides a description of the methodology used in the study and includes a statement of
the problem, the research questions, the research design and methodology, a discussion of
the population and sample, and a detailed description of the instrumentation, data
Statement of Problem
critical thinking knowledge, skills, and dispositions. This study was an exploration and
52
evaluation of instructional pedagogy designed to promote students’ critical thinking
college critical thinking course could make a difference in students’ critical thinking
Thinker’s Guides, based on Paul’s model of critical thinking; and Socratic questioning.
These interventions were supplemented with course work that involved reader/writer
Research Questions
1. What were the students’ perceptions of critical thinking before and after direct
instruction in argument mapping?
2. What were students’ perceptions of critical thinking before and after direct
instruction in Paul’s Thinker’s Guides?
3. What were the students’ perceptions of critical thinking before and after direct
instruction in Socratic questioning?
4. What did the students report about the changes in their knowledge, skills, and
dispositions relating to critical thinking after the direct instruction in rhetorical
analysis?
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
53
Research Design and Methodology
methods and “gives the researcher greater understanding of the issues under
investigation” (Cowden, Chivore, Maravanyika, Nyagura, & Sibanda, 1999, p. 11). This
dispositions. The independent variable for this study was the critical thinking
interventions. The dependent variables were the posttest scores on the CCTST–2000 and
on the Pre- and Post- Critical Thinking Surveys. The researcher examined the
relationship between the variables involving teacher instruction of the interventions, the
students’ mastery of those skills, and the students’ perceptions of the effectiveness of the
interventions. This blend of methodologies enhanced the relevance and strength of the
the validity of data. The quantitative approach is formal and conclusive and involves
“collecting numerical data on observable behaviors of samples and then subjecting these
The researcher was the teacher of an intact class, and no attempt at randomization
occurred; therefore, the study was limited to a description of an existent situation rather
than creating an experimental situation. In teacher action research, the data are collected
and analyzed to assess and improve educational practice. Since the researcher was the
instructor of the critical thinking course, there was potential for researcher bias as the
researcher may have had authority over the participants. The researcher understood this
54
and attempted to approach the study with caution and without preconceived assumptions
of possible outcomes or results. Care was taken to maintain clarity and objectivity
through sharing with research participants and through personal reflection, assessment,
the course to measure the dependent variable; implemented the experimental treatment,
or the independent variable; and administered a Post- Critical Thinking Survey at the end
of the course. The qualitative survey was used to support description and analysis by
conceptualize (Cowden et al., 1999). Further, the qualitative data aided in looking for
patterns, and the observations made by the survey results offered insights into student
administered as a pretest at the beginning of the course and measured the students’
critical thinking skills. The same test was given as a posttest and measured growth in
critical thinking abilities at the end of the course. Statistical analysis was administered on
The population of this study was a convenience sample of students who attended
serves a geographic area that is semirural in nature. The college offers two-year academic
community college reflect the diversity of the surrounding community. The demographic
55
makeup of the college reveals that 66.5% of the student population is White/non-
Hispanic, 8.7% Black, 20% Hispanic, 3.7% Asian, and 1.1% Native American. The total
however, it was not convenient for the researcher to collect data from this entire
population. Hence, the population was limited to those students enrolled in a critical
Convenience sampling suited the purposes of this study as it was the most
practical approach for the researcher. The sample consisted of students enrolled in an on-
campus critical thinking course taught by the researcher. Total enrollment in the critical
thinking course was 29 students at the beginning of the semester; 19 of those students
agreed to participate in the study; however, three students dropped from the course and
one student did not show up the final day for the posttest; thus, the final research sample
consisted of 15 students.
The critical thinking course was designed to develop the students’ critical
thinking, reading, and writing skills. Basic English is a prerequisite to this course. The
critical thinking course is not a required course; however, it is required for students who
are English majors. The critical thinking course focused primarily on the analysis and
argumentative essays. The objectives of the course and the purpose of studying
composition was to help students improve their writing, reading, and critical thinking
skills to a level that will enable them to transfer those skills to their professional and
personal lives. All of the students in the sample received the interventions that were
56
provided throughout the semester. Gains in student critical thinking abilities were
Instrumentation
The CCTST–2000 was ordered from Insight Assessment and was administered by
the researcher as a pretest and posttest. There are three versions of the CCTST: Form A,
Form B, and Form 2000. The CCTST–2000 is based on the APA Delphi consensus
teachers, and critical thinking assessment specialists, and was reaffirmed in the national
survey conducted by the National Center for Higher Education Teaching, Learning and
critical thinking has been endorsed by educators and scholars around the world (N. C.
Facione, Facione, Blohm, & Gittens, 2008). The CCTST–2000 is specifically designed to
measure the skills dimension of critical thinking. The items on the CCTST–2000
emphasize a variety of critical thinking skills, which include analysis of the meaning of a
given sentence, drawing a correct inference from a set of assumptions, and evaluating or
gather valid and reliable data about the baseline, entrance-level, or exit-level critical
thinking skills of various groups of people, commonly college level students and working
adults” (p. 11). The items are given in standard English and are set in contexts and
address topics that are familiar to college-age students. The CCTST–2000 is appropriate
57
for college undergraduate and graduate students and for the researcher’s study (N. C.
minutes to complete under timed conditions and can also be administered untimed. The
Reliability
Items in the CCTST–2000 include the critical thinking skills identified by the
Delphi experts: interpretation, analysis, evaluation, explanation, and inference. They are
discipline-neutral; sex-role and social class stereotypic context have been avoided; and
“equal numbers of male and female referents are used to decrease possible gender and
cultural bias” (N. C. Facione et al., 2008, p. 27). Validation studies “have produced
consistency estimates . . . ranging from .68 to .80” (N. C. Facione et al., 2008, p. 27).
researchers (Bondy, Koenigseder, Ishee, & Williams, 2001; N. C. Facione & Facione,
1997; P. A. Facione, Facione, Blohm, Howard, & Giancarlo, 1998; Jacobs, 1995, 1999).
According to Pike (as cited in Laird, 2005), the CCTST–2000 is considered to be more
reliable than Forms A and B. Laird suggested the instrument is suitable for educational
58
assessment; however, “test administrators should pay particular attention to the reliability
and inter-correlation of the total score and subscales within the group under study” (p. 4).
Validity
construct validity, and criterion validity. Content validity refers to how well the test items
represent the critical thinking domain. CCTST–2000 test items were chosen for their
Facione et al., 2008). Construct validity refers to the extent the CCTST–2000 measures
conceptualization. As more students enroll in critical thinking courses, and as their test
in the CCTST–2000, can validly and reliably measure this improvement in higher order
thinking skills. Further, face validity is supported by the types of questions asked on the
CCTST–2000 that encourage students to make judgments, identify correct analyses, draw
inferences, evaluate reasoning, and justify inferences and evaluations (N. C. Facione et
al., 2008). Criterion validity refers to a test’s ability to predict external criteria, such as a
higher level of college success. The CCTST–2000 is reported to have a content validity
of .74 and moderate criterion validity with grade point average and SAT math and verbal
The second and third instruments that were used in the study, the Pre- and Post-
Critical Thinking Surveys, were qualitative in nature. The second instrument was
comprised of two sections: a student demographic survey and the Pre- Critical Thinking
Survey. The Post- Critical Thinking Survey included three sections: the student
59
demographic survey, the Critical Thinking Survey, and a Student Perceptions of Critical
Thinking Survey. The student demographic portion of the survey included questions
concerning gender, race, age, current year in college, academic major, student status,
students’ perceptions of their writing and comprehension skills, and previous academic
experience in critical thinking. The Pre- and Post- Critical Thinking portion of the survey
offered useful information regarding the students’ critical thinking knowledge, skills, and
Field Test
To determine whether the surveys had merit and to correct any flaws, the Pre- and
Post- Critical Thinking Surveys were administered to four teachers at the community
college level for review before conducting the research study. The participants of this
field test were encouraged to make criticisms and recommendations for improving the
surveys. Following the review, respondents indicated the surveys were appropriate,
Data Collection
On July 13, 2009, a research permission letter was sent to the dean of humanities
course during the Fall semester, along with a detailed explanation of the purpose of the
research. During the first week of the course, students and the researcher read and
60
discussed the critical thinking course syllabus, a detailed description of the research study
was given, and a request was made for those students who wished to participate in the
study. Participation in the research by students was strictly voluntary. Data collection
began and an informed consent letter was given to those who were interested in
participating in the study. Students were assured that participation in the study was
grade, and that their personal identity would not be released in the dissertation. At the end
of the first week of the semester, the informed consent letters were collected from those
collection of data to determine student views of their own critical thinking dispositions
and abilities was gathered during the first week of the course. Students completed the
and skills. During the second week of the course, the CCTST–2000 was administered as a
pretest.
was introduced during the first 4 weeks; this was followed by instruction in Paul’s
Thinker’s Guides; during the final 4 weeks, instructional content included Socratic
questioning.
61
During the final week of the semester, the CCTST–2000 was again administered
documents that contain directions and the 34 multiple-choice questions) and scannable
answer forms. Before beginning the test, the students were given an identification
number, and a master list was kept of the test takers’ names matched with their personal
identification numbers (PINs). This list was kept in a secure place to protect the privacy
Upon completion of the CCTST–2000 pretest and posttest, exams were shipped
back to Insight Assessment, where they were scanned and scored using the CapScore
system. Insight Assessment then sent the research data file and a report summarizing the
survey results. Reports were generated that included descriptive statistics for the group as
a whole as well as subscale statistics. The CCTST–2000 is normed for 4-year college
employees. These norm groups were used in the analysis of the test-taker data, and the
comparison with norm sample information was included in the Basic CapScore (N. C.
the final week of the semester that consisted of the following sections: the Student
skills, and dispositions; and questions concerning the extent to which instruction in
argument mapping, Paul’s Thinker’s Guides, and Socratic questioning increased their
62
life. The data from the Pre- and Post- Critical Thinking Surveys (Appendixes A & B) was
Data Analysis
questioning, and the acquisition of critical thinking skills. Research Question 4 focused
on the students’ perceptions of the changes in their knowledge, skill, dispositions, and
attitudes after direct instruction in critical thinking. These questions were addressed in the
Likert-scaled Student Perceptions of Critical Thinking and were analyzed using tables in
a Microsoft Word document and the highlighting tool to identify significant themes and
abilities. Additional charts and graphs were used to investigate patterns and insights. This
Research Question 5 focused on the extent to which the pretest and posttest scores
indicate changes in the students’ knowledge, skills, and dispositions relating to critical
thinking after direct instruction in argument mapping, Paul’s Thinker’s Guides, and
Socratic questioning. For both the pretest and posttest, the CCTST–2000 was the sole
critical thinking skills and five subscales. The five subscales are Analysis, Evaluation,
Evaluation “draw together the major core skills identified in the theory of critical
63
thinking advanced in The APA Delphi Report. Inductive Reasoning vs. Deductive
Reasoning is represented in the fourth and fifth CCTST–2000 sub-scale” (N. C. Facione
et al., 2008, p. 12). The total score on the CCTST–2000 provides the overall measure of
information to be coded onto the response form. For the pretest, each student indicated
001 on his or her response form. For the posttest, each student indicated 002 on his or her
response form. CapScore provided a total and scale CCTST–2000 scores for each test
taker by PIN and descriptive statistics for both Groups 1 and 2 separately. The CapScore
answer forms were returned along with a PC-formatted disk with an Excel and a tab-
delimited text file, including total and subscale CCTST–2000 scores for each test taker by
PIN (N. C. Facione et al., 2008). Changes in students’ critical thinking scores were
Ethical Considerations
research study. A written description of the research design was submitted to the
must be informed about what will occur during the research study. Consent was obtained
from the students and the administrative faculty. Participants received an explanation of
the test and course room procedures to be used and a letter describing the research and
64
conditions of their participation (Gall et al., 2003). Meetings with the administration
conduct and practice and the participants provisions of the Code of Federal Regulations
for the Protection of Human Subjects” (Gall et al., 2003, p. 66). Student confidentiality
was respected, and participants were told who would have access to data and that their
privacy would be protected. Random codes were given to participants in the study to
A limitation to this study was the small sample size, which limits the conclusions
that can be generalized to the target population. A random sample from the accessible
population in the critical thinking course was used in the study. The researcher gathered
data from the Pre- Critical Thinking Survey (Appendix A) to determine the degree of
similarity between the accessible sample population with regard to age, gender
distribution, and ethnic distribution in the critical thinking course to the target population.
In view of the fact that the researcher adopted a teacher action research approach
in which the researcher was the instructor, there was the potential for researcher bias as
the researcher could have real or perceived authority over the participants; however, the
65
Thinking Surveys as pre- and posttests. Since the Student Perceptions of Critical
of the findings. These findings were checked by the results generated by the CCTST–
2000, which is a standardized test and in which there is no room for bias. This
perspectives and data sources and helped to “eliminate biases that might result from
relying exclusively on any one data-collection method” (Gall et al., 2003, p. 464).
Suitable controls over extraneous variables that could have threatened the validity
data. Experimental designs often provide adequate controls for sources of internal
validity, but the results have restricted external validity. Another possible limitation of
the study was that the researcher had a certain amount of control over what occurred in
the classroom and very little control over what occurred in the students’ lives outside of
the classroom.
Other possible limitations to the research study included (a) the length of the
study as the development of critical thinking skills requires practice over an extended
period of time and (b) the response rate of the participants involved in the final data
analysis.
Summary
critical thinking course can improve students’ critical thinking skills. Argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning were used as treatment interventions in
66
increments for 12 weeks during the semester. Data were collected from the pretest at the
beginning of the course and from the posttest during the final week of the course using
the results from the CCTST–2000. Students were also required to complete Pre- and
they believed to be the most beneficial and applicable method for enhancing their critical
thinking skills. Final scores on the CCTST–2000 posttest and the Student Perceptions of
thinking.
67
CHAPTER 4. DATA COLLECTION AND ANALYSIS
Introduction
The purpose of this study was to determine whether there would be improvement
in students’ critical thinking abilities, including knowledge, skills, and dispositions, after
argument mapping; Thinker’s Guides, based on Paul’s model of critical thinking; and
Socratic questioning. The goal of the study was, on the one hand, to promote critical
thinking knowledge, skills, and dispositions and, on the other hand, to determine
dispositions so they may transfer those skills across the academic disciplines and into
demographics. This is followed by an overview of the data and the results as they relate
68
Statement of the Problem
learners’ critical thinking knowledge, skills, and dispositions. This study explored
course could make a difference in students’ critical thinking skills. The interventions
thinking; and Socratic questioning. These interventions were supplemented with course
Research Questions
1. What were the students’ perceptions of critical thinking before and after direct
instruction in argument mapping?
2. What were the students’ perceptions of critical thinking before and after direct
instruction in Paul’s Thinker’s Guides?
3. What were the students’ perceptions of critical thinking before and after direct
instruction in Socratic questioning?
4. What did the students report about the changes in their knowledge, skills, and
dispositions relating to critical thinking after the direct instruction in rhetorical
analysis?
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
69
Overview of Data Collection
The mixed-methodology approach was used in this action research study. The
study pertained to one semester of a college critical thinking course that was taught in a
naturalistic setting by the researcher. The overall goal of the study was to examine the
skills, and dispositions. Twenty-nine students were initially enrolled in the critical
thinking course. Of the 29 students, 19 agreed to participate in the study. Three of those
students dropped the course and one student was not present the final day of the course
and did not take the posttest; thus, the final research sample was comprised of 15 (N =
skills, and dispositions. The independent variable for this study was the critical thinking
interventions. The dependent variables were the posttest scores on the CCTST–2000 and
on the Pre- and Post- Critical Thinking Surveys. The researcher examined the
relationship between the variables involving teacher instruction of the interventions, the
students’ mastery of those skills, and the students’ perceptions of the effectiveness of the
interventions.
70
questioning, and the acquisition of critical thinking skills. Research Question 4 focused
on the students’ perceptions of the changes in their knowledge, skill, dispositions, and
attitudes after direct instruction in critical thinking. These questions were addressed in the
Likert-scaled Pre- and Post- Critical Thinking Surveys and were analyzed using tables in
Research Question 5, the null hypothesis, focused on the extent to which the
pretest and posttest scores indicate changes in the students’ knowledge, skills, and
Paul’s Thinker’s Guides, and Socratic questioning. For both the pretest and posttest, the
CCTST–2000 was the sole quantitative instrument used. The CCTST–2000 test involves
an overall score on one’s critical thinking skills and five subscales. The five subscales are
Analysis, Inference, and Evaluation “draw together the major core skills identified in the
theory of critical thinking advanced in The APA Delphi Report. Inductive Reasoning vs.
Deductive Reasoning is represented in the fourth and fifth CCTST–2000 sub-scale” (N.
C. Facione et al., 2008, p. 12). The total score on the CCTST–2000 provided the overall
information to be coded onto the response form. For the pretest, each student indicated
001 on his or her response form. For the posttest, each student indicated 002 on his or her
response form. CapScore provided a total and scale CCTST–2000 scores for each test
71
taker by PIN and descriptive statistics for both Groups 1 and 2 separately. The CapScore
answer forms were returned along with a PC-formatted disk with an Excel and a tab-
delimited text file, including total and subscale CCTST–2000 scores for each test taker by
PIN (N. C. Facione et al., 2008). Changes in students’ critical thinking scores were
The second and third instruments used in the study, the Pre- and Post- Critical
Thinking Surveys (Appendixes A & B) were qualitative in nature. The Post- Critical
Thinking Survey included three sections: the student demographic survey, the Critical
Thinking Survey, and a Student Perceptions of Critical Thinking Survey. The Student
Perceptions of Critical Thinking portion of the Post- Critical Thinking Survey provided
Demographic Characteristics
The characteristics of the sample were based on the data obtained from the
student demographic portion of the Pre- Critical Thinking Survey. This portion of the
survey included questions concerning gender, race, age, current year in college, academic
major, student status, students’ perceptions of their writing and comprehension skills, and
72
Table 1. Sample Demographic Characteristics
Category n
Gender
Male 8
Female 7
Race
Black 1
Indian/Alaskan 1
Asian/Pacific Islander 0
Hispanic 2
White/non-Hispanic 11
Age
17–19 10
20–22 3
23–25 1
26–30 1
Over 30 0
Year in college
Freshman 2
Sophomore 13
Junior 0
Senior 0
Discipline major
Humanities 3
Sciences 4
Mathematics 2
Education 3
Social Sciences 3
Student status
Full-time, degree-seeking 13
Part-time, degree-seeking 2
Full-time, other credit 0
Part-time, other credit 0
73
Table 2. Sample Demographic: Skills and Abilities
Category N
English skills
Excellent 2
Very good 4
Good 6
Fair 2
In need of improvement 1
Writing skills
Excellent 0
Very good 6
Good 7
Fair 1
In need of improvement 1
Reading comprehension
Excellent 0
Very good 6
Good 6
Fair 3
In need of improvement 0
74
Of the 15 research participants, 8 were male and 7 were female, 11 were
sciences, mathematics, education, and social sciences. The majority of the students (13)
had never taken a college critical thinking course, were full-time degree-seeking
sophomores between the ages of 17 and 22, and believed their English and writing skills
Results
This section has been organized around the study research questions. The
following format is used for each of the research questions: restatement of the research
question, a paragraph provided for clarification, a visual representation of the data, and an
questioning, and the acquisition of critical thinking skills. Research Question 4 focused
on the students’ perceptions of the changes in their knowledge, skill, dispositions, and
attitudes after direct instruction in critical thinking. These questions were addressed in the
Thinking Survey. The null hypothesis focused on finding no difference in student critical
thinking skills before and after direct instruction in rhetorical analysis. This hypothesis
75
Research Question 1: What were the students’ perceptions of critical thinking
Argument mapping is a way of laying out visually reasoning and evidence for and
against a statement or claim. Good argument maps clarify and organize thinking by
showing the logical relationships between thoughts that are expressed simply and
precisely. They help students organize and navigate around complex information, clarify
reasoning, and communicate reasoning effectively while supporting critical thinking. The
independent variable used to address this question was the instruction and
implementation of argument mapping. Student responses on the Pre- and Post- Critical
asked to circle the appropriate number for each item in response to the overall
effectiveness of argument mapping with 1 being the low score and 5 being the high score.
There were 10 questions in this portion of the survey. Analysis using this Likert-
type scale revealed that 9 (60%) of the 15 students in the course designated argument
mapping as an effective to highly effective strategy for identifying simple and complex
arguments; the author’s central thesis; the rhetorical function of each paragraph; claims,
objections, counterarguments, and inferences made by the author; the structure of the
reasoning process; and the missing layers in an argument. Six (40%) of the students had
mixed feelings, 3 students indicated the intervention was ineffective in helping them
identify a simple argument, 2 a complex argument, 3 the author’s central thesis, 3 the
76
rhetorical function of each paragraph, 1 the claims made by the author, 2 the objections
made by the author, 2 the counterarguments, 3 the inferences made by the author, 2 the
Student responses
Low High
score score
To what extent does argument mapping help
you identify . . . 1 2 3 4 5
A simple argument 1 2 5 5 2
A complex argument 2 4 7 2
77
Research Question 2: What were the students’ perceptions of critical thinking
intellectual standards. Paul and Elder took the essential elements of good critical thinkers
and developed The Thinker’s Guide to the Art of Socratic Questioning: Based on Critical
Thinking Concepts & Tools (Paul & Elder, 2006) and The Thinker’s Guide to Analytic
Thinking: How to Take Thinking Apart and What to Look for When You Do (Elder &
Paul, 2007). The researcher used both of these Thinker’s Guides in the study. The
independent variable used to address this questions was the instruction and
implementation of Paul’s Thinker’s Guides, which was the second intervention used in
this study. Student responses on the Post- Critical Thinking Survey (Appendix B) served
Analysis using this Likert-type scale indicated that 6 (40%) of the students
believed Paul’s Thinker’s Guides had been effective to highly effective in helping them
identify the author’s purpose, questions or problems the author had, point of view,
evidence and conclusions, and the elements of thought and how they assist in critical
thinking skills. Nine (60%) of the students offered mixed responses, indicating the
Thinker’s Guides had been ineffective in helping them identify the point of view,
78
Table 4. Student Perceptions of Critical Thinking: Paul’s Thinker’s Guides
Student responses
Low High
score score
To what extent does Paul’s Thinker’s Guides
help you identify . . . 1 2 3 4 5
Author’s purpose 2 5 8
Point of view 1 2 4 8
Unstated assumptions 1 6 3 5
Inferences 2 4 6 3
Bias or narrowness 2 2 9 2
Author’s evidence 5 5 5
Author’s conclusions 2 6 7
aids students in developing and evaluating their ideas. The Socratic method provides
opportunities for instructors to explore student thinking and model the necessary moves
79
for Socratic questioning as they question, analyze, and speculate out loud. The instruction
responses on the Post- Critical Thinking Survey (Appendix B) served as the dependent
Student responses
Low High
score score
To what extent does Socratic questioning
help you identify . . . 1 2 3 4 5
80
Analysis using this Likert-type scale indicated that 12 (80%) of the students
indicated Socratic questioning was effective to highly effective in helping them identify
the goals and purposes of the writer, the problem being addressed, relevant data and
information to support, key concepts and ideas, alternative points of view, and preparing
them for discussion, writing, and developing arguments. Three (20%) of the students
offered mixed responses, suggesting that Socratic questioning was ineffective in helping
Students were also asked to rate which of the instructional interventions extended
their knowledge of critical thinking. The rating criteria consisted of 1 = true, 2 = false,
and 3 = I don’t know. Eight (53%) of the students believed argument mapping had
extended their knowledge of critical thinking, 13 (86%) indicated the Thinker’s Guides
had extended their knowledge of critical thinking, and all 15 (100%) of the students
The final question on the Post- Critical Thinking Survey (Appendix B) offered
in the course. Students were asked to identify which of the three strategies taught in the
critical thinking course had been most effective in helping them become better thinkers.
Responses to this question revealed the students preferred Paul’s Thinker’s Guides or
Socratic questioning and that argument mapping had not been effective in helping them
become better thinkers. Nine (60%) of the students designated the Thinker’s Guides as
being most effective in helping them become better thinkers and 6 (40%) reported
Socratic questioning had been most effective in helping them become better thinkers.
81
Research Question 4: What did the students report about the changes in their
knowledge, skills, and dispositions relating to critical thinking after the direct instruction
in rhetorical analysis?
Student responses to the Knowledge, Skills, and Dispositions portions of the Pre-
and Post- Critical Thinking Surveys (Appendixes A & B) served as dependent variables.
Students were asked to rate each of the questions for how it described them using a 3-
Table 6 presents student responses to the Knowledge portion of the Pre- and Post-
Data from the Knowledge portion of the Pre- and Post- Critical Thinking Surveys
synthesis as a higher level thinking skill and of fallacious reasoning; 4-point gains were
achieved in students’ ability to clearly define critical thinking and evaluation and in their
definitions of comprehension and analysis as they relate to Bloom’s taxonomy; and there
were 1-point gains in students’ understanding of evaluation, bias, inference, and premise.
82
Table 6. Responses to Critical Thinking Survey: Knowledge
Correct responses
Define bias 12 13 1
Define conclusion 14 13 –1
Define evaluation 9 13 4
Define evidence 14 14 0
Define inference 8 9 1
Define premise 10 11 1
83
Table 7 presents student responses to the Skills portion of the Pre- and Post-
Student responses
Pretest Posttest
Don’t Don’t
Question True False know True False know Gain
Difficulty recognizing 2 12 1 3 12 0
difference between
assumptions & facts
Avoid generalizations 10 4 1 11 2 2 1
Analyze arguments, 12 2 1 14 1 2
interpretations, & beliefs
Difficulty comparing 2 12 1 2 11 2 –1
perspectives & considering
opposing points of view
84
Table 7. Responses to Critical Thinking Survey: Skills (continued)
Student responses
Pretest Posttest
Don’t Don’t
Question True False know True False know Gain
Difficulty recognizing 6 8 1 8 7 –1
contradictions
Fair-minded 12 1 2 12 1 2 0
As indicated in Table 7, for the Skills portion of both Pre- and Post- Critical
opposing points of view, with scores of 14 and 15, respectively, out of 15 possible points.
The highest gains, 5 points, occurred in recognizing the connotative and denotative
meaning of words, recognizing the author’s purpose, evaluating information for its
relevance, and accurately comprehending one’s point of view. A 3-point gain occurred in
there was a 2-point increase in students grasping the importance of analyzing and
evaluating arguments, interpretations, and beliefs. Further, there were 1-point gains in
85
acknowledging the merit of evaluating the credibility of sources of information, and
Table 8 presents student responses to the Dispositions portion of the Pre-and Post-
Student responses
Pretest Posttest
Don’t Don’t
Question True False know True False know
86
Table 8. Responses to Critical Thinking Survey: Dispositions (continued)
Student responses
Pretest Posttest
Don’t Don’t
Question True False know True False know
On the pretest and posttest surveys, all of the research participants (N = 15)
surveys reported they enjoyed learning new ways to think. There were 4-point gains in
students’ willingness to discover ways to improve their thinking and to handle situations
that require a lot of thinking. Data from the survey reflected 3-point gains in students’
preference for tasks that challenge their thinking abilities, for attempting projects that
require considerable thinking, for abstract thinking, and for deliberating about issues even
when those issues do not affect them personally. Further, there were 2-point gains in
87
students’ preference for tasks that are intellectually difficult, for solving complex
problems, and for tasks that challenge their thinking abilities. Finally, the data revealed 1-
point gains in student preference for thinking about long-term projects, relying on
thought to make their way to the top, and discovering new solutions to problems.
Table 9 provides the total scores for student responses to the Knowledge, Skills,
The total sum of all scores for the Knowledge portion of the Pre -Critical
Thinking Survey for the 15 students was 141 points (62%) out of a possible 225 points.
The measure of central tendency, or mean score of 9.4 was calculated by dividing the
sum of all scores by the number of participants (N = 15). The total sum of all scores for
the Skills portion of the Pre- Critical Thinking Survey was 156, with an average or mean
score of 10.4. Finally, the total sum of all scores for the Dispositions portion of the Pre-
Table 10 presents the total scores of student responses to the Knowledge, Skills,
88
Table 10. Results of Post- Critical Thinking
Survey
The total sum of all scores for the 15 research participants on the Knowledge
portion of the Post- Critical Thinking Survey was 174 (77%). The average or mean score
was 11.6. The total sum of all scores for the Skills portion of the Post- Critical Thinking
Survey was 182, with an average or mean score of 12.3. The total sum of all scores for
the Dispositions portion of the Post- Critical Thinking survey was 171, yielding a mean
score of 11.4.
Table 11 provides a display of the total raw scores and gains for the Pre- and Post-
Table 11 illustrates a 33-point (2.2 mean) gain in student scores in the Knowledge
portion of the Post- Critical Thinking Survey. Data obtained from the Skills portion of the
Post- Critical Thinking Survey conveyed a 26-point (1.8 mean) increase in student scores,
and data acquired from the Dispositions portion of the Post- Critical Thinking Survey
89
Table 11. Pre- and Post- Critical Thinking Surveys: Gains
Pretest Posttest
On the Post- Critical Thinking Survey, students were asked to rate the
interventions used in the research project using a 3-point scale. Fourteen of the 15
students (93%) indicated their knowledge, skills, and dispositions had improved through
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
The independent variable used to address this question was direct instruction in
rhetorical analysis using the treatment interventions, which included argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning. The dependent variable was the
The CCTST–2000 was ordered from Insight Assessment and was administered by
the researcher as a pretest and posttest. The CCTST–2000 was based on the APA Delphi
90
consensus conceptualization of critical thinking. The CCTST–2000 targets the core
At the end of the semester, the batch of CapScore CCTST answer forms was
returned to Insight Assessment. Insight Assessment then scanned and scored those
answer forms and returned a digital file reporting total and scale CCTST scores for each
test taker by PIN and descriptive statistics for the group of test takers as a whole.
The five subscales reported by Insight Assessment for the CCTST are analysis,
evaluation, inference, deductive reasoning, and inductive reasoning. A total score for
worth 1 point, for a total raw score of 34. The test takes 45 minutes to complete under
timed conditions; however, for the purposes of this study, the test was administered
Insight Assessment provided an Excel file and a Word file. The Excel file
included the scored pretest and posttest data for each test taker by PIN. The Word file
included descriptive statistics as well as additional t-test analyses. The basic descriptive
statistics provided as part of the CapScore scoring service were mean, median, trimmed
mean (the mean of the middle 90% of the data, used to remove outlier effect), standard
deviation, standard error of the mean, minimum, maximum, and first and third quartiles.
The median is the second quartile. This split the data into four groups, with 25% of the
The pretest scores for the CCTST–2000 are provided in Table 12. Note the total
91
Table 12. CCTST–2000 Pretest Scores
Table 13 displays the statistical analysis of the posttest scores. Note the total mean
score is 16.1, with a standard deviation of 6.35. As the total mean score for the CCTST–
2000 pretest was 14.9, these data suggested a 2.2 mean gain in student scores.
92
A paired samples t test and confidence interval (CI) were calculated and analyzed
by Insight Assessment for the CCTST–2000 pretest and posttest composite and subscale
scores to determine whether the difference in mean scores was statistically significant.
The CI was calculated to determine if all of the values within the range were defined by
the confidence limits of a sample statistic. The confidence limits “define the upper and
lower value of a range of values for a sample statistic that is likely to contain a population
parameter” (Gall et al., 2003, p. 638). The results of the t-test analyses are presented in
There did not appear to be a statistical significance between the pretest and
posttest in any of the five categories at the 5% significance level; however, a paired t test
revealed a p value of 0.123 for the difference in values from the pre- to the posttest for
the variable deduction and a p value of 0.112 for the variable inference.
93
Table 15. T-Test Analyses: Analysis, Inference, and Evaluation
Analysis
Inference
Evaluation
difference in deduction scores from the pretest to the posttest scores and approximately
89% confident there is a difference in inference scores from the pretest to posttest scores.
There were no significant differences between the pretest and posttest scores for
between the pretest and posttest scores for inference and deduction, the significance level
94
Table 16. T-Test Analysis: Induction and Deduction
Induction
Deduction
95
In this study, the null hypothesis was accepted. No statistical evidence was found
to suggest changes in student critical thinking skills; however, there was a difference
between the pretest and posttest scores for the variables deduction and inference.
Summary
This chapter provided a description of the research sample along with a discussion
procedures and an overview of the data and results as they relate to the four research
questions. Research Questions 1–3 were concerned with students’ perceptions of critical
thinking before and after direct instruction in argument mapping, Paul’s Thinker’s
Guides, and Socratic questioning. Student responses to the survey questions indicated
that although students found all three of the instructional strategies to be effective, the
majority of the students preferred Socratic questioning and Paul’s Thinker’s Guides as
highly effective interventions for developing their knowledge of critical thinking and for
The fourth question was concerned with students’ perceptions of the changes in
critical thinking knowledge, skills, and dispositions after direct instruction in rhetorical
analysis. For this question, the researcher examined the Knowledge, Skills, and
Dispositions portions of the Pre- and Post- Critical Thinking Surveys. Student responses
on the Knowledge portion of the Post- Critical Thinking Survey revealed a 2.2 increase in
student scores, the Skills section suggested a 1.8 increase in student scores, and the
average increase in student scores on the Dispositions portion of the survey was 1.8.
96
Data from the CCTST was used to address the null hypothesis: There will be no
difference in scores on the pretest and posttest on the California Critical Thinking Skills
Test–Form 2000 indicating changes in student critical thinking skills before and after
direct instruction in rhetorical analysis. Results from the CCTST–2000 revealed that none
of the paired t tests was significant at the 0.05 level of significance; however, a paired t
test revealed a p value of 0.123 for the difference in values from the pre- to the posttest
for the variable deduction and a p value of 0.112 for the variable inference. There were
no significant differences between the pretest and posttest scores for induction, analysis,
and evaluation. Although there was a slight difference between the pretest and posttest
scores for inference and deduction, the significance level was negligible (p > 0.2).
97
CHAPTER 5. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS
Introduction
The purpose of this study was to determine whether there would be improvement
in students’ critical thinking abilities, including knowledge, skills, and dispositions, after
argument mapping; Thinker’s Guides, based on Paul’s model of critical thinking; and
Socratic questioning. The goal of the study was, on the one hand to promote, critical
thinking knowledge, skills, and dispositions and, on the other hand, to determine
dispositions so they may transfer those skills across the academic disciplines and into
conclusions that were drawn from the data presented in chapter 4. Further, it provides
suggestions for further research, recommendations for practice, and connections to what
98
Overview of Methodology
The mixed methodology approach was used in this action research study. The
study pertained to one semester of a college critical thinking course that was taught in a
naturalistic setting by the researcher. The overall goal of the study was to examine the
skills, and dispositions. Twenty-nine students were initially enrolled in the Critical
Thinking 104 course. Of the 29 students, 19 agreed to participate in the study. Three of
those students dropped the course and one student was not present the final day of the
course and did not take the posttest; thus, the final research sample was comprised of 15
skills, and dispositions. The independent variable for this study was the critical thinking
interventions. The dependent variables were the pre- and posttest scores on the CCTST–
2000 and Pre- and Post- Critical Thinking Surveys. The researcher examined the
relationship between the variables involving teacher instruction of the interventions, the
students’ mastery of those skills, and the students’ perceptions of the effectiveness of the
interventions.
Summary of Findings
and after direct instruction in argument mapping. There were 10 questions in this portion
99
of the Post- Critical Thinking Survey (Appendix B). Analysis using a Likert-type scale
revealed that 9 (60%) of the 15 students in the course designated argument mapping as an
effective to highly effective strategy for identifying simple and complex arguments; the
author’s central thesis; the rhetorical function of each paragraph; claims, objections,
counterarguments, and inferences made by the author; the structure of the reasoning
process; and the missing layers in an argument. Six (40%) of the students had mixed
responses; students indicated that the intervention was ineffective in helping them
identify (a) simple and complex arguments; (b) the author’s central thesis and the
rhetorical function of each paragraph; (c) the claims, objections, and inferences made by
the author; (d) the counterarguments; (e) the structure of the reasoning process, and (f)
encouraged students to evaluate and modify the premise, structure, support, and logic of
their own written arguments; as well, it offered them practical tools for evaluating the
validity of arguments presented in the text and in a variety of editorials and media
images. Once students became familiar with and grew accustomed to practicing this
argument mapping was the first intervention presented by the teacher/researcher during
the first 5 weeks of an 18-week college course. At this point in time, students were
gradually becoming accustomed to close prose rhetorical analysis. Van Gelder (2005)
100
(p. 5), and student responses to argument mapping indicated they found this process to
be, at times, somewhat tiresome. Although the majority of the students believed argument
mapping contributed to their perceptions of critical thinking, students may not have been
given a sufficient amount of time to practice and develop the skills necessary for
identifying simple and complex arguments, the author’s thesis, the rhetorical functions of
each paragraph, objections, counterarguments, inferences made by the author, and the
As with this current study, Bessick’s (2008) study examined the effect of direct
instruction in critical thinking using van Gelder’s rational argument mapping and Paul’s
Thinker’s Guides. Her study consisted of freshman students who took repeat courses and
who received individual tutoring throughout the semester. Bessick’s study revealed no
achievement, which may have been a result of the individual tutoring students received
throughout the semester and the fact they were taking repeated courses. Although Bessick
believed direct instruction in critical thinking and reasoning can improve student learning
outcomes, she also acknowledged the ultimate educational goal should be to enable
students to transfer those skills learned in the classroom to their everyday lives.
Similarly, van Gelder (2005) acknowledged individuals are not naturally inclined
toward critical thinking; nonetheless, students’ critical thinking can improve through
guided instruction, which exposes students to good critical thinking and engages them in
the critical thinking process. Argument mapping offers students the essential tools for
recognizing the logical structure of rhetoric; however, the students had a limited amount
101
of time to practice and develop this skill as each intervention was facilitated at 5-week
result of effective instruction, repeated modeling of the procedures involved, and practice
after direct instruction in Paul’s Thinker’s Guides. The independent variable used to
address this question was the instruction and implementation of Paul’s Thinker’s Guides,
which was the second intervention used in this study. Student responses on the Post-
Critical Thinking Survey served as the dependent variable. Analysis using a Likert-type
scale indicated that 6 (40%) of the students believed Paul’s Thinker’s Guides had been
effective to highly effective in helping them identify the author’s purpose, questions or
problems the author had, point of view, assumptions, inferences, bias or narrowness,
contradictions in point of view, the author’s evidence and conclusions, and the elements
of thought and how they assist in critical thinking skills. Nine (60%) of the students
offered mixed responses, indicating the Thinker’s Guides had been ineffective in helping
them identify the point of view, unstated assumptions and inferences, bias or narrowness,
guides offer a comprehensible set of guidelines for analysis of textual as well as visual
arguments. The teacher/researcher taught Paul’s model explicitly, provided handouts, and
guided the students through frequent modeling of the process. Student written responses
and in-class discussion of the required readings suggested that the use of Paul’s elements
102
authors; nevertheless, students appeared to have the most difficulty with identifying
unstated assumptions and inferences, bias, and contradictions in point of view and,
initially, found Paul’s model to be difficult and even frustrating. For all of the required
essays for the course, students were guided through a variety of activities to assist them in
creating final drafts. The teacher/researcher engaged the students in prewriting activities;
students then used their prewriting as a guide as they developed a rough draft of their
essays. This was followed by peer-editing sessions in which students examined the drafts
peer-editing sessions and of student final drafts suggested this intervention assisted
students in offering a more systematic analysis and constructive criticism of their peers’
integrating Paul’s model of critical thinking into a U.S. history course on community
college students’ ability to think critically about U.S. history and everyday issues,
dispositions toward thinking critically, and knowledge of history content. Unlike the
current study, Reed’s study consisted of four sections: two sections were experimental
groups and two sections were control groups. Students participating in the study took
three pretests and four posttests to determine the effectiveness of Paul’s model. Students
in both the control and experimental groups took the Ennis–Weir Critical Thinking Essay
Test as a pretest and posttest. Unlike the current study, results from Reed’s study revealed
that “posttest means increased by 3.28 points in the experimental group” (p. 143). One
possible reason could be this researcher’s small sample size of (N = 15) compared to
103
Reed’s size of (N = 52). Another significant contributing factor for Reed’s success could
have been that the instructor for Reed’s study participated in intensive training in Paul’s
model, while this researcher had limited preparation, which consisted of 1 week of
training, for implementing this intervention. Regardless, Reed’s study revealed that
students’ ability to think critically can improve in a single course when provided with
deliberate instruction and that Paul’s model “can be successfully integrated into an
think critically within a domain and to their general critical thinking abilities” (p. 160).
Paul’s model of critical thinking and the Thinker’s Guides are invaluable tools for
facilitating and encouraging substantive critical thinking skills and for providing a
common model for instructional design, which can contribute to higher level learning.
Nevertheless, it is vital that students possess the “values, traits of mind, and dispositions
that underlie those skills” (Paul, 1995f, p. 119). As assessment should be specific,
consistent, and appropriate, it is necessary for students to possess the intrinsic motivation
and dispositions that contribute to the changes in perspective needed for substantial
assignments reflected an increase in their ability to think critically and analytically and to
class discussion and of students’ written homework assignments and essays revealed that
students’ ability to analyze the logic of articles, essays, and their own written work
104
improved. Concurrently, the students who illustrated this growth in analytical skills were
also those who appeared to be intrinsically motivated and who approached the course
work and the in-class discussions and collaboration with vigor and enthusiasm.
before and after direct instruction in Socratic questioning. The instruction and
responses on the Post- Critical Thinking Survey served as the dependent variable.
Analysis using a Likert-type scale indicated that 12 (80%) of the students revealed
Socratic questioning was effective to highly effective in helping them identify the goals
and purposes of the writer, the problem being addressed, relevant data and information to
support, key concepts and ideas, alternative points of view, and preparing them for
discussion, writing, and developing arguments. Three (20%) of the students offered
mixed responses, suggesting that Socratic questioning was ineffective in helping them
identify alternative interpretations of data and information, assumptions being made, and
There are three kinds of Socratic questioning that can be utilized by educators to
provides opportunities for teachers to explore with students “how [they] might find out if
something is true, logical, or reasonable” (Paul & Elder, 2006b, p. 48). This type of
questioning provides opportunities for educators to “listen critically” (Paul & Elder,
2006b, p. 48) and for students to “become self-correcting” (Paul & Elder, 2006b, p. 48).
reasons; asking a group whether they agree, suggesting parallel examples; and providing
105
an analogy, asking for a paraphrase, or rephrasing a student response (Paul & Elder,
2006b).
on diverse issues. It can be used to explore bias or student values, to identify clarity of
thought, or to probe controversial issues (Paul & Elder, 2006b). This type of questioning
can be used for introducing a subject to the class and for analyzing a topic. An issue can
be raised for discussion and students could be required to write about the issue or to form
Focused Socratic questioning occurs when instructors wish to explore and discuss
encourages them to “clarify, sort, analyze and evaluate thoughts and perspectives,
distinguish the known from the unknown [and] synthesize relevant factors and
knowledge” (Paul & Elder, 2006b, p. 50). Focused questioning requires preplanning,
implications, and consequences, and constructing a set of questions for the Socratic
There are also a variety of patterns in which Socratic dialogue can be facilitated in
the classroom. To encourage reasoning and discussion, Socratic questioning can begin
with whole-class discussion, which then leads to small-group speaking and listening,
whole-class Socratic speaking and listening, individual writing, and small-group speaking
and listening. An alternative to this would be a schema that begins with whole-class
106
Socratic discussion to clarify students’ understanding of the question; this would be
evaluation of the research. A third pattern would begin with whole-class Socratic
discussion with an introduction of the lesson and concepts involved, whole-class listening
and observing as the teacher models the task, followed by individual or group practice
as the teacher/researcher discussed the concepts and ideas taken from the required
text, on the Internet, or in the news media. The researcher would introduce the key
concepts or questions from the readings to stimulate student thinking. In order to allow
students to reflect on the concepts or issues, students were given 5–10 minutes to free-
write a response. This was followed by class discussion of the issue as the
Exploratory Socratic questioning was used when whole group discussion was
used to introduce key questions or concepts to the class and to stimulate thinking.
Students were often given a prompt that dealt with a controversial issue discussed in a
previous reading assignment. They were then allowed 5–10 minutes to free-write their
response. This was often followed by small-group speaking and listening in order to
encourage students to think and reason cooperatively and to assess their ideas. For the
purpose of disciplining the questioning, students were given a script that required them to
107
focus on Paul’s elements of reasoning. The teacher/researcher assigned a questioner and a
recorder for each group. In this way, students worked with their peers in an informal
setting rather than having the teacher/researcher address focused formal Socratic
and assessing their ideas using preassigned questions to assist them in developing clarity,
accuracy, precision, relevance, depth, breadth, logic, significance, and fairness. This was
followed by class discussion for review and assessment of the small-group work and for
The collaborative process was used most frequently for the Socratic questioning
in the critical thinking course and, at times, it was highly productive; however, using the
single Socratic questioner to the whole group might have been more powerful and
engaging for the students. Further, researcher observations revealed that in order for the
modeled and monitored, and students must clearly comprehend their assigned roles in the
Socratic process; otherwise, the likelihood exists that students will either enter into a
debate or that certain students will monopolize the dialogue. Everyone should be allowed
to participate and to go beneath the surface of what is being discussed, constantly probing
the complexities of their ideas and the questions. Students must also be assured they do
not need to come to a group consensus, only that open discussion is allowed and that
pedagogy, Yang’s (2008) study was designed to investigate the effects of teaching critical
thinking skills, with the major goal of investigating whether students’ critical thinking
108
skills would improve after they participated in Socratic dialogues. Yang measured the
critical thinking scores using the CCTST; however, Yang achieved positive results,
indicating that instructors who use Socratic dialogues during small-group online
discussions can successfully improve students’ critical thinking skills. This improvement
in learning was accomplished “as students interact with their peers or instructor and
benefit from combining their levels of expertise, offering support, distributing the
thinking load, and confronting alternative points of view” (Yang, 2008, p. 261).
Paul (1995e) believed Socratic questioning embodies the spirit of critical thinking
as it presents opportunities for students to develop and evaluate their ideas; nevertheless,
the vital component for success with this instructional strategy is appropriate modeling of
the necessary moves for this kind of questioning. Moreover, rather than indoctrinating
learners, Newton (1978) contended that educators should stimulate learners while
encouraging them to become active participants in the learning process. Although higher
cognitive questioning is a viable strategy for teaching critical thinking, and “intelligence
is a reaction to the social and physical world” (Newton, 1978, p. 289), learners must
possess the willingness to acknowledge alternative perspectives while they adapt, refine,
and apply their newly acquired knowledge in a problem-solving manner (Newton, 1978).
Open-Ended Questions
Students were also asked to rate which of the instructional interventions extended
their knowledge of critical thinking. Eight (53%) of the students designated argument
critical thinking, 13 (86%) of the students indicated the Thinker’s Guides had extended
109
their knowledge of critical thinking, and all 15 (100%) of the students suggested Socratic
thinking.
Concurrently, students were asked to identify which of the three strategies that
had been taught in the critical thinking course had been most effective in helping them
become better thinkers. Responses to this question revealed argument mapping had not
been effective in helping them become better thinkers. Nine (60%) of the students stated
Paul’s Thinker’s Guides had been most effective in helping them become better thinkers,
and 6 (40%) of the students designated Socratic questioning as being effective in helping
scrutiny required for rhetorical analysis. Once students had been exposed to Paul’s
Thinker’s Guides and Socratic questioning, they might have been more accustomed to
this process. Further, students may not have been given a sufficient amount of time to
practice and develop the skills necessary for identifying the structure and logic of an
argument and for effective argument mapping. As well, students may have preferred
Paul’s Thinker’s Guides because they appear to encourage more reflection on the
elements of thinking, on key concepts, and on the assumptions and inferences being
Argument maps are spatial layouts for presenting information, they rely on encouraging
the reader or writer to view the structure and the overall essence of the argument, and
they assist in conveying the essential logic of the rhetoric. Students may not have found
110
this process to be as interesting as Paul’s model, which appears to be more analytical and
their knowledge, skills, and dispositions relating to critical thinking after the direct
Dispositions portions of the Pre- and Post- Critical Thinking Surveys (Appendixes A &
Halpern (1999) contended that critical thinking skills can be taught as rhetorical
analysis, problem solving, and decision making, yet her contention is that these skills
should be taught within a variety of contexts and for transfer. However, students must not
only be taught the necessary skills needed for critical thinking, they must also have a
Data from the Knowledge portion of the Pre- and Post- Critical Thinking Surveys
a higher level thinking skill and of fallacious reasoning. Gains were also realized in
critical thinkers, and to define comprehension and analysis as they relate to Bloom’s
111
For the Skills portion of both the Pre- and Post- Critical Thinking Surveys,
view, with the highest gains in recognizing the connotative and denotative meaning of
words, recognizing the author’s purpose, evaluating information for its relevance, and
accurately comprehending one’s point of view. There were also gains in students’
oversimplifications.
On the pretest and posttest Dispositions portions of the surveys, all of the research
15 students on both surveys reported they enjoyed learning new ways to think. There
were also gains in students’ willingness to discover ways to improve their thinking, for
attempting tasks that challenge their thinking abilities, for abstract thinking, and for
deliberating about issues even when those issues do not affect them personally.
think about long-term projects, rely on thought to make their way to the top, and discover
The total sum of all scores for the Knowledge portion of the Pre- Critical
Thinking Survey for the 15 students was 141 points (62%) out of a possible 225 points.
The measure of central tendency, or mean score of 9.4 was calculated by dividing the
sum of all scores by the number of participants (N = 15). The total sum of all scores for
112
the Skills portion of the Pre- Critical Thinking Survey was 156, with an average or mean
score of 10.4. Finally, the total sum of all scores for the Dispositions portion of the Pre-
The total sum of all scores for the 15 research participants on the Knowledge
portion of the Post- Critical Thinking Survey was 174 (77%). The average or mean score
was 11.6. The total sum of all scores for the Skills portion of the Post- Critical Thinking
Survey was 182, with an average or mean score of 12.3. The total sum of all scores for
the Dispositions portion of the Post- Critical Thinking survey was 171, yielding a mean
score of 11.4. These data revealed a 33-point (2.2 mean) gain in Knowledge, a 26-point
(1.8 mean) increase in Skills, and a 26-point (1.8 mean) increase in Dispositions on the
Data from the Post- Critical Thinking Survey suggested instructors can make a
positive impact on students’ perceptions of their own critical thinking abilities through
direct instruction in rhetorical analysis. The highest gains were achieved in the
Knowledge portion of the Post- Critical Thinking Survey. This 2.2 mean gain in student
reasoning, ability to clearly define critical thinking and evaluation, and discernment of
strong critical thinkers. Students also experienced gains in their ability to characterize
evaluation, comprehension, and analysis as they relate to Bloom’s taxonomy, and in their
The 1.8 mean gain in the Skills portion of the Post- Critical Thinking Survey
viewpoints. The highest gains, 5 points, occurred in students’ ability to distinguish the
113
connotative and denotative meaning of words, recognize the author’s purpose, evaluate
information for relevance, and accurately comprehend point of view. Further, gains were
analyzing and evaluating arguments, consider alternative points of view, acknowledge the
The 1.8 mean gain in the Dispositions portion of the Post- Critical Thinking
Survey revealed an increase in students’ desire to enjoy learning new ways to think. The
highest gains were achieved in students’ willingness to discover ways to expand their
thinking and to handle situations that require extensive thinking. Additional gains were
made in students’ preference for tasks that challenge their thinking abilities for
attempting projects that require considerable thinking, for abstract thinking, and for
deliberating about issues. Finally, gains were realized in students’ preference for
intellectually difficult tasks, for solving complex problems, for deliberating about long-
term projects, for relying on thought for successful living, and for discovering new
solutions to problems.
The final question on the Post- Critical Thinking Survey asked students if they
believed their knowledge, skills, and dispositions had improved through direct instruction
in critical thinking. Fourteen of the 15 students (93%) indicated direct instruction had
Although the students’ overall scores on the CCTST–2000 were not at the level of
critical thinking and their perceptions of their own critical thinking abilities. Explicit
114
teaching of the components of critical thinking provided the students with the necessary
vocabulary, tools, and awareness of their own critical thinking dispositions essential for
effective argument analysis, and it could provide them with the knowledge of how to
apply that vocabulary and knowledge to the analysis of their own writing, to the rhetoric
in written texts, and to the rhetoric with which they are confronted in the media.
The null hypothesis focused on the extent to which the pretest and posttest scores
indicate changes in the students’ knowledge, skills, and dispositions relating to critical
thinking after direct instruction in argument mapping, Paul’s Thinker’s Guides, and
Socratic questioning. For both the pretest and posttest, the CCTST–2000 was the sole
quantitative instrument used. The total score on the CCTST–2000 provided the overall
measure of students’ critical thinking skills. Changes in students’ critical thinking scores
were analyzed comparing the mean pre-CCTST–2000 score to the mean post-CCTST–
2000 score using a t test. The independent variable used to address the null hypothesis
was direct instruction in rhetorical analysis using the treatment interventions, which
included argument mapping, Paul’s Thinker’s Guides, and Socratic questioning. The
Data from the pretest scores for the CCTST–2000 indicated the total mean score
was 14.9, with a deviation of responses of 4.55. Statistics from the CCTST–2000 posttest
scores revealed a total mean score of 16.1, with a standard deviation of 6.35. As the total
mean score for the CCTST–2000 pretest was 14.9, these data suggested a 2.2 mean gain
in student scores.
Assessment for the CCTST–2000 pretest and posttest composite and subscales scores to
115
determine whether the difference in mean scores was statistically significant. The CI was
calculated to determine if all of the values within the range were defined by the
confidence limits of a sample statistic. The resulting data suggested there does not appear
to be a statistical significance between the pretest and posttest in any of the five
0.123 for the difference in values from the pre- to the posttest for the variable deduction
and a p value of 0.112 for the variable inference. This indicated the researcher can be
approximately 88% confident there is a difference in deduction scores from the pretest to
the posttest scores and approximately 89% confident there is a difference in inference
scores from the pretest to posttest scores. There were no significant differences between
the pretest and posttest scores for induction, analysis, and evaluation. Although there was
a slight difference between the pretest and posttest scores for inference and deduction, the
In this study, the null hypothesis was accepted. No statistical evidence was found
to suggest changes in student critical thinking skills; however, there was a slight
difference between the pretest and posttest scores for the variables deduction and
have been demonstrated to capture gains in CT over time periods of one quarter or one
semester” (p. 28), there were no significant differences in students’ pretest and posttest
scores on the CCTST–2000. Nonetheless, this research study still could provide insight
for educators who are interested in instructional pedagogy that may well promote
116
The critical thinking course could be revised to include more emphasis on
instructional strategies that target the core critical thinking skills of analysis,
spent on the analysis of the logic of a variety of arguments, on recognizing bias, and on
evaluating rhetoric for its credibility. Further, students often have a tendency to base their
decisions on preconceived notions, assumptions, and inferences, and, often, they appear
recognize the logic or the fallacious reasoning in texts, in the media, and in their own
thinking and writing could promote the quality of fair-mindedness and the critical
Facione, and Giancarlo (2000) affirmed, “To the open-minded, imaginative, and
Encouraging students’ critical thinking abilities and designing pedagogy that can
the critical thinking course suggested that student motivation was a contributing factor to
student success and achievement. P. A. Facione et al. (2000) argued it is not sufficient to
focus on critical thinking skill development without nurturing the “internal motivation to
use those skills in the appropriate circumstances” (p. 34). An effective tool for nurturing
students’ critical thinking dispositions, such as modeling good critical thinking behaviors,
could have been used more frequently and deliberately by the researcher in the critical
thinking course.
117
Tucker’s (2008) study at Kent School of Social Work examined the development
infused curriculum. Tucker utilized the CCTST–2000 to measure the students’ critical
thinking performance on pre- and posttest scores. Similar to this researcher’s study,
Tucker’s pretest and posttest scores revealed no significant differences except for the
and question relevant information from a variety of sources, consider alternatives and
Tucker’s study did not yield an overall improvement in critical thinking, as with the
current study, data analysis revealed a slight increase in deductive scores. Tucker
maintained the reason for the lack of improvement in scores was, in part, due to an
ineffective curriculum and lack of motivation on the part of teachers and faculty.
Regardless of this, she believed students’ critical thinking skills can be improved by
providing a course specifically designed for critical thinking. She further claimed that
“critical thinking skills are not the result of merely rewording goals and objectives, but of
explicit attention to skill development in each individual course and session through the
is essential; however, the need exists for the faculty to be motivated to create critical
reflection.
118
Limitations
research sample. Although the findings should prove useful, it is doubtful they can be
generalized to a target population in other locations unless the sample sizes share similar
to other cases can be done, but it must be done on a case-by-case basis” (p. 10).
Possible limitations exist due to the modest sample size of 15 (N = 15) students
and the short term of instruction necessitated by an 18-week semester course. Research
suggests that critical thinking is a lifelong process and that successful interventions may
need to occur over an academic year or over several years. Ideally, a longitudinal study
and dispositions as a result of the proposed interventions. The researcher was the teacher
of an intact class, and no attempt at randomization occurred; therefore, the study was
situation. In teacher action research, data are collected and analyzed to assess and
improve educational practice. Since the researcher was the instructor in the critical
thinking course, there was potential for researcher bias as the researcher may have
authority over the participants. The researcher understood this and attempted to approach
the study with caution and without preconceived assumptions of possible outcomes or
results. Care was taken to maintain clarity and objectivity through sharing with research
119
participants and through personal reflection, assessment, and modification of
employed in a critical thinking course and did not attempt to measure the influence of
the Hawthorne effect, research participants were aware of the fact they were participating
in a research study, and this may have affected their performance (Gall et al., 1996).
Recommendations
thinking abilities, including knowledge, skills, and dispositions, after the intervention of
direct instruction in rhetorical analysis using argument mapping; Thinker’s Guides, based
on Paul’s model of critical thinking; and Socratic questioning. Further research should be
knowledge, skills, and dispositions so students may transfer those skills across the
academic disciplines and into their personal and professional lives. Additional
thinking skills and critical thinking dispositions and designing research studies that
This research study consisted of a modest sample size of 15 students and a short
needed using a larger sample size and, possibly, a control group and an experimental
group to determine the extent to which direct instruction using argument mapping, Paul’s
120
Thinker’s Guides, and Socratic questioning can improve students’ critical thinking
abilities.
To increase student learning and achievement in critical thinking and the overall
effectiveness of the course, the length of the study could be extended to an entire year, or
a longitudinal study could be conducted across several critical thinking courses. This
could provide instructors with more time to implement the instructional interventions,
members who wish to teach critical thinking. The Delphi Report’s definition of critical
thinking could prove useful for those who are attempting to conceptualize and implement
Although the students’ overall scores on the CCTST–2000 were not at the level of
critical thinking and their perceptions of their own critical thinking abilities. Educators
provide the students with the necessary vocabulary, tools, and awareness of their own
critical thinking dispositions essential for effective argument analysis, and with the
knowledge of how to apply that vocabulary and knowledge to the analysis of their own
writing, to the rhetoric in written texts, and to the rhetoric with which they are confronted
in the media.
121
To improve students’ critical thinking knowledge, skills, and dispositions,
educators could develop instructional pedagogy with purposeful learning activities that
encourage critical thinking abilities. Students must be taught how to think critically, and
frequent and explicit teacher modeling of argument mapping, Paul’s Thinker’s Guides,
As well, support from the administrative staff along with the implementation of
Paul’s Thinker’s Guides, and Socratic questioning, could improve the quality of
instruction and student learning. Ideally, staff members could be encouraged to observe
and collaborate with instructors from other colleges and universities who have
successfully implemented critical thinking into their course curriculum. In California, the
including serviceable and functional technology, are not necessarily imperative but
perspectives. Students should be allowed to openly express their opinions without fear of
judgment, censure, or reproach, and educators can encourage optimal critical thinking
122
Conclusions
that will promote students’ critical thinking skills. The researcher employed a variety of
Data from the Post- Critical Thinking Survey suggested instructors can make a
positive impact on students’ perceptions of their own critical thinking abilities through
direct instruction in rhetorical analysis. The highest gains were achieved in the
Knowledge portion of the Post- Critical Thinking Survey. This 2.2 mean gain in student
reasoning, in their ability to clearly define critical thinking and evaluation, and in their
discernment of strong critical thinkers. There was also a 1.8 mean gain in the Skills
portion of the survey. This gain in student scores occurred in students’ ability to
distinguish the connotative and denotative meaning of words, recognize the author’s
purpose, evaluate information for relevance, and accurately comprehend point of view.
As well, there was a 1.8 mean gain in the Dispositions portion of the Post- Critical
Thinking Survey. The highest gains in this category were achieved in students’
willingness to discover ways to expand their thinking and to handle situations that require
extensive thinking. Finally, when asked if they believed their knowledge, skills, and
students (93%) indicated direct instruction had been instrumental in developing their
critical thinking.
123
Although data from the research findings indicated there were no significant
differences between the pretest and the posttest scores for induction, analysis, and
evaluation, there was a slight difference between the pretest and posttest scores for
inference and deduction; however, the significance level was negligible (p > 0.2).
Nonetheless, findings from the study suggested that repeated, purposeful implementation
their own critical thinking abilities. Further, the results and the procedures used in this
by participating in teacher training courses that support and facilitate the development of
effective pedagogies and assessment tools for the encouragement of student critical
thinking abilities.
124
REFERENCES
Bondy, K. N., Koenigseder, L. A., Ishee, J. H., & Williams, B. G. (2001). Psychometric
properties of the California Critical Thinking Tests. Journal of Nursing
Measurement, 9, 309–328.
Brookfield, S. D., Tennant, M., & Pogson, P. (2005). Theory and methods of educating
adults. New York: Wiley.
Carroll, R. T. (2007). Teaching critical thinking. Retrieved November 11, 2008, from
http://www.skepdic.com/essays1
Cowden, E. M., Chivore, B. R. S., Maravanyika, O. E., Nyagura, L. M., & Sibanda, I. M.
(1999). Overview of educational research methodologies and assessment criteria
of the quality of educational research. In Zimbabwe Education Sector Analysis
Review Group (Ed.), Review of education sector analysis in Zimbabwe 1990–
1996 (pp. 9–14). Retrieved March 26, 2010, from http://unesdoc.unesco.org/
images/0011/001177/117705eo.pdf
125
Dahms, M., Geonnotti, K., Passalacqua, D., Schilk, J. N., Wetzel, A., & Zulkowsky, M.
(2008). The educational theory of Lev Vygotsky: An analysis. Retrieved March 1,
2009, from http://www.newfoundations.com/GALLERY/Vygotsky.html
Elder, L. (2000). Why critical thinking is essential to the community college mission and
why it will be difficult to achieve. Retrieved December 7, 2008, from http://www
.criticalthinking.org/page.cfm?PageID=529&CategoryID=73
Elder, L. (2007). Why critical thinking? Retrieved March 26, 2010, from http://www
.criticalthinking.org/page.cfm?PageID=796&CategoryID=103
Elder, L., & Paul, R. (2007). The thinker’s guide to analytic thinking: How to take
thinking apart and what to look for when you do. Dillon Beach, CA: Foundation
for Critical Thinking.
Engelmann, K. (n.d.). What is direct instruction (DI)? Retrieved December 2, 2008, from
http://www.nifdi.org/
Ennis, R. (1992, March). Critical thinking: What is it? Proceedings of the 48th annual
meeting of the Philosophy of Education Society, Denver, Colorado. Retrieved
May 26, 2006, from http://www.ed.uiuc.edu/PES/92_docs/Ennis.HTM
Ennis, R. (2002). An outline of goals for a critical thinking curriculum and its
assessment. Retrieved October 12, 2009, from http://faculty.ed.uiuc.edu/rhennis/
outlinegoalsctcurasses3.html
Facione, N. C., Facione, P. A., Blohm, S. W., & Gittens, C. A. (2008). California Critical
Thinking Skills Test: CCTST test manual 2008 edition. Millbrae, CA: California
Academic Press.
Facione, P. A. (1992). Manual, the California Critical Thinking Skills Test, Form A and
Form B. Millbrae, CA: California Academic Press.
126
Facione, P. A. (2000). The California Critical Thinking Skills Test: CCTST—Form 2000.
Millbrae, CA: Insight Assessment/The California Academic Press
Facione, P. A. (2006). Critical thinking: What it is and why it counts. Retrieved October
20, 2006, from www.calpress.com/pdf_files/what&why.pdf
Facione, P. A., Facione, N. C., Blohm, S. W., Howard, K., & Giancarlo, C. A. (1998).
The California Critical Thinking Skills Test manual (revised). Millbrae, CA: The
California Academic Press.
Facione, P. A., Facione, N. C., & Giancarlo, C. A. (2000). The disposition toward critical
thinking: Its character, measurement, and relationship to critical thinking skill.
Informal Logic, 20(1), 61–84. Retrieved June 12, 2009, from http://www.insight
assessment.com/9articles2.html
Gall, M. D., Gall, J. P., & Borg, W. R. (1996). Educational research: An introduction
(6th ed.). New York: Allyn & Bacon.
Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction
(7th ed.). New York: Allyn & Bacon.
Gardner, H. (1993). Multiple intelligences: The theory in practice. New York: Basic
Books.
Halpern, D. F. (1999, December). Teaching for critical thinking: Helping college students
develop skills and dispositions of a critical thinker. New Directions for Teaching
and Learning, 80, 69–74. Retrieved November 18, 2008, from http://www3
.interscience.wiley.com/journal/101523939/abstract?CRETRY=1&SRETRY=0
Hatcher, D. L. (2000). Critical thinking: A new definition and defense. Retrieved October
12, 2009, from The Center of Critical Thinking Web site: http://www.bakeru.edu/
crit/literature/dlh_ct_defense.htm
Jackson, M. (2008). The erosion of attention and the coming dark age: Distracted. New
York: Prometheus Books.
127
Jacobs, S. S. (1999). The equivalence of Forms A and B of the California Critical
Thinking Skills Test. Measurement and Evaluation in Counseling and
Development, 31, 211–222.
Laird, T. F. N. (2005). The California Critical Thinking Skills Test (CCTST). Retrieved
February 1, 2009, from Wabash College, Center of Inquiry in the Liberal Arts
Web site: http://liberalarts.wabash.edu/cila/displayStory_print.cfm?news_ED=
5121
McPeck, J. (1981). Critical thinking and education. New York: St. Martin’s.
Mendelman, L. (2007, December). Critical thinking and reading. Journal of Adolescent &
Adult Literacy, 51(4), 300–302. Retrieved October 1, 2008, from Academic
Search Premier database.
National Center for Education Statistics. (2007). Digest of education statistics: 2007.
Retrieved March 2, 2009, from http:/nces.ed.gov/programs/digest/d07/
Ormrod, J. E. (2004). Learning theory and the educational process. Boston: Pearson.
Paul, R. (1993). Critical thinking, moral integrity, and citizenship: Teaching for the
intellectual virtues. Retrieved October 25, 2006, from http://www.criticalthinking
.org/resources/articles/ct-moral-integrity.shtml
Paul, R. (1995a). The critical connection: Higher order thinking that unifies curriculum,
instruction, and learning. In J. W. Willsen & A. J. A. Binker (Eds.), Critical
thinking: How to prepare students for a rapidly changing world (pp. 103–151).
Santa Rosa, CA: Foundation for Critical Thinking.
128
Paul, R. (1995b). Introduction. In J. W. Willsen & A. J. A. Binker (Eds.), Critical
thinking: How to prepare students for a rapidly changing world (pp. xi–xv). Santa
Rosa, CA: Foundation for Critical Thinking.
Paul, R. (with Binker, A. J. A.). (1995c). Critical thinking: Basic questions and answers.
In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare
students for a rapidly changing world (pp. 91–100). Santa Rosa, CA: Foundation
for Critical Thinking.
Paul, R. (with Binker, A. J. A.). (1995d). Glossary: A guide to critical thinking terms and
concepts. In J. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to
prepare students for a rapidly changing world (pp. 521–552). Santa Rosa, CA:
Foundation for Critical Thinking.
Paul, R. (with Nosich, G.). (1995f). A model for the national assessment of higher order
thinking. In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to
prepare students for a rapidly changing world (pp. 103–151). Santa Rosa, CA:
Foundation for Critical Thinking.
Paul, R. (with Willsen, J.). (1995g). Critical thinking: Identifying the targets. In J.
Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a
rapidly changing world (pp. 17–36). Santa Rosa, CA: Foundation for Critical
Thinking.
Paul, R. (2004). The state of critical thinking today: As the organizer in developing
blueprints for institutional change. Retrieved December 11, 2008, from http://
www.criticalthinking.org/professionalDev/the-state-ct-today.cfm
Paul, R., & Elder, L. (1997, April). Socratic questioning. Retrieved December 7, 2008,
from http://www.criticalthinking.org/page.cfm?PagelID=606&CategoryID=64
Paul, R., & Elder, L. (2000). The role of questions in teaching, thinking and learning.
Retrieved December 6, 2008, from http://www.criticalthinking.org/articles/
thinking-some-purpose.cfm
129
Paul, R., & Elder, L. (2003, Winter). Critical thinking . . . and the art of close reading,
part I. Journal of Development Education, 27(2), 36–39. Retrieved October 1,
2008, from Academic Search Premier database. (AAT 11542636)
Paul, R., & Elder, L. (2004, Fall). Critical thinking . . . and the art of close reading, part
III. Journal of Developmental Education, 28(1), 36–37. Retrieved October 1,
2008, from Academic Search Premier database. (AAT 14576885)
Paul, R., & Elder, L. (2006a). First International Academy on Critical Thinking.
Retrieved February 8, 2007, from http://www.criticalthinking.org/conference/
cambridge.shtml
Paul, R., & Elder, L. (2006b). The thinker’s guide to the art of Socratic questioning:
Based on critical thinking concepts & tools. Dillon Beach, CA: Foundation for
Critical Thinking.
Paul, R., Elder, L., & Bartell, T. (1997a). A brief history of the idea of critical thinking.
Retrieved November 8, 2008, from http://www.criticalthinking.org/aboutCT/brief
HistoryCT.cfm
Paul, R., Elder, L., & Bartell, T. (1997b). Research findings and policy
recommendations: Study of 38 public universities and 28 private universities to
determine faculty emphasis. Retrieved December 7, 2008, from http://www
.criticalthinking.org/page.cfm?PageID=506&CategoryID=69
Scanlan, J. S. (2006). The effect of Richard Paul’s universal elements and standards of
reasoning on twelfth grade composition. Master’s thesis, Alliant International
University, San Diego, CA. Retrieved March 2, 2009, from http://www.critical
thinking.org/research/Abstract-JSCANLAN.cfm
Scriven, M., & Paul, R. (1987). Defining critical thinking. Retrieved November 9, 2008,
from http://www.criticalthinking.org/aboutCT/define_critical_thinkng.cfm
130
Shea, R. H., Scanlon, L., & Aufses, R. D. (2008). The language of composition: Reading,
writing, rhetoric. Boston: Bedford/St. Martin’s.
Sternberg, R. J. (2003). Four alternative futures for education in the United States: It’s
our choice. School Psychology Quarterly, 18(4), 431–445. Retrieved October 11,
2008, from ProQuest Education Journal database. (AAT 611229291)
Texas State Technical College. (2004). Critical Thinking Surveys. Retrieved January 4,
2009, from Center for Excellence in Teaching Web site: http://www.harlingen.tstc
.edu/cetl/about.aspx
Twardy, C. (2003). Argument maps improve thinking. Teaching Philosophy, 27(2), 1–19.
van Gelder, T. (2005). Teaching critical thinking. College Teaching, 53(1), 41–46.
Retrieved September 2, 2005, from Academic Search Premier database. (AAT
15348172)
Yang, Y. (2008, June 1). A catalyst for teaching critical thinking in a large university
class in Taiwan: Asynchronous online discussion with the facilitation of teaching
assistants. Educational Technology Research and Development, 56(3), 241–264.
Retrieved January 28, 2009, from ERIC database. (ERIC Document Reproduction
Service No. EJ791102)
131
APPENDIX A. PRE- CRITICAL THINKING SURVEY
The following information is being gathered for statistical purposes only. Please answer
each question on the form provided.
1. Gender
a. Male
b. Female
2. Race
a. African-American
b. American Indian/Alaskan Native
c. Asian or Pacific Islander
d. Hispanic
e. White – Non-Hispanic
3. Current Age
a. 17-19
b. 20-22
c. 23-25
d. 26-30
e. Over 30
132
6. Student status:
a. Full-time degree seeking student
b. Part-time degree seeking student
c. Full-time other credit student
d. Part-time other credit student
10. Have you ever taken a Critical Thinking course (or a course similarly labeled) that
was devoted to teaching critical thinking skills or abilities?
a. Yes
b. No
11. Did you take one or more courses in grades 9-12 that explicitly taught critical
thinking skills while incorporating them into regular course work?
a. Yes
b. No
12. Have you taken one or more courses at a two or four year college or university
that explicitly taught critical thinking skills while incorporating them into regular
course work?
a. Yes
b. No
133
Please rate each of the following statements for how it describes you. Use a 3-point scale
in which:
3: Knowledge
13) _____Synthesis is a higher-level thinking skill than summary.
14) _____Critical thinking is using disciplined, self-directed thinking to achieve a desired
outcome.
15) _____Comprehension is a lower-level thinking skill than knowledge.
16) _____Evaluation focuses on making a judgment based on analysis of a statement or
proposition.
17) _____Synthesis can be equated with creative thinking.
18) _____Strong critical thinkers are those who use the intellectual skills of critical
thinking selectively to promote and serve their own personal interests.
19) _____Analysis is a higher level skill than synthesis.
20) _____An argument consists of reasons for or against something
21) _____To be biased toward a particular idea or issue is to be fair-minded toward
opposing points of view.
22) _____A conclusion is the last step in the reasoning process.
23) _____To evaluate is to judge or determine the worth or quality of something.
24) _____Evidence is the data on which a judgment or conclusion might be based or by
which proof or probability might be established.
25) _____Fallacies or fallacious reasoning conforms to the rules of good reasoning.
26) _____An inference is an intellectual act by which one concludes that something is
true in light of something else being true
27) _____A premise is the last step in the reasoning process.
134
38) _____I have difficulty taking into account opposing points of view.
39) _____I find it easy to offer relevant, constructive evidence to support my points of
view.
40) _____I frequently compare or transfer insights learned in the classroom to real life
situations.
41) _____When exploring an issue, I sometimes have difficulty recognizing
contradictions..
42) _____I consider myself to be fair-minded.
Note. The questions in the “Dispositions” portion of the Pre- Critical Thinking Survey are from Assessing a
Feasible Effective Critical Thinking Protocol for Community Colleges (p. 227), by K.C. Metcalfe, 2007,
Ph.D. Dissertation, Capella University, United States – Minnesota. Adapted for use with permission.
135
APPENDIX B. POST- CRITICAL THINKING SURVEY
The following information is being gathered for statistical purposes only. Please answer
each question on the form provided.
1. Gender
a. Male
b. Female
2. Race
a. African-American
b. American Indian/Alaskan Native
c. Asian or Pacific Islander
d. Hispanic
e. White – Non-Hispanic
3. Current Age
a. 17-19
b. 20-22
c. 23-25
d. 26-30
e. Over 30
6. Student status:
136
a. Full-time degree seeking student
b. Part-time degree seeking student
c. Full-time other credit student
d. Part-time other credit student
10. Have you ever taken a Critical Thinking course (or a course similarly labeled) that
was devoted to teaching critical thinking skills or abilities?
a. Yes
b. No
11. Did you take one or more courses in grades 9-12 that explicitly taught critical
thinking skills while incorporating them into regular course work?
a. Yes
b. No
12. Have you taken one or more courses at a two or four year college or university
that explicitly taught critical thinking skills while incorporating them into regular
course work?
a. Yes
b. No
137
Please rate each of the following statements for how it describes you. Use a 3-point scale
in which:
138
39) _____I find it easy to offer relevant, constructive evidence to support my points of
view.
40) _____I frequently compare or transfer insights learned in the classroom to real life
situations.
41) _____When exploring an issue, I sometimes have difficulty recognizing
contradictions..
42) _____I consider myself to be fair-minded.
Please circle the appropriate number for each item. Low High
Score Score
Argument Mapping
To what extent does Argument Mapping help you identify
58) a simple argument 1 2 3 4 5
59) a complex argument 1 2 3 4 5
60) the author’s central thesis 1 2 3 4 5
61) the rhetorical/argumentative function of
each paragraph 1 2 3 4 5
62) the claims made by the author 1 2 3 4 5
63) the objections made by the author 1 2 3 4 5
64) the counter arguments 1 2 3 4 5
65) the inferences made by the author 1 2 3 4 5
66) the structure of the reasoning process 1 2 3 4 5
139
67) the missing layers in an argument 1 2 3 4 5
Socratic Questioning
To what extent does Socratic Questioning help you identify
78) goals and purposes of writer 1 2 3 4 5
79) the nature of the question 1 2 3 4 5
80) the problem being addressed 1 2 3 4 5
81) relevant data and information to support 1 2 3 4 5
82) alternative interpretations of the data and information 1 2 3 4 5
83) key concepts and ideas 1 2 3 4 5
84) assumptions being made 1 2 3 4 5
85) implications and consequences of what is being said 1 2 3 4 5
86) alternative points of view 1 2 3 4 5
87) How effective was Socratic Questioning in preparing
you for discussion, writing, and for developing your
arguments 1 2 3 4 5
Please rate each of the following statements for how it describes you. Use a 3-point scale
in which:
140
90) _____The instruction in Richard Paul’s Thinker’s Guides has extended my
knowledge of critical thinking.
91) _____I believe that my knowledge, skills, and dispositions have improved through
direct instruction in critical thinking.
92) _____The critical thinking process that I have learned in English 104 has helped me
to make better and more informed decisions in my everyday life.
93) _____I will be able to apply what I have learned in this course about thinking
critically to a practical situation in my life.
94) What has been the most frustrating or difficult aspect of learning to think more
critically?
95) Of the three strategies that you learned in Critical Thinking 104, which has been the
most effective in helping you become a better thinker?
Note. The questions in the “Dispositions” portion of the Post-Critical Thinking Survey are from Assessing a
Feasible Effective Critical Tthinking Protocol for Community Colleges (p. 227), by K.C. Metcalfe, 2007,
Ph.D. Dissertation, Capella University, United States – Minnesota. Adapted for use permission.
141
APPENDIX C. REQUEST FOR PERMISSION TO DR. METCALF
Dr. Metcalfe,
Thank-you,
Lauren McGuire
142
APPENDIX D. LETTER OF PERMISSION FROM DR. METCALF
Yes, you are free to use whatever you need from my work. Be sure to contact Dr. Diane
Halpern, McKenna College Claremont University, (here in CA.), as much of my work
was built on hers. She will be delighted, as am I, that you are engaged in this type of
research and that you have found our work together useful.
Kim C. Metcalfe
Assistant Professor of Early Childhood Studies & Psychology
Educational Psychology Ph.D.
Assistant Chair Health Human & Public Services
143