Learning and Instruction 36 (2015) 86e103

Contents lists available at ScienceDirect

Learning and Instruction

journal homepage: www.elsevier.com/locate/learninstruc

The role of epistemic perspectives in comprehension of multiple

author viewpoints
Sarit Barzilai a, *, Yoram Eshet-Alkalai b
a
Department of Learning, Instruction, and Teacher Education, Faculty of Education, University of Haifa, 199 Abba Hushi Boulevard, Mount Carmel, Haifa
3498838, Israel
b
Department of Education and Psychology, The Open University of Israel, 1 University Road, Ra'anana 4353701, Israel

a r t i c l e i n f o a b s t r a c t

Article history: A key challenge of fostering digital literacy is developing learners' ability to construct knowledge from
Received 19 December 2013 information sources that present diverse viewpoints. This study investigated the relation between
Received in revised form learners' epistemic perspectives and their comprehension of authors' viewpoints. Additionally, the study
14 December 2014
examined if epistemic perspectives and viewpoint comprehension predict information source integra-
Accepted 19 December 2014
Available online
tion and explored how epistemic perspectives moderate the impact of conflicts on viewpoint compre-
hension. 170 participants responded to an epistemic thinking assessment, read conflicting or converging
blog-posts regarding a socio-scientific controversy, wrote arguments, and completed tasks assessing
Keywords:
Epistemic thinking
viewpoint comprehension. Absolutism and multiplism were found to be negative predictors and eval-
Digital literacy uativism a positive predictor of viewpoint comprehension. Viewpoint comprehension mediated the
Multiple document comprehension relation between epistemic perspectives and information source integration in written arguments.
Viewpoint comprehension Conflicts between sources improved viewpoint comprehension only in high levels of multiplism and
Information source integration evaluativism. The findings advance the understanding of the relation between learners' epistemic
thinking and multiple document comprehension.
© 2014 Elsevier Ltd. All rights reserved.

1. Introduction specifically, often missing from learners' comprehension of multi-

One of the ongoing challenges of developing learners' digital
literacy is fostering learners' ability to construct knowledge from
diverse, multiple, and conflicting online information sources
(Alexander, 2012; Goldman & Scardamalia, 2013; Leu et al., 2007).
Previous research has indicated that learners of all ages find it
difficult to construct knowledge from multiple online information
sources in critical and meaningful ways (e.g., Brand-Gruwel &
Stadtler, 2011; Eshet-Alkalai & Chajut, 2009; Eshet-Alkalai & Geri,
2007; Goldman, Braasch, Wiley, Graesser, & Brodowinska, 2012;
Walraven, Brand-Gruwel, & Boshuizen, 2008; Wiley et al., 2009). A
central difficulty in comprehension of multiple information sources
is the tendency to focus on the information presented and to pay
low attention to the source of that information (e.g., Bråten,
Strømsø, & Salmero  n, 2011; Britt & Aglinskas, 2002; Flanagin &
Metzger, 2007; Gasser, Cortesi, Malik, & Lee, 2012). More
* Corresponding author.
E-mail addresses: sarit.barzilai@edu.haifa.ac.il (S. Barzilai), yorames@openu.ac.il
(Y. Eshet-Alkalai).
ple information sources is an awareness of the connections be-
tween source and content, such as the ways in which authors'
viewpoints underlie the construction of knowledge (Rouet, 2006;
Wineburg, 1991). This is no minor matter because, as several re-
searchers have argued, comprehension of authors' viewpoints is
necessary for knowledge construction from multiple information
sources (Bråten, Britt, Strømsø, & Rouet, 2011; Britt & Rouet, 2012).
For example, in order to make sense of conflicting online accounts
regarding a socio-scientific issue, one needs to understand how
these accounts are shaped by the viewpoints of diverse experts and
stakeholders.
Author viewpoints are the situated and distinct ways in which
authors perceive the issue at hand. Author viewpoints can reflect
their stances toward that issue (e.g., pro or con), their disciplinary
perspectives (e.g., environmental or economic), and more.
Comprehension of author viewpoints entails constructing a rep-
resentation that captures the important aspects of these view-
points. Although understanding authors' viewpoints is assumed to
be necessary for comprehending diverse information sources, we
know very little about the factors that might contribute to author
viewpoint comprehension.

http://dx.doi.org/10.1016/j.learninstruc.2014.12.003
0959-4752/© 2014 Elsevier Ltd. All rights reserved.
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
Previous studies have shown that learners' epistemic thinkingd
that is, thinking about knowledge and about how people knowd
plays a significant role in learning from multiple information sour-
ces (e.g., Barzilai & Zohar, 2012; Bråten, Britt, et al., 2011; Ferguson,
Bråten, & Strømsø, 2012; Mason, Ariasi, & Boldrin, 2011; Pieschl,
Stahl, & Bromme, 2008). This study extends prior research by
examining the relation between learners' epistemic perspectives,
defined as “theories-in-action” regarding knowledge and knowing
(Kuhn & Weinstock, 2002), and the comprehension of multiple
author viewpoints when reading online information sources. We
conjectured that epistemic perspectives may predict author view-
point comprehension because one needs to acknowledge the con-
structed nature of knowledge in order to appreciate the impact of
authors' viewpoints on their writing. An epistemic perspective that
situates the source of knowledge in a reality that is directly know-
able might downplay the role of authors' viewpoints in knowledge
construction. In contrast, an epistemic perspective that locates the
source of knowledge in human minds might emphasize the role of
authors' viewpoints in knowledge construction. Therefore, learners'
epistemic perspectives may affect how much effort they devote to
understanding author viewpoints as they read and how they eval-
uate the significance of these viewpoints. Consequently, compre-
hension of author viewpoints might help explain the relations found
in prior studies between epistemic thinking and comprehension of
multiple information sources.
However, the characteristics of online information sources may
also impact their comprehension. Specifically, recent studies pro-
posed that conflicts between information sources can lead learners
to pay more attention to sources and to their characteristics
(Braasch, Rouet, Vibert, & Britt, 2012; Stadtler & Bromme, 2014;
Strømsø, Bråten, Britt, & Ferguson, 2013). This seems to suggest a
promising instructional approach for improving author viewpoint
comprehension by introducing conflicts between viewpoints.
However, learners whose epistemic perspectives dispose them to
downplay author viewpoints might also be less attentive to dis-
crepancies between these viewpoints. Such learners may therefore
benefit less from the introduction of conflicts between sources.
Hence, several related objectives guided the current study. First,
we sought to examine the relations between learners' epistemic
perspectives and author viewpoint comprehension in order to
better understand the contributors to viewpoint comprehension.
Second, we wanted to test if author viewpoint comprehension
indeed predicts better integration of multiple information sources,
and if it can thus help explain the relations between epistemic
perspectives and multiple document comprehension. Third,
assuming author viewpoint comprehension might be related to
learners' epistemic perspectives, we wondered whether intro-
ducing conflicts between viewpoints can improve author view-
point comprehension, when learners' epistemic perspectives are
taken into account.
In the following sections, we provide a succinct overview of
multiple document comprehension, address the importance of
author viewpoint comprehension, and describe some of the factors
that might impact viewpoint comprehension. We then focus on one
of those factors, epistemic thinking, and present the prior research
that led to this study.
1.1. Multiple document comprehension
Critically constructing knowledge from multiple and diverse
information sources is a core competence for learning in the in-
formation age (Alexander, 2012; Goldman & Scardamalia, 2013; Leu
et al., 2007). Multiple document comprehension research has
grown in recent years, as investigators endeavor to understand the
processes of multiple document comprehension and the roles of
87
learning conditions, learning tasks, and learner characteristics in
the enactment of these complex processes (for recent reviews and
summaries see Bråten, Britt, et al., 2011; Britt & Rouet, 2012;
Goldman & Scardamalia, 2013; Rouet, 2006; Stadtler & Bromme,
2013). To clarify the terminology used in the current study, we
use the terms “document” and “information source” interchange-
ably to refer to varied informational artifacts (Britt & Rouet, 2012).
The term “source” is reserved for the author, setting, and form
features of the document (Britt & Rouet, 2012).
The challenge of comprehending multiple documents stems
from the fact that documents often do not provide a single coherent
account of the issue or situation at hand, but rather may provide
partially or wholly inconsistent accounts and describe multiple
possible situations (Rouet, 2006; Wineburg, 1991). Indeed, Bråten
and his colleagues define multiple documents comprehension as
involving the “building of a coherent mental representation of an
issue from the contents of multiple texts that deal with the same
issue from different perspectives” (Bråten, Ferguson, Anmarkrud, &
Strømsø, 2013, pp. 322e323). A prominent framework for
analyzing this complex task is the Documents Model framework
initially proposed by Perfetti, Rouet, and Britt (1999) and further
developed by Rouet, Britt, and their colleagues (Bråten, Britt, et al.,
2011; Britt & Rouet, 2012; Britt, Rouet, & Braasch, 2013; Rouet,
2006; Rouet & Britt, 2011).
In brief, the Documents Model framework extends the situation
model theory of text comprehension (Kintsch, 1988) to cases in
which learners read multiple documents that provide diverse ac-
counts of the same situation. In such cases, readers need to construct
not a single situation model but rather a multiple situations model
that represents the agreements and discrepancies in the accounts
they read. The situations model has more recently been referred to
as the integrated mental model (Britt & Rouet, 2012; Britt, Rouet &
Braasch, 2013). Importantly, in addition to representing the con-
tent, the Documents Model framework proposes that readers also
construct an additional layer of representation, called the intertext
model, which includes information about document features, such
as author, genre, audience, and so forth. These features are repre-
sented as document nodes that are connected to the situation model
content (source-toecontent links) and to other document nodes
(source-to-source links). The steps involved in construction of a
documents model are detailed in the Multiple Documents e Task-
based Relevance Assessment and Content Extraction (MD-TRACE)
model (see Bråten, Britt, et al., 2011; Rouet & Britt, 2011).
Thus, in order to understand multiple documents that deal with
the same issue, readers need to build a coherent mental repre-
sentation of the information described in the documents as well as
to represent where that information came from and how the
sources of information are interrelated (e.g., in opposition or
agreement). The tagging of information to sources enables readers
to differentiate between documents, to evaluate the contribution of
each document to the global representation of the situation, and to
understand why different sources provide divergent accounts of
the event or phenomenon in question (Rouet, 2006). As this
description makes clear, multiple document comprehension is a
complex task that entails multiple components and processes. In
the current study, we focused on the relation between viewpoint
comprehension and one of these processes e integration of mul-
tiple documents. Integration involves organizing and combining
information from multiple documents to form an integrated model
of the issue at hand (cf. Goldman, Lawless, & Manning, 2013).
1.2. Author viewpoint comprehension
Understanding how different points of view come into play in
written discourse is vital for multiple document comprehension.
88 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
Britt and Rouet (2012) noted that learners need to be able to
represent multiple perspectives in order to understand in-
consistencies and conflicts between documents. This claim is sup-
ported by previous research that has shown that memory for
sources, that is, memory of who-said-what, contributes to multiple
document comprehension (Strømsø, Bråten, & Britt, 2010). How-
ever, the specific contribution of author viewpoint comprehension
to multiple document comprehension has not yet been examined,
to our knowledge. More generally, author viewpoint comprehen-
sion has received little empirical attention, and there is a need to
better understand the predictors and outcomes of viewpoint
comprehension.
Author viewpoints can be generally described as the situated
and distinct ways in which authors perceive the subject they are
writing about. Authors' viewpoints can shape their presentation
and interpretation of events and phenomena, influence the infor-
mation that they choose to include or exclude, determine the as-
pects they choose to emphasize, and so forth. Author viewpoints
are wide ranging and may reflect differences in authors' opinions,
attitudes, purposes, commitments, worldviews, professional back-
grounds, theoretical and methodological approaches, prior
knowledge and experience, and more (Britt & Rouet, 2012; Miller &
Boix Mansilla, 2004). In the present study, we focused on two as-
pects of authors' viewpoints: the positions of the authors regarding
the controversy in question and the disciplinary perspectives of the
authors. These facets of authors' viewpoints were chosen because
of their central importance for understanding information sources
about socio-scientific controversies. Thus, author viewpoints were
more narrowly defined, in the current study, as the particular po-
sitions and disciplinary perspectives from which the authors
perceive the issue at hand. It is important to note that author
viewpoints are distinct from author expertise and trustworthiness.
For example, two authors may both be trustworthy experts and yet
argue from different viewpoints. Conversely, viewpoints do not
necessarily indicate expertise and trustworthiness. These require
further assessment of the author's competence and motivations
(Sperber et al., 2010).
Author positions were defined in the present study as the
opinions or attitudes of the authors regarding the issue at hand. For
example, in online information sources about socio-scientific con-
troversies authors can express pro, con, or balanced positions
regarding decisions or actions. Disciplinary perspectives were
defined, following Miller and Boix Mansilla (2004), as perspectives
that are “based on commitment to a system of theories, a body of
professional knowledge, a discipline, or a discourse community” (p.
4). Author positions and disciplinary perspectives were considered
as interrelated aspects of authors' viewpoints: Disciplinary per-
spectives may underlie authors' positions and provide grounds for
their claims. In the Documents Model framework, author positions
and perspectives are encoded in the document node (Bråten, Britt,
et al., 2011; Perfetti et al., 1999).
Author viewpoint comprehension involves constructing a rep-
resentation that captures the pertinent aspects of these viewpoints.
Comprehension of author viewpoints is necessary in order to un-
derstand how the content of the message might be shaped by the
ways in which particular authors perceive the issue. Authors' po-
sitions and disciplinary perspectives are not always stated explicitly
in the text, and readers need to infer them from the authors' ar-
guments and from author information such as profession, work
experience, and affiliations. Disciplinary perspectives and domains
of expertise may be also be inferred from lexical cues, such as use of
technical terms (Jucks, Becker, & Bromme, 2008). However, readers
do not necessarily pay attention to authors' viewpoints and may fail
to infer or evaluate them accurately (Barzilai & Zohar, 2012; Jucks &
Bromme, 2011). Therefore, it is important to understand which
factors might improve readers' comprehension of author
viewpoints.
1.3. Contributors to source comprehension
Author viewpoint comprehension is a particular aspect of source
comprehension, which can be generally described as constructing a
representation of the source. Several individual and contextual
factors have been found to be related to source comprehension and
might, therefore, impact author viewpoint comprehension as well.
Readers' goals can shape how they approach the reading task and
their attention to sources (Britt et al., 2013). Interest in the topic
may lead people to pay closer attention to sources and to evaluate
them more carefully and critically (Petty & Cacioppo, 1986;
Pornpitakpan, 2004; Strømsø et al., 2010). Disciplinary expertise
and prior knowledge may also affect learners' tendency to evaluate
sources (Rouet & Britt, 2011; Rouet, Favart, Britt, & Perfetti, 1997;
Wineburg, 1991). Epistemic thinking has also been found to play
an important role in source evaluation (e.g., Bråten, Britt, et al.,
2011), and will be discussed in greater depth in the next sections.
Source comprehension is also related to document characteris-
tics. Specifically, conflicting, contradictory, disrupted, or missing
information may increase source salience (Braasch et al., 2012;
Stadtler & Bromme, 2014; Strømsø et al., 2010; Strømsø et al.,
2013). Braasch et al. (2012) proposed the discrepancy-induced
source comprehension assumption (D-ISC assumption). According
to this assumption, conflicting or discrepant information leads
readers to pay greater attention to source information, encode it
more deeply, and, as a result, increases the likelihood that readers
will construct source-content links. In line with this assumption,
conflicts between sources should be expected to increase author
viewpoint comprehension. However, readers may also respond to
conflicts by ignoring them or by explaining them away (Stadtler &
Bromme, 2014). Stadtler and Bromme (2014) suggested that
readers' responses to conflicts may be influenced by their epistemic
thinking.
1.4. Epistemic thinking
Epistemic thinking can be generally described as people's
thinking about knowledge and about how people know (Hofer &
Bendixen, 2012; Hofer & Pintrich, 1997; Kuhn, 2001; Kuhn &
Weinstock, 2002). Epistemic thinking has been studied from mul-
tiple approaches and conceptualizations. The current study adopts
a model of epistemic thinking developed by Kuhn and her col-
leagues (Kuhn, 1991, 2001; Kuhn & Weinstock, 2002; Weinstock,
2009). According to this model, epistemic thinking is a devel-
oping “theory-in-action” that comes into play in the course of
everyday knowledge judgments and knowledge construction
(Kuhn & Weinstock, 2002). The “theory-in-action” approach views
epistemic thinking not as a set of static beliefs but rather argues
that epistemic thinking emerges in multidimensional forms when
people reason about specific challenges (Kuhn & Weinstock, 2002).
Thus, epistemic thinking informs the ways in which people un-
dertake tasks, but is also activated in and shaped by contexts (Kuhn,
Cheney, & Weinstock, 2000; Kuhn, Iordanou, Pease, & Wirkala,
2008; Tabak & Weinstock, 2008).
The Kuhn et al. model describes several developing epistemic
positions or perspectives that encompass multiple interrelated di-
mensions of knowledge and knowing such as source and justifi-
cation of knowledge (Kuhn et al., 2000; Weinstock, 2009). The
developmental trajectory describes a shift from an absolutist
perspective that the source of knowledge is a directly knowable
reality and that knowledge is therefore objective, certain, and
justified by appeal to reality, to a multiplist perspective according to
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
which the source of knowledge is in the self and knowledge is
multiple, subjective, uncertain, and justified by personal prefer-
ences and judgments. The further shift to an evaluativist perspec-
tive is driven by the need to coordinate the subjective and objective
dimensions of knowledge. According to the evaluativist view,
knowledge is constructed by people within a particular perspective
but is also anchored in evidence and shared standards. Thus, un-
certainty is acknowledged without forsaking the need to evaluate
knowledge production.
Epistemic thinking is a multi-faceted construct that includes
cognitive-level epistemic processes and strategies as well as met-
acognitive knowledge, skills, and experiences related to the nature
of knowledge and to knowing strategies and processes (Barzilai &
Zohar, 2014). The epistemic strategies involved in multiple docu-
ment comprehension, such as corroboration and sourcing, are
viewed here as cognitive-level epistemic strategies, because they
involve thinking about the epistemic properties of specific knowl-
edge claims and information sources (e.g., their reliability), and not
about the way one thinks about these claims and information
sources (e.g., how one evaluates reliability) (Barzilai & Zohar, 2014).
Absolutist, multiplist, and evaluativist epistemic perspectives entail
diverse epistemic metacognitive knowledge about the nature of
people as sources of knowledge and about epistemic tasks and
strategies (Kuhn, 2001). We argue that this epistemic meta-
cognitive knowledge informs and guides the selection and execu-
tion of the epistemic strategies and processes involved in multiple
document comprehension, such as inferring and evaluating author
viewpoints. In the next section we expand these claims by
considering prior research on epistemic thinking and multiple
document comprehension.
1.5. Epistemic thinking and multiple document comprehension
Learners' epistemic beliefs, i.e. their views about knowledge and
knowing (Bråten, Britt, et al., 2011), have been found in previous
studies to play an important role in multiple document compre-
hension. For example, learners who view knowledge as complex
and tentative are more likely to form connections between texts, to
integrate information from multiple texts, to self-regulate their
reading of multiple texts, and therefore achieve better compre-
hension of multiple documents (Bråten, Britt, et al., 2011; Bråten &
Strømsø, 2009, 2010; Pieschl et al., 2008; Rukavina & Daneman,
1996; Strømsø, Bråten, & Samuelstuen, 2008). Views of knowl-
edge as constructed from multiple information sources by corrob-
orating accounts have also been found to be positively related to
multiple document comprehension (Bråten, Ferguson, Strømsø, &
Anmarkrud, 2013; Ferguson, Bråten, Strømsø, & Anmarkrud,
2013). However, epistemic beliefs in the knower as a constructor
of knowledge and in personal justification have been found be
negatively correlated with multiple document comprehension
(Bråten, Ferguson, Strømsø, et al., 2013; Bråten, Strømsø, &
Samuelstuen, 2008; Strømsø et al., 2008).
Few studies have examined the relation between the epistemic
perspectives described in the Kuhn et al. model and evaluation and
integration of multiple information sources. Brem, Russell, and
Weems (2001) identified an absolutist approach to website evalu-
ation that was characterized in attempts to seek out “true” facts and
“real” scientists. Barzilai and Zohar (2012) found that sixth-grade
students who expressed evaluativist views more successfully in-
tegrated multiple information sources than students who
expressed absolutist views. Specifically, evaluativists were more
likely to identify and compare websites' viewpoints while reading
and were somewhat more likely to refer to source viewpoints as a
trustworthiness evaluation criterion.
89
Bråten, Britt, Strømsø, and Rouet (2011) proposed extending the
Documents Model framework in order to explain why epistemic
beliefs predict multiple document comprehension. Three main
proposals recur in their analysis: First, epistemic beliefs may be
related to readers' perceptions of the task and its goals; for
example, to whether the goal of the task is viewed as accumulation
of facts or as integration of information. Second, epistemic beliefs
might be related to the standards and criteria readers adopt for task
performance and completion; for example, to standards of source
evaluation. Third, epistemic beliefs may be related to the strategic
processes readers employ to meet task goals and standards; for
example, to how much attention they pay to document sources and
whether they engage in corroboration.
In light of the proposals made by Bråten, Britt, et al. (2011) and by
Kuhn (2001), we suggest that learners' epistemic perspectives entail
epistemic metacognitive knowledge that informs and guides task
perceptions, standards for task performance, and selection and
performance of epistemic strategies when reading multiple infor-
mation sources. For example, an absolutist perspective that the
source of knowledge outside the self might lead learners to perceive
the goals of the task as lining up facts transmitted by reliable ex-
perts, to evaluate source trustworthiness primarily using standards
of author expertise, and to employ strategies of corroborating facts.
In contrast, an evaluativist perspective that knowledge is con-
structed by knowers might lead learners to perceive the goals of the
task as understanding the different points of view on the matter, to
take into consideration in their source evaluations that experts can
have different perspectives and positions, and, consequently, to
engage in epistemic strategies of inferring, evaluating, and
comparing author viewpoints in order to determine how these
viewpoints shape authors' accounts.
2. The present study
Although previous studies have shown that learners' epistemic
thinking plays a significant role in learning from multiple docu-
ments (e.g., Barzilai & Zohar, 2012; Bråten, Britt, et al., 2011;
Ferguson et al., 2012; Mason et al., 2011; Pieschl et al., 2008),
there is still a need to better understand how and why epistemic
thinking is related to multiple document comprehension. Hence,
the purpose of the current study was to expand the understanding
of the role of learners' epistemic perspectives in multiple document
comprehension by focusing on the relation between learners'
epistemic perspectives and comprehension of multiple online
author viewpoints.
Furthermore, although comprehension of author viewpoints is
considered vital for forming coherent representations of multiple
documents (Britt & Rouet, 2012; Rouet, 2006), we know very little
about how readers comprehend author viewpoints and about the
factors that might contribute to author viewpoint comprehension.
More specifically, to our knowledge, there has been only one pre-
vious study that examined the connections between epistemic
perspectives and author viewpoint comprehension (Barzilai &
Zohar, 2012). Barzilai and Zohar's study was conducted among a
small sample of sixth graders, and compared absolutist and eval-
uativist epistemic perspectives only. In the current research, we
examined whether the relation between epistemic perspectives
and author viewpoint comprehension is evident in a large sample
of university students and investigated multiplist perspectives as
well.
The study addressed two additional research gaps. First, the
study extended prior research on the relation between source
memory and comprehension and multiple document comprehen-
sion (Strømsø et al., 2010) by specifically examining the contribu-
tion of author viewpoint comprehension to integration of multiple
90 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
documents in students' written arguments. The contribution of
author viewpoint comprehension to integration of multiple docu-
ments can help explain the processes through which epistemic
thinking impacts multiple document comprehension. Second, the
study expanded prior research on the contribution of conflicts to
source memory and comprehension (e.g., Braasch, et al., 2012) by
examining how learners' epistemic perspectives moderate the
impact of conflicts between viewpoints on author viewpoint
comprehension. It stands to reason that if epistemic perspectives
are related to author viewpoint comprehension then they might
also impact readers' reactions to conflicts between viewpoints.
Thus, assessment of the effect of conflicts can provide supporting
evidence for our theoretical conjectures and extend the examina-
tion of their implications. The present study is part of a wider
investigation into learners' understanding of conflicting expert
accounts conducted by the authors.
The current study was guided by three research questions:
1 How are learners' absolutist, multiplist, and evaluativist epistemic
perspectives related to their ability to comprehend multiple online
author viewpoints?
From an absolutist epistemic perspective, knowledge is
perceived as originating in a directly knowable reality (Kuhn &
Weinstock, 2002). The role of the authors is to report the facts
(Weinstock, 2011), and differences in author positions may be
attributed to erroneous, incomplete, or biased reports (Kuhn &
Weinstock, 2002). Therefore, endorsement of absolutist perspec-
tives may lead learners to downplay the role of author viewpoints in
knowledge construction and pay less attention to the ways in which
authors' viewpoints shape their claims. Hence, we hypothesized
that absolutism will be a negative predictor of author viewpoint
comprehension (H1a). In contrast, multiplist and evaluativist per-
spectives locate the source of knowledge in human minds and
acknowledge the role of subjectivity in knowledge construction
(Kuhn, 1991; Kuhn & Weinstock, 2002; Leadbeater & Kuhn, 1989).
However, from a multiplist perspective, divergent accounts are
perceived as highly personal and subjective opinions; whereas from
an evaluativist perspective, divergent accounts are grounded in
wider theoretical, social, and cultural frames of reference and reflect
differences in evaluative standards. Furthermore, from a multiplist
perspective, discrepant accounts represent idiosyncratic positions
that cannot be reconciled, whereas from an evaluativist perspective,
discrepant accounts can be compared, evaluated, and possibly
reconciled (Kuhn, 1991; Kuhn & Weinstock, 2002; Weinstock &
Cronin, 2003). Hence, endorsement of evaluativist views might
lead learners to pay greater attention to author viewpoints as they
read and engage in more comparison and evaluation of divergent
viewpoints. Therefore, we hypothesized that multiplism will be a
positive predictor of viewpoint comprehension (H1b), but that
evaluativism will be an even stronger positive predictor of view-
point comprehension than multiplism (H1c).
2 How are epistemic perspectives and author viewpoint compre-
hension related to the integration of multiple information sources
in written arguments?
Representing authors' viewpoints, understanding how these
viewpoints are related to each other, and realizing how the infor-
mation presented is shaped by the authors' viewpoints, have been
proposed as necessary for forming a coherent representation of
multiple accounts (Bråten, Britt, et al., 2011; Britt & Rouet, 2012).
Therefore, we expected that participants who exhibit better
comprehension of author viewpoints will also have a more
coherent and well-connected representation of the information
sources they read, and would be more likely to write arguments
that take into consideration multiple views and multiple informa-
tion sources (H2a). Based on studies that documented positive re-
lations between views of knowledge as uncertain and processing of
multiple texts (Pieschl et al., 2008; Strømsø et al., 2008), we hy-
pothesized that multiplism will be a positive predictor of integra-
tion (H2b). An alternative hypothesis might have been that
multiplist views of justification as based on personal knowledge
and experience would negatively impact integration (Bråten,
Ferguson, Strømsø, et al., 2013). Yet, we expected that awareness
of the uncertain and relative nature of knowledge might exert a
stronger influence, based on the observation that the transition
from absolutist to relativist thinking enables better reasoning in
complex tasks (Weinstock & Cronin, 2003). In light of previous
research on the role of epistemic thinking in integration of multiple
information sources (Barzilai & Zohar, 2012; Bråten, Britt, et al.,
2011), we also expected evaluativism to be a positive predictor of
integration because it entails awareness of the constructed and
tentative nature of knowledge and of the need to evaluate multiple
accounts (H2c).
3 Do epistemic perspectives moderate the impact of conflicts be-
tween viewpoints on author viewpoint comprehension?
According to the D-ISC assumption, conflicting or discrepant
information leads readers to pay greater attention to source infor-
mation and to process it more deeply (Braasch et al., 2012). How-
ever, some learners might not recognize conflicts between
information sources (Barzilai & Zohar, 2012; Mason & Boldrin,
2008) and may, therefore, benefit less or not benefit at all from
introduction of such conflicts. Specifically, limited attention to
author viewpoints might cause learners to fail to identify or suffi-
ciently process conflicts between them. Therefore, we hypothe-
sized that conflicts between viewpoints will have no impact or low
impact on viewpoint comprehension in high levels of absolutism
(H3a), a positive impact in high levels of multiplism (H3b), and a
positive impact in high levels of evaluativism as well (H3c).
3. Method
3.1. Overview
The study employed an experimental design. Participants'
epistemic perspectives were assessed before reading using a
scenario-based epistemic thinking assessment that referred to the
study topic. Participants were then randomly assigned to reading
conflicting or converging blog-posts that presented various ac-
counts regarding seawater desalination. After reading, author
viewpoint comprehension was assessed using three tasks: A
viewpoint identification task, a viewpoint description task, and a
viewpoint evaluation task. All three tasks were used as indicators of
author viewpoint comprehension, enabling us to triangulate data
sources. Participants were also asked to write integrative argu-
ments regarding the controversy. The extent to which participants
considered and justified multiple accounts in their arguments and
based their arguments on multiple information sources were
employed as indicators of integration of the multiple information
sources. Additionally, we measured two possible confounding
variables, which, according to prior research, might have impacted
our findings e topic knowledge and topic interest.
3.2. Participants
The participants were 170 Hebrew-speaking students studying
in an Israeli university (61% female; 81% BA students; 19% MA
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
students; Mage ¼ 28.02 years, SD ¼ 7.35). 83.5% were born in Israel,
15.3% were immigrants from Russian-speaking countries, and 1.2%
from other countries. However, the immigrants had lived in Israel
for many years (M ¼ 17.24 years, SD ¼ 4.87). The participants had
diverse educational backgrounds and studied in various university
departments in the fields of social sciences, humanities, and exact
sciences. The sample did not include students with reading,
writing, or visual learning difficulties. 98.8% of the participants had
an Internet connection at home; 94.7% of the participants reported
that they use the Internet daily; 92.9% reported that they use the
Internet to search for information on a daily or weekly basis. Some
students participated in the study as part of their degree re-
quirements; others were recruited through ads on the university
website and on notice boards. These students were compensated
for their participation (~10 euro).
3.3. Materials
The topic chosen for the study was the socio-scientific contro-
versy regarding the increasing use of seawater desalination in Israel
to supply growing water demand. This topic interested us as a case
study for exploring how well students comprehend multiple
viewpoints regarding a socio-scientific topic in online information
sources. In order to examine author viewpoint comprehension, we
designed several short blog-posts regarding seawater desalination
in which we manipulated author viewpoints but kept all other
source and content features as similar as possible. This was done in
order to test how differences between author viewpoints might
impact author viewpoint comprehension while controlling for other
information source characteristics. Information source character-
istics have been shown to have an impact on sourcing and
comprehension (e.g., Bråten, Strømsø, & Britt, 2009; Pornpitakpan,
2004). Specifically, differences in source reliability have been
demonstrated to prompt greater attention to more reliable sources
(Goldman et al., 2012). Therefore, we presented the sources as
reliable experts in order to create as similar as possible reading
conditions across all blog-posts. Such a design also recreates one of
the challenges of online learning about controversial issues: eval-
uating expert disagreement. When experts disagree, author exper-
tise and authority are necessary but insufficient criteria for
evaluating and integrating online information sources, and deeper
understandings of authors' perspectives and positions are needed.
The information sources were designed as blog-posts for several
reasons: First, this enabled us to keep the type of source constant
across all documents. Second, blogs are a personal form of
communication that might draw greater attention to the role of the
authors. Third, blogs are an increasingly common form of science
communication (Brumfiel, 2009). To prepare the blog-posts, we
surveyed numerous online information sources about seawater
desalination and identified recurring arguments and key stake-
holders. The information sources we surveyed included official
websites of relevant government agencies, articles in scientific and
professional journals and magazines, online conference pre-
sentations, professional blogs, websites of advocacy groups,
newspaper articles, and teaching resources. Examples of informa-
tion sources are provided in Appendix A. The blog-posts were based
on arguments that recurred in the online information sources we
found, were central to understanding the issue of desalination, and
represented the opinions of actual stakeholders. Content validity of
the texts was examined by two content experts.
The authors of the blogs were presented as experts in their field,
holding a doctoral degree, and working as consultants to various
government agencies and companies that are stakeholders in water
policy. Each text presented one main argument regarding desali-
nation, supported by explanations and data. The blog-posts were of
91
similar length (M ¼ 213.7 words, SD ¼ 9.7) and writing style. The
blog-posts were written in clear language and refrained from
technical jargon.
The four authors were presented as having different profes-
sional backgrounds and different positions regarding the topic of
desalination. Two authors were presented as economists who were
primarily concerned with finding viable economic solutions for
supplying water demands. Two authors were presented as hy-
drologists who were primarily concerned with environmental
preservation. In the conflicting condition, one economist and one
hydrologist raised arguments in support of desalination and one
economist and one hydrologist raised arguments against desali-
nation. In the converging condition, all authors presented argu-
ments in support of desalination. Thus, the conflicting condition
introduced conflicts between the positions of the authors. An
overview of the blog-posts and of the differences between the two
conditions is provided in Table 1. The full texts of blog-posts 1 and 2
and their sources are presented in the online supplement. Addi-
tional materials are available from the authors on request.
In the conflicting condition, participants read the following
blogs: In blog-post 1, an economist affiliated with the Ministry of
Finance argued for desalination on economic grounds: Israel has an
annual water shortage that has caused an increase in water prices.
Recent technological developments have reduced the cost of
seawater desalination and have turned it into an economically
feasible solution. In blog-post 2, a hydrologist associated with the
Water Authority argued for desalination on environmental
grounds: Desalination can halt the excess pumping of groundwater,
which has led to a nearly irreversible salinization and pollution of
groundwater wells and reservoirs and has resulted in environ-
mental damages. In blog-post 3, an economist affiliated with the
Ministry of Environmental Protection argued that desalination is an
expensive and wasteful solution, and that the water shortage could
be better solved by improving water management policy, for
example, by increasing water conservation through raising water
prices and by investing in water reuse. In blog-post 4, a hydrologist
associated with the Nature and Parks Authority argued against
desalination on the grounds that the desalination discharge into
the sea has a high concentration of hot brine that can cause damage
to the marine environment.
In the converging condition, participants read blog-post 1 and
blog-post 2 as well as two additional blog-posts that argued in
support of desalination. In blog-post 5, an economist affiliated with
the Ministry of Infrastructure argued that water management and
water conservation can only partially mitigate the water shortage
and therefore these steps need to be complemented by desalina-
tion. In blog-post 6, a hydrologist associated with the National
Water Company argued that the desalination plants monitor their
impact on the marine environment and that no evidence has been
found for significant negative environmental impact of desalina-
tion, beyond the immediate vicinity of the facilities.
Each text included, from top to bottom: the blog-post title; the
author description, which included the author's name (e.g., Dr.
Rabinovitz), profession (e.g., hydrologist), and affiliation (e.g.,
consultant to the Water Authority); and the body of the text.
Therefore, participants could use both author descriptions and ar-
guments to infer authors' viewpoints.
3.4. Measures
3.4.1. Epistemic thinking assessment
Epistemic thinking was assessed using a scenario-based
epistemic thinking assessment that referred to the desalination
context (Barzilai & Weinstock, 2014, 2015). The assessment is a
topic-specific measure that assesses the three main epistemic
92 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103

Table 1
Blog-post descriptions (differences between conditions are in bold).

# Blog-post title Author description Main claim Position regarding Disciplinary

desalination perspective

Conflicting condition
1 A Look Ahead on
Israel's Water
Economy
2 The Water Reserves
of the State of Israel
3 It's Time to Take
Care of our Water
4 The Implications
of Desalination
Converging Condition
1 A Look Ahead on
Israel's Water
Economy
2 The Water Reserves
of the State of Israel
5 It's Time to Take Care
of our Water
6 The Implications of
Desalination
Dr. Ben-Basat, Economist, Consultant Desalination is economically worthwhile For Economic
to the Ministry of Finance and will help meet water supply needs.
Dr. Rabinovich, Hydrologist, Consultant Desalination will help stop For Environmental
to the Water Authority groundwater pollution.
Dr. Savyon, Economist, Consultant Changing public habits and improving Against Economic
to the Ministry of Environmental water management are viable and less
Protection expensive solutions than desalination.
Dr. Ohana, Hydrologist, Consultant Desalination will cause damage to the Against Environmental
to the Nature and Parks Authority marine environment.
Dr. Ben-Basat, Economist, Consultant Desalination will help lower water prices For Economic
to the Ministry of Finance and meet water supply needs.
Dr. Rabinovich, Hydrologist, Consultant Desalination will help stop For Environmental
to the Water Authority groundwater pollution.
Dr. Savyon, Economist, Consultant to Changing public habits and improving For Economic
the Ministry of Infrastructure water management are insufficient
solutions and need to be complemented
by desalination.
Dr. Ohana, Hydrologist, Consultant to Desalination does not have a negative For Environmental
the National Water Company impact on the marine environment.

perspectives described in Kuhn et al.'s model of epistemic devel-
opment (Kuhn, 1991; Kuhn & Weinstock, 2002) by tapping partic-
ipants' epistemic metacognitive knowledge regarding the nature,
sources, certainty, veracity, justification, and reliability of knowl-
edge. Thus, the assessment refers to multiple epistemic issues
which are viewed as interrelated aspects of learners' “theories-in-
action”. Table 2 provides examples of some of the epistemic di-
mensions assessed by the instrument, their mapping onto the Kuhn
et al. model, and respective items.
In the current study, participants read an introductory para-
graph that described the current state of seawater desalination in
Israel. The introduction concluded by stating that scientists in Israel
have, in recent years, begun to research the environmental impacts
of seawater desalination. After reading the introduction, partici-
pants responded to a series of questions that referred to the study
topic. Each question was followed by three responses that reflect
typical absolutist, multiplist, and evaluativist responses. The par-
ticipants were asked to rate all responses on a ten-point scale that
ranged from 1 (very much disagree) to 10 (very much agree). In the
current study, we employed a 36-item version of the instrument.
All of the items retained for use in the current study are provided in
the online supplement.
To examine construct validity we conducted a principal com-
ponents analysis (PCA) using Varimax rotation. PCA was employed
because it identifies the minimum number of factors that explain
maximum variance (Hair, Black, Babin, Anderson, & Tatham, 2006;
Tabachnick & Fidell, 2014). Varimax rotation was preferred because
it maximizes the variance of the loadings on each factor and sim-
plifies interpretation of the factor structure (Hair et al., 2006;
Tabachnick & Fidell, 2014). Because the instrument was designed
to assess three epistemic perspectives, we chose a priori to extract
three components (Hair et al., 2006). Successive analyses were
conducted in which items were removed when they loaded less
than .32 on one of the components or loaded at .32 or higher on two
or more components (Tabachnick & Fidell, 2014). The clearest so-
lution emerged with 28 items loading on three components that
explained 41.9% of the variance. The Kaiser-Meyer-Olkin measure
indicated good sampling adequacy, KMO ¼ .78; All KMO values for
the individual variables were greater than .62, which is above the
acceptable limit of .50 (Hair et al., 2006). The results of the PCA are
detailed in the online supplement. Internal consistency reliabilities
of the scales were good for absolutism (10 items, a ¼ .82), multi-
plism (8 items, a ¼ .78), and evaluativism (10 items, a ¼ .80). A
score for each perspective was calculated based on the item mean.
Scores ranged from one to ten.
In scoring epistemic perspectives, we chose to retain three
scores per participant rather than assign each participant to a single
category. Individuals may sometimes endorse more than one
epistemic view (Feucht, 2011; King & Kitchener, 1994; Kuhn &
Weinstock, 2002; Schraw, 2013). This may result from the devel-
oping nature of epistemic thinking; that is, learners may be in
transition between perspectives, but it may also reflect more
enduring conflicts and struggles in epistemic views (e.g., Feucht,
2011). Assigning participants to a single epistemic category can
result in loss of data regarding the strength in which they endorse
that epistemic perspective and the degree to which they endorse or
reject alternative perspectives. Although, categorization may be
unavoidable in some cases, the current analysis was more infor-
mative when all three perspectives were retained as continuous
variables.
3.4.2. Author viewpoint identification task
The aim of this task was to assess participants' ability to accu-
rately identify author viewpoints after reading. The task was
inspired by a memory for sources task employed by Braasch et al.
(2012). In the current study, participants were asked to match 16
statements, which represented the bloggers' viewpoints but did not
appear in the blog-posts, to their respective sources. The state-
ments were inferences that reflected the author's positions and
disciplinary perspectives. These included explicit statements of the
authors' viewpoints (e.g., “Seawater desalination is an economically
advantageous solution” [blog-post 1], “Seawater desalination has
environmental benefits” [blog-post 2]), as well as more implicit
statements (e.g., “Seawater desalination will eventually lead to
lowering the water prices” [blog-post 1], “Seawater desalination
will prevent salinization of the water wells” [blog-post 2]). A
unique set of statements was designed for each of the six blogs. In
the conflicting condition, the statements presented claims both for
and against desalination. In the converging condition, all
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
Table 2
Characterization of core epistemic perspectives in the Kuhn et al. model: sample epistemic dimensions and items (based on Barzilai & Weinstock, 2014, 2015).
Absolutism
Source of knowledge In reality, outside the self
Sample item: “The source of knowledge should be only “The source of knowledge should be mainly
“What should be the source of in evidence that can be gathered.”
knowledge of those who
study the effects
of desalination?”
Certainty of knowledge Knowledge is certain./Certainty
is an achievable goal.
Sample item: “Eventually one could know for certain.”
“Can there be certainty about the effects
of desalination?”
Justification of knowing Appeal to reality
Sample item: “The most important thing is to check
“How should one evaluate explanations if the explanation reports exact data
about the effects of desalination?” and not opinions.”
statements expressed positive claims regarding desalination but
offered different grounds for these claims.
Participants read the following instructions: “Below is a list of
sentences about desalination. These sentences were written by the
authors of the blogs you have just read. Please read the sentences
and mark which of the four authors wrote each sentence.” For each
statement, the participants could choose one of the four authors.
The authors were presented by name and the title of their blog-
post, and the participants did not have access to the texts when
performing this task. The viewpoint identification score was based
on the sum of the correct matches of statement to author and could
range from 0 to 16.
3.4.3. Author viewpoint description task
The aim of this task was to assess participants' ability to accu-
rately recall and describe author viewpoints. The task was based on
a task employed by Hobbs and Frost (2003). Participants read the
following instructions: “In this task, you are asked to describe the
purposes and the points of view of the blogs you have read.” Par-
ticipants were then presented with the title and the author
description of each blog-post and asked to answer in writing two
open-ended questions: “What is the blog's purpose?” “What is the
blogger's point of view?” This process was repeated four times,
once for each blog-post. Participants did not have access to the texts
when responding. The first question was intended to probe un-
derstanding of author positions. However, a pilot study indicated
that participants were not sure what “position” meant. Therefore,
we asked about authors' “purposes,” as this term was more easily
understood and elicited descriptions of authors' positions.
Participants' responses to both questions sometimes mentioned
positions and sometimes referred to disciplinary perspectives.
Therefore, the responses to both questions were coded as one unit.
Participants received one point for correctly describing the author's
position, that is, stating if the author is for or against desalination
(e.g., “To encourage desalination” [P36B], “Against desalination”
[P53A]); and one point for correctly describing the author's disci-
plinary perspective by explicitly stating this perspective (e.g., “A
financial perspective only” [P39B], “An ecological point of view”
[P18A]) and/or by mentioning economic or environmental consid-
erations (e.g., “The economic problem caused by water shortage”
[P34B], “Desalination harms nature and animals” [P69A]). The latter
types of responses indicated the specific aspects of the content of
93
Multiplism Evaluativism
In human minds, within the self In human minds, anchored in standards
of what counts as reliable knowledge,
both outside and within the self
“The source of knowledge should be mainly
in peoples' opinions and ideas.” in how people interpret the evidence that
was gathered.”
Knowledge is fundamentally uncertain. Knowledge is fundamentally uncertain
but it is possible to improve the degree
of certainty.
“One could never know for certain because “There is never full certainty, but it is
it is impossible to find out what happened.” possible to improve the degree
of certainty.”
Appeal to personal preferences and Coordination of theory and evidence,
judgments better explanation of data, appeal to
shared norms and standards
“The most important thing is to check “The most important thing is to check if
if the explanation matches the reader's the explanation helps improve
view of the topic.” understanding of what is known about
the topic.”
the blog-posts that were perceived by the participants as indicators
of authors' viewpoints. It should be noted that the assessment only
required evidence for some understanding of authors' disciplinary
perspectives and that we did not examine to what extent partici-
pants fully understand this construct. Participants could receive up
to two points per blog and up to eight points in total. Coding was
conducted by the first author and by a trained research assistant.
The first author and the research assistant independently coded
120 blog descriptions (30 per blog). Inter-rater reliability was
Cohen's Kappa of .75, a good level of agreement (Landis & Koch,
1977).
3.4.4. Author viewpoint evaluation task
The aim of this task was to assess if participants consider author
viewpoints when faced with a need to evaluate information source
reliability. First, participants were presented again with the blog-
posts and asked to evaluate each blog's reliability using scales
based on those used in website credibility studies (Eastin, Yang, &
Nathanson, 2006; Flanagin & Metzger, 2007; Sundar, 1998). Par-
ticipants assessed the extent to which they found the blog as a
whole to be believable, inaccurate (reversed), professional,
balanced, reliable, correct, false (reversed), and trustworthy on a
scale from 1 (very incorrect) to 6 (very correct). Internal consistency
ranged from a ¼ .87 to a ¼ .91 per blog. A blog reliability score was
calculated for each blog, using the mean of the eight items.
Second, participants were asked to provide reliability justifica-
tions for each blog by answering an open question: “Why is the blog
reliable or not reliable, in your opinion?” The participants had ac-
cess to the texts when performing this task in order to enable them
to provide more detailed justifications. These justifications were
coded for mentions of author viewpoints as reliability justifications.
Mentions of author viewpoints were considered in this task to be
any direct reference of author position (e.g., “It refers to only one
single direction that the only solution is desalination” [P15B]), and/or
disciplinary perspective (e.g., “It does not consider the other side of
the coin, which is the damages caused by seawater desalination (on
the ecological side and not the economic side)” [P19A]). We also
coded author viewpoint references that were included in descrip-
tion of intent or motivation (e.g., “The blogger's purpose is to save
money” [P80A], “It seems as if she has green interests” [P15A]). The
participants received one point for each blog evaluation in which
they mentioned the author's viewpoint (position and/or
94 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
disciplinary perspective) and could therefore receive zero to four
points. The first author and the research assistant independently
coded 40 blog evaluations. Inter-rater reliability was Cohen's
kappa ¼ .80.
3.4.5. Argument task
The aim of this task was to assess if and how participants
spontaneously integrate multiple information sources when con-
structing arguments regarding the topic. We employed an open-
ended argument task that presented an integrative question
about seawater desalination that indirectly required participants to
integrate accounts across texts (Rukavina & Daneman, 1996).
Bråten, Anmarkrud, Brandmo, and Strømsø (2014) noted that
integrative short-essay questions are suitable for measuring
multiple-text comprehension because they assess students' abili-
ties to corroborate information from different sources and to reason
about an issue from different perspectives. Furthermore, several
studies have found that argument tasks that require readers to
present and justify a position facilitate multiple document
comprehension (e.g., Bråten & Strømsø, 2009; Hagen, Braasch, &
Bråten, 2014; Le Bigot & Rouet, 2007). The argument writing in-
structions were: “Please write an argument that addresses the
question: Should the State of Israel continue to encourage the
construction of seawater desalination plants? Present your position
on this issue and justify it.” Because our focus in this manuscript is
not on general argumentation skills, we report on only two di-
mensions of the participants' arguments that were chosen as in-
dicators of integration: argument structure and argument sources.
Argument structure reflected the degree to which participants
considered and justified multiple accounts in their arguments. The
number of positions presented and the justification of these posi-
tions have been used as indicators of multiple document compre-
hension in previous studies as well (e.g., Bråten, et al., 2014;
Ferguson et al., 2013). Coding of argument structure in the cur-
rent study was based on criteria that were adapted from Schwarz,
Neuman, Gil, and Ilya (2003):
 No sound argument (0 points) e A claim is not provided or a
claim is made but no justifications are provided for a claim. For
example, “Yes. I think that the State of Israel should build more
seawater desalination plants [claim]” [P5B].
 One-sided argument (1 point) e A single claim is supported by a
reason or a series of reasons. For example, “I think that the State
of Israel should encourage the construction of seawater desali-
nation plants [claim] since I believe that eventually this will
lower the water prices because there will be no water problem
in the State of Israel. Sea water is unlimited. Additionally, the
freshwater sources in the country will not be damaged and it
will damage the environment less [reasons]” [P15A].
 Two-sided argument with partial justification (2 points) e
These arguments considered both pros and cons of desalination
and included a claim and a reservation or a counter-claim. Two-
sided arguments were partially justified when only one of the
sides was supported by a reason or reasons. Although partici-
pants in the converging condition read blog-posts that all
argued for desalination, they could construct two-sided argu-
ments based on various environmental and economic consid-
erations raised in the blog-posts. For example, “Construction of
seawater desalination plants should be encouraged [claim #1].
However, the short- and long-term effects of desalination
should be constantly monitored [claim #2] in order to check
that there is no damage to the marine and land environment
and that this is the optimal solution [reasons for claim #2]”
[P39B].
 Two-sided argument with full justification (3 points) e These
were two-sided arguments in which reasons were provided for
both sides. For example, “The population of the State of Israel is
growing in a very high rate and its natural water resources will
not be sufficient in the future [reasons for claim #1]. Therefore,
constructing desalination plants will ensure a future clean water
supply to the population [claim #1]. However, this does not
mean that desalination should start immediately if it is not
imperative [claim #2], because it is better to learn how to use
the natural resources as well as possible and to save the State a
lot of money [reasons for claim #2]” [P22A].
In addition to coding argument structure, we also examined the
number of information sources on which the argument was based.
This indicator has also been used in several previous studies to assess
integration or synthesis of multiple texts (e.g., Barzilai & Zohar, 2012;
Gil, Bråten, Vidal-Abarca, & Strømsø, 2010; Goldman et al., 2013).
Participants did not typically make explicit references to the blog-
posts in their arguments. Therefore, we created a list of unique ideas
for each blog-post. We identified these ideas in the students' argu-
ments and awarded one point for each blog-post that was referenced.
Thus, participants could receive zero to four points. For example, the
following argument scored two points: “I think that the State of Israel
should encourage the construction of seawater desalination plants
since I believe that eventually this will lower the water prices because
there will be no water problem in the State of Israel [blog-post 1]. Sea
water is unlimited. Additionally, the freshwater sources in the country
will not be damaged and it will damage the environment less [blog-
post 2]” [P15A]. The arguments were coded by the first author and
the research assistant. 40 arguments were independently coded, and
inter-rater reliability of argument structure and argument sources
were Cohen's kappa ¼ .92 and .74, respectively.
3.4.6. Topic interest measure
Participants' interest in the topic of desalination was assessed
using a 10-item questionnaire by Mason, Gava, and Boldrin (2008)
that was translated to Hebrew and adapted to the desalination
topic. Items were scored on a six-point scale (from very much
disagree to very much agree). Internal consistency reliability was
a ¼ .87. The topic interest score was based on the item mean and
ranged from one to six.
3.4.7. Topic knowledge measure
We developed a multiple-choice test composed of 12 items that
related to the claims made in the texts (e.g., “In the past decade
water consumption per capita in Israel has decreased” [incorrect]).
The participants judged if each statement was correct or incorrect
(they could also answer “I don't know”). Content validity of the
measure was examined by two content experts. Testeretest reli-
ability was examined in an independent sample of university stu-
dents (N ¼ 77), with two weeks between test and retest. The
resulting reliability estimate was Pearson's r ¼ .73. The topic
knowledge score was based on the sum of the correct responses,
and ranged from 0 to 12.
3.5. Procedure
Data collection took place in a university computer lab with each
participant working individually on a computer. A research assis-
tant scheduled appointments with the participants, welcomed
them, and seated each participant at a computer. Participants were
randomly assigned to a converging or a conflicting blog-posts
condition based on their order of arrival, balancing gender. All
questionnaires and tasks were completed online using survey
software.
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
Participants first responded to a demographic questionnaire, the
epistemic thinking assessment, and the topic knowledge and topic
interest measures. Epistemic thinking was assessed prior to reading
because we were interested in examining the role of pre-reading
epistemic perspectives in reading processes. The epistemic
thinking assessment might have raised participants' awareness of
the open-ended and complex nature of the topic, and awareness of
complexity may increase readers' intent to seek out information
sources (Porsch & Bromme, 2011; Scharrer, Stadtler, & Bromme,
2014). Nonetheless, the assessment described multiple ways of
reasoning about divergent accounts and did not suggest that one
way is better than another. Hence, we hypothesize that it did not
have an effect on participants' source evaluations.
Before reading the blog-posts, the participants read the
following instructions: “The following pages present four blog-
posts from blogs about water in Israel. Please read each blog-post
from beginning to end. The blog-posts include information that
can help you form your position regarding desalination. After
reading you will be asked to write an argument that addresses the
question: Should the State of Israel continue to encourage the
construction of seawater desalination plants? You will be asked to
present and justify your position.” The blog-posts were presented
consecutively on separate pages and in random order. The partici-
pants read at their own pace. When the participants finished
reading a blog-post they proceeded to the next and could not go
back and re-read previous posts.
The post-reading measures were presented in the following
order: viewpoint identification task, argument task, viewpoint
evaluation task, and viewpoint description task. The viewpoint
identification task was presented first because a pilot study indi-
cated that participants found it difficult to remember the sources
otherwise. This task might have increased participants' attention to
sources. However, the instructions did not explicitly refer to author
viewpoints. The viewpoint evaluation task was presented after the
argument writing task in order to assess if participants spontane-
ously refer to author viewpoints in their reliability evaluations.
Assessment of trustworthiness is based on multiple content and
source-related criteria (e.g., Bråten, Strømsø, et al., 2011; Metzger,
2007), and many participants did not consider author viewpoints
in their evaluations. The viewpoint description task was presented
last because it explicitly asked about author viewpoints. On
average, the entire session lasted about 50 min.
3.6. Data analysis
First, descriptive statistics and correlations were calculated for
all study variables and participants' epistemic perspectives were
compared. Second, structural equation modeling (SEM) with
maximum-likelihood estimation was conducted in Amos 21 in or-
der to model the relations between the variables. Missing variables
were rare and were imputed using regression estimation.
Maximum-likelihood estimation is considered robust when skew-
ness is below 2 and kurtosis is below 7 (Schreiber, 2008). The
skewness of the variables in the study ranged from 0.05 to 1.27, and
the kurtosis ranged from .01 to 2.41, indicating acceptable
normality. Model fit was estimated using three indices: chi-square,
comparative fit index (CFI), and the root mean square error of
approximation (RMSEA). Cut-off values were based on the recom-
mendations of Hair et al. (2006) for samples of N < 250 and fewer
than 12 variables: non-significant Chi-square p-values, CFI  .97,
and RMSEA < .08. Lastly, in order to examine if epistemic per-
spectives moderate the relation between conflicts and author view
point comprehension, we used hierarchical regression analyses to
test the interaction between conflicts and epistemic perspectives
(Aiken & West, 1991; West, Aiken, & Krull, 1996).
95
4. Results
4.1. Preliminary analyses
Descriptive statistics and Pearson correlations between the
study variables among all of the participants are provided in
Table 3. Topic knowledge and topic interest were correlated to each
other but not to any of the other variables. Therefore, these vari-
ables were not included in subsequent analyses. However, it should
be noted that topic knowledge was rather low (M ¼ 4.74 out of 12),
and that topic interest was moderate (M ¼ 4.05 out of 6).
A repeated measures ANOVA, with epistemic perspective as a
within-subjects variable, indicated highly significant differences
between epistemic perspectives, F(2,388) ¼ 424.52, p < .001,
h2p ¼ .72. Pairwise comparisons revealed that absolutism was
significantly higher than both multiplism, p < .001, and evaluati-
vism, p < .001. Evaluativism was significantly higher than multi-
plism, p < .001. Thus, participants endorsed absolutism to the
greatest extent, evaluativism to a lesser extent, and endorsement of
multiplism was relatively low. In order to compare epistemic per-
spectives across the two conditions, we conducted a MANOVA with
condition (conflicting or converging) as a between-subjects vari-
able and the three epistemic perspectives as dependent variables.
Levene's test for equality of variances yielded non-significant re-
sults, indicating homogeneity of variances in the two conditions.
No significant effect of condition on epistemic perspectives was
found, F(3,166) ¼ 1.41, p ¼ .24, h2p ¼ .025. Means and standard de-
viations of the epistemic perspectives in are provided in Table 4.
Performance in the viewpoint identification task was below
mid-scale yet higher than chance performance (M ¼ 6.70 out of 16).
The mean of the perspective description task was slightly above
mid-scale (M ¼ 4.38 out of 8) and indicated that, when prompted,
the participants described about half of the bloggers' positions and
disciplinary perspectives correctly. In the perspective evaluation
task participants spontaneously referred to authors viewpoints in a
little less than one blog out of four on average (M ¼ .85 out of 4). The
mean argument structure score (M ¼ 1.18 out of 3) indicated that
the participants tended to produce more one-sided arguments than
two-sided arguments. Participants referenced close to two blog-
posts in their arguments on average (M ¼ 1.74 out of 4).
Blog reliability was not one of the study variables, yet we
examined this variable in order check the possible impact of
epistemic perspectives on perceived reliability. The mean of the
blog reliability scores was 4.3 out of 6 (SD ¼ .57), indicating that the
participants on the whole viewed the blogs as moderately reliable.
Multiple regression analysis indicated that absolutism was a sig-
nificant positive predictor of blog reliability, b ¼ .18, p ¼ .036;
R2 ¼ .05, F(3,166) ¼ 2.65, p ¼ .051. That is, the more participants
endorsed absolutist views, the more they tended to regard the
blogs as reliable. Multiplism and evaluativism were not significant
predictors of blog reliability, b ¼ .04, .05, respectively.
4.2. Epistemic perspectives and author viewpoint comprehension
The first research question focused on the relation between
learners' absolutist, multiplist, and evaluativist perspectives and
viewpoint comprehension. To examine this question, a hypothe-
sized model was constructed in which author viewpoint compre-
hension was defined as a latent variable and estimated using three
indicators: viewpoint identification, viewpoint description, and
viewpoint evaluation. Absolutism, multiplism, and evaluativism
were added to the model as predictors of author viewpoint
comprehension. See Model A in Fig. 1. Model fit indices indicated
excellent fit, c2 ¼ 4.69, df ¼ 6, p ¼ .584, CFI ¼ 1.00, RMSEA ¼ .00.
Author viewpoint comprehension positively and significantly
96 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103

Table 3
Descriptive statistics and Pearson correlations of all of the study variables among all participants.

Variable 1 2 3 4 5 6 7 8 9 10

1. Topic knowledge e
2. Topic interest .19* e
3. Absolutism .06 .00 e
4. Multiplism .05 .03 .32** e
5. Evaluativism .12 .00 .26** .15 e
6. Viewpoint identification .14 .07 .01 .06 .06 e
7. Viewpoint description .05 .03 .04 .21** .13 .22** e
8. Viewpoint evaluation .04 .10 .12 .21* .10 .18* .31** e
9. Argument structure .00 .06 .03 .05 .18* .03 .27** .17* e
10. Argument sources .00 .08 .09 .15 .05 .09 .26** .23** .41** e
Maximum score 12 6 10 10 10 16 8 4 3 4
M 4.74 4.05 7.51 3.22 6.56 6.70 4.38 .85 1.18 1.74
SD 2.04 .89 1.36 1.47 1.52 2.87 1.71 1.16 .84 .94
Skewness .05 .41 1.19 .71 .41 .44 .08 1.27 .75 .11
Kurtosis .34 .22 2.14 .10 .15 .01 .40 .70 .24 .24
* **
p < .05, p < .01.

predicted viewpoint identification, viewpoint description, and
viewpoint evaluation. Absolutism had a negative effect on author
viewpoint comprehension, b ¼ .34, p ¼ .004, confirming hy-
pothesis H1a. Contrary to our expectations and to hypothesis H1b,
multiplism had a negative effect on author viewpoint compre-
hension, b ¼ .50, p < .001. However, evaluativism did have a
positive effect on author viewpoint comprehension, b ¼ .36,
p ¼ .002, confirming hypothesis H1c.
4.3. Author viewpoint comprehension and integration of multiple
information sources
Our second research question focused on the relation between
epistemic perspectives, author viewpoint comprehension, and
integration of multiple information sources in participants' written
arguments. Information source integration was added as a latent
variable to the model and estimated using the argument structure
and argument sources scores. Epistemic perspectives and author
viewpoint comprehension were hypothesized to predict informa-
tion source integration. See Model B in Fig. 2. The model had good fit
indices, c2 ¼ 17.21, df ¼ 13, p ¼ .190, CFI ¼ .97, RMSEA ¼ .04. Infor-
mation source integration significantly predicted argument struc-
ture and argument sources. Author viewpoint comprehension had a
significant positive effect on information source integration, b ¼ .68,
p < .001, confirming hypothesis H2a. However, multiplism, and
evaluativism did not have significant direct effects on information
source integration, contrary to hypotheses H2b and H2c. Thus, the
impact of learners' epistemic perspectives on information source
integration was mediated by author viewpoint comprehension.
Fig. 2 provides the standardized solution of Model B. Direct and
indirect standardized coefficients are provided in Table 5.
4.4. Impact of conflicts on author viewpoint comprehension
The third research question asked whether epistemic per-
spectives moderate the impact of conflicts between information
sources on author viewpoint comprehension. Because we
Table 4
Means and standard deviations of epistemic perspective variables in the two
conditions.
Variable Converging condition Conflicting condition
M SD M SD
hypothesized that each epistemic perspective will have a different
individual moderating effect we conducted three hierarchical
regression analyses in order to clarify the independent impact of
each moderator on the association between conflicts and view-
point comprehension. The conflicts variable was dummy coded
(converging ¼ 0, conflicting ¼ 1), and the epistemic perspective
variables were standardized before analysis. In all regressions, two
epistemic perspectives (controls) were entered in the first step,
the conflicts variable (predictor) and the third epistemic
perspective (moderator) were entered in the second step, and a
product term of epistemic perspective  conflicts (interaction)
was entered in the third step. The composite score of author
viewpoint comprehension, imputed in Amos using Model A, was
entered as the dependent variable. Results of the three regressions
are presented in Table 6.
The effects of the epistemic perspectives were similar to those
found in the SEM analysis, with the exception that absolutism
emerged as a significant predictor only when multiplism and
evaluativism were controlled for. Entering conflicts in the second
step had a non-significant effect. The first and second steps
explained 14% of the variance in viewpoint comprehension. In the
third step, the interaction of absolutism  conflicts yielded no
significant effect, b ¼ .03, p ¼ .666, suggesting that effect of conflicts
was not moderated by absolutism. However, the interaction effect
of multiplism  conflicts was significant, b ¼ .13, p ¼ .030, and
explained an additional 2% of the variance. The interaction effect of
evaluativism  conflicts was also significant, b ¼ .13, p ¼ .036, and
added 2% to the explained variance. These interactions suggested
that multiplism and evaluativism each individually moderate the
impact of conflicts on viewpoint comprehension.
Simple slope analyses were conducted to explore the nature of
the significant interactions (Aiken & West, 1991); see Fig. 3.
Conflicts had non-significant effects on viewpoint comprehension
in low multiplism (1 SD), b ¼ .05, p ¼ .577, and a significant
positive effect in the high multiplism (þ1 SD), b ¼ .21, p ¼ .018.
Similarly, the impact of conflicts on viewpoint comprehension
was found to be non-significant in low evaluativism (1 SD),
b ¼ .05, p ¼ .565, and significantly positive in the high evalua-
tivism (þ1 SD), b ¼ .21, p ¼ .019. Thus, conflicts had a significant
positive effect on author viewpoint comprehension only in high
levels of multiplism and evaluativism, confirming our hypotheses.
5. Discussion

Absolutism 7.39 1.46 7.64 1.26 The aim of this investigation was to explore the role of learners'
Multiplism 3.12 1.47 3.32 1.47 epistemic perspectives in comprehension of multiple online author
Evaluativism 6.38 1.63 6.74 1.38
viewpoints; to examine the relation between epistemic
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103 97

Fig. 1. Model A. Effect of epistemic perspectives on author viewpoint comprehension (N ¼ 170). Standardized coefficients appear on the single-headed arrows. Correlations appear
on the double-headed arrows. Squared multiple correlations appear above the variables. Bold arrows indicate significant correlations and regression weights.

perspectives, viewpoint comprehension, and information source
integration; and to test if epistemic perspectives moderate the
impact of conflicts between viewpoints on viewpoint comprehen-
sion. The results of the study indicated that epistemic perspectives
indeed had a significant impact on viewpoint comprehension.
Viewpoint comprehension, in turn, was a significant predictor of
integration of multiple information sources in participants' written
arguments. Unexpectedly, no significant direct effect of epistemic
perspectives was found on information source integration. Thus,
the impact of epistemic perspectives on integration of multiple
information sources was mediated by viewpoint comprehension.
Conflicts between information sources were found to improve
viewpoint comprehension only in high levels of multiplism or
evaluativism. The two variables that were controlledd topic
knowledge and topic interestd were not found to be related to
author viewpoint comprehension. This may have been due to the
fact that the texts we designed could be easily understood even
with low prior knowledge and that interest in the topic was
moderately high in our sample. In the next sections, we discuss
possible interpretations of our findings, point to the limitations of
our study, suggest directions for future research, and raise potential
instructional implications.
5.1. The relation between epistemic perspectives and author
viewpoint comprehension
Comprehension of author viewpoints is not an easy task; It re-
quires inferring authors' positions and perspectives from the text
and from source information and evaluating the significance of
authors' viewpoints for understanding the issue at hand. However,
in many circumstances, readers consider source information as
peripheral and do not adequately represent the sources of the in-
formation (Britt et al., 2013). Our findings indicated that learners'
epistemic perspectives can be one of the factors that predict
comprehension of source viewpoints. In what follows, we interpret
this relation by considering how epistemic perspectives may
inform learners' perceptions of the goals of the task, their standards
of task performance, and the epistemic strategies they employ as
they construct a documents model (Bråten, Britt, et al., 2011).
It should be noted that, in the ensuing discussion, epistemic
perspectives are not viewed as fixed categories but rather as
“theories-in-action” that emerge in particular contexts (Kuhn &
Weinstock, 2002). Furthermore, we assume that although most
learners have a predominant perspective they may still consider or
struggle with other perspectives to some extent (King & Kitchener,

Fig. 2. Model B. Relations between epistemic perspectives, author perspective comprehension, and information source integration (N ¼ 170). Standardized coefficients appear on
the single-headed arrows. Correlations appear on the double-headed arrows. Squared multiple correlations appear above the variables. Bold arrows indicate significant correlations
and regression weights.
98 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103

Table 5 information source reliability. This suggests that endorsement of

Standardized direct and indirect effects of epistemic perspectives on author view- absolutist views was related to greater trust in authors' expertise (cf.
point comprehension and information source integration, among all participants.
Kuhn, 1991) and to greater reliance on authors' expertise and cre-
Variable Absolutism Multiplism Evaluativism dentials as evaluation criteria. From an absolutist perspective, the
Direct Indirect Direct Indirect Direct Indirect goal of the task might be perceived as finding the truth of the matter
** *** ** and distinguishing accurate sources and reports from mistaken or
Author viewpoint .31 .50 .36
comprehension biased ones. Standards of content evaluation might be based on the
Information source .17 .21 **
.20 .34 **
.04 .24*** degree to which the document provides accurate and “proven” facts,
integration and standards of source evaluation may rely primarily on source
Viewpoint .09* .14** .10**
expertise as an indicator of whether the source is reliable or not. In
identification
Viewpoint description .18* .29** .21** some circumstances, such an approach can aid comprehension and
Viewpoint evaluation .18* .29** .21** result in reliable outcomes. This is, however, not an optimal
Argument structure .03 .09 .13 approach in cases in which discrepancies between accounts result
Arguments sources .03 .09 .14 from differences in author viewpoints. In such cases, expertise may
* ** ***
p < .05, p < .01, p < .001. be a necessary but insufficient criterion for understanding differ-
ences between sources. When experts evaluate documents they do
1994; Weinstock & Cronin, 2003). Thus the strength in which an not simply accept or reject sources and claims based their trust-
epistemic perspective is endorsed is considered as an indicator of worthiness, but rather have a more subtle understanding of reli-
learners' tendency to adopt that perspective in a particular context. ability that takes into account the ways in which the particular
The findings of this study revealed that endorsement of each points of view of the sources impact the content and quality of their
epistemic perspective uniquely predicted viewpoint comprehen- reports (Britt & Rouet, 2012; Wineburg, 1991).
sion, beyond the contribution of other epistemic perspectives. A Unexpectedly, the results indicated that multiplism was a
possible interpretation of these findings is that the more strongly a negative predictor of viewpoint comprehension, contrary to hy-
particular epistemic perspective is endorsed the greater its relative pothesis H1b. Endorsement of multiplism was substantially lower
contribution to viewpoint comprehension. than endorsement of absolutism and evaluativism, yet even at this
Absolutism was found to be a negative predictor of viewpoint low level of endorsement multiplism had a significant negative
comprehension, in line with hypothesis H1a. This finding is effect. Our original conjecture was that multiplist awareness of the
consistent with the results of previous studies that have demon- constructed and subjective nature of knowledge might lead to more
strated that views of knowledge as factual and certain may hinder attention to the subjectivity of the authors. However, the results
multiple document comprehension (Barzilai & Zohar, 2012; Bråten, suggest that endorsement of multiplist views might lead learners to
Britt, et al., 2011). Absolutism was also a positive predictor of devote their attention to their own personal opinions rather than to

Table 6
Hierarchical regression analyses predicting author viewpoint comprehension.

Step 1 Step 2 Step 3

b SEb b b SEb b b SEb b

Moderator: absolutism
Multiplism .12 .03 .27*** .15 .03 .36*** .15 .03 .36***
Evaluativism .08 .03 .19* .10 .03 .25** .10 .03 .24**
Absolutism .09 .03 .22** .10 .04 .24*
Conflicts .08 .06 .10 .08 .06 .10
Absolutism  conflicts .03 .06 .04
R2 .09 .14 .14
DR2 .05 .00
F(2,167) ¼ 8.66, p < .001 F(4,165) ¼ 6.54, p < .001 F(5,164) ¼ 5.24, p < .001
DF(2,165) ¼ 4.09, p ¼ .018 DF(1,164) ¼ .19, p ¼ .666
Moderator: multiplism
Absolutism .03 .03 .07 .09 .03 .22** .09 .03 .22**
Evaluativism .07 .03 .17* .10 .03 .25** .10 .03 .25**
Multiplism .15 .03 .36*** .22 .05 .51***
Conflicts .08 .06 .10 .08 .06 .10
Multiplism  conflicts .13 .06 .22*
R2 .03 .14 .16
DR2 .11 .02
F(2,167) ¼ 2.28, p ¼ .106 F(4,165) ¼ 6.54, p < .001 F(5,164) ¼ 6.30, p < .001
DF(2,165) ¼ 10.54, p < .001 DF(1,164) ¼ 4.77, p ¼ .030
Moderator: evaluativism
Absolutism .05 .03 .12 .09 .03 .22** .09 .03 .22**
Multiplism .12 .03 .28*** .15 .03 .36*** .15 .03 .36***
Evaluativism .10 .03 .25** .05 .04 .12
Conflicts .08 .06 .10 .08 .06 .09
Evaluativism  conflicts .13 .06 .20*
R2 .07 .14 .16
DR2 .07 .02
F(2,167) ¼ 6.49, p ¼ .002 F(4,165) ¼ 6.54, p < .001 F(5,164) ¼ 6.24, p < .001
DF(2,165) ¼ 6.18, p ¼ .003 DF(1,164) ¼ 4.47, p ¼ .036
Note: The conflicts variable was dummy coded and the epistemic perspective variables were standardized before analysis.
*
p < .05, **p < .01, ***p < .001.
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
Fig. 3. Simple slopes for the interaction effect of multiplist and evaluativist epistemic perspectives and conflicts on viewpoint comprehension.
those of the authors. From a multiplist perspective, the goal of the
task might be perceived as confirming personal beliefs or as
forming a personal opinion. Standards of content evaluation might
be based on the degree to which the document coheres with the
reader's prior knowledge and experience, and standards of source
reliability might be of relatively minor importance. Consequently,
the primary strategy employed might be content evaluation based
on personal knowledge and experience. Arguably, this might be a
viable strategy when prior knowledge and expertise are high. But
when laypeople read discrepant accounts, such an approach can
lead to superficial processing that might result in low compre-
hension (Bråten, Ferguson, Strømsø, et al., 2013).
Thus, the results of our study support findings by Bråten,
Strømsø, and their colleagues that views of the knower as a source
of knowledge and of justification as personal are negative pre-
dictors of multiple document comprehension (Bråten, Ferguson,
Strømsø, et al., 2013; Bråten et al., 2008; Strømsø et al., 2008).
But our findings also suggest that views of the knower as a source of
knowledge and justification may be detrimental when subjective
considerations are paramount, as in the multiplist position. When
views of the knower as a source of knowledge and justification are
balanced by the need to take into account the objective aspects of
knowing, as in the evaluativist position, they might not have a
negative effect on multiple document comprehension.
Indeed, evaluativism was found to be a significant positive
predictor of viewpoint comprehension, in accordance with hy-
pothesis H1c and with the results of Barzilai and Zohar (2012). One
of the key differences between a multiplist and an evaluativist
perspective is recognition of the need to weigh alternative ac-
counts. From an evaluativist perspective, the goal of the task might
be perceived as understanding and evaluating the different points
of view on the matter. Standards of source evaluation might take
into account the positions and perspectives of the authors, and
standards of content evaluation might be based on recognition that
knowledge is constructed within a perspective. This may lead to
placing greater priority on strategies of identifying and evaluating
author viewpoints.
Bråten, Britt, Strømsø, and Rouet (2011) pointed to a theoretical
conundrum in conceptualizing the relation between epistemic
thinking and multiple document comprehension: If at least some of
the strategies involved in document model construction, such as
sourcing and corroboration, are epistemic strategies, then the
“demarcation between epistemic beliefs and mental representa-
tions and processes related to those beliefs” is not clear (p. 66). We
propose that greater clarity on this issue may be achieved by
99
distinguishing between specific cognitive and metacognitive as-
pects of epistemic practices. Sourcing and corroboration, we argue,
are cognitive-level epistemic strategies because their goal is to
make cognitive progress and not to monitor cognitive progress
(Flavell, 1979). Particularly, their objective is ascertaining the
epistemic properties (e.g., reliability and validity) of specific infor-
mation sources and knowledge claims. In contrast, epistemic per-
spectives entail epistemic metacognitive knowledge about the
nature of knowledge and knowing, such as knowledge about peo-
ple as sources of knowledge and knowledge about epistemic stra-
tegies and tasks (Barzilai & Zohar, 2014). This epistemic
metacognitive knowledge can inform and guide the setting of task
goals and standards, and the selection and performance of
epistemic strategies. It may also inform subsequent metacognitive
monitoring and control (in accordance with metacognitive models,
Efklides, 2008; Veenman, 2011).
5.2. Importance of author viewpoint comprehension
Multiple document comprehension research has emphasized
the importance of understanding authors' viewpoints for con-
struction of integrated representations from multiple texts (e.g.,
Britt & Rouet, 2012). Our findings provide unique support for this
claim by demonstrating that viewpoint comprehension is indeed
a strong positive predictor of integration of multiple information
sources in written arguments, in accordance with hypothesis
H2a. Participants who successfully identified, described, and
evaluated author viewpoints were more likely to construct ar-
guments that referred to multiple accounts and that were based
on multiple information sources. Better understanding of au-
thors' positions and perspectives might increase the likelihood
that these would be considered in participants' arguments.
Furthermore, when learners attend to authors' viewpoints, they
may be more likely to experience and represent documents as
social entities that were written by someone for some purpose
(Britt et al., 2013). Viewpoint comprehension can thus aid
learners in constructing detailed intertext models, in which
sources are more richly represented and more meaningfully
linked to the content and to each other. Such integrated repre-
sentations may help readers construct complex arguments that
take into account multiple views of the problem and integrate
information from multiple documents.
Hypotheses H2b and H2c that multiplist and evaluativist per-
spectives will predict information source integration were not
confirmed: The relation between epistemic perspectives and
100 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
information source integration was mediated by viewpoint
comprehension. A possible explanation of this finding is that
argument structure and argument sources are insufficient mea-
sures of integration, and that better and more holistic measures are
needed to capture differences in integration (e.g., Anmarkrud,
McCrudden, Bråten, & Strømsø, 2013). However, this finding also
raises the possibility that when integration is measured separately
from understanding of author viewpoints, viewpoint comprehen-
sion emerges as an important mediator of the effect of epistemic
thinking on integration. According to this explanation, under-
standing of multiple viewpoints underlies the positive relation
between epistemic perspectives and integration of multiple infor-
mation sources.
5.3. The impact of conflicts on author viewpoint comprehension
Previous studies have suggested that conflicts may contribute
to source memory and comprehension (Braasch et al., 2012;
Strømsø et al., 2013). Our findings extend prior research by
demonstrating that contribution of conflicts to source compre-
hension can be moderated by learners' epistemic perspectives. In
line with our hypotheses, conflicts had a significant positive effect
on viewpoint comprehension only in high levels of multiplism or
evaluativism. Learners who highly endorsed multiplist or eval-
uativist perspectives might have been more aware of the sub-
jective and constructed dimensions of knowledge and therefore
contrasts between sources might have drawn their attention to
the roles of the authors. A complementary explanation is that
when learners with low endorsement of multiplism and evalua-
tivism detected conflicts between claims, they were less likely to
attempt to resolve these conflicts by attributing them to differ-
ences between sources and might have attempted to restore
coherence by alternative methods such as ignoring conflicts or
providing explanations for inconsistencies (Stadtler & Bromme,
2014). However, it is important to note that conflicts only
partially mitigated the negative relation between multiplism and
viewpoint comprehension and that the effect sizes of the in-
teractions were small.
As several authors have argued, the development of epistemic
thinking is marked by increased ability to appropriately adapt
epistemic strategies and processes to task contexts and demands
(Barzilai & Zohar, 2012; Bromme, Kienhues, & Stahl, 2008; Elby &
Hammer, 2001; Pieschl et al., 2008). The findings of this study
support this view by demonstrating that endorsement of multiplist
and evaluativist perspectives can be related to greater adaptivity to
contextual variation. Learners who highly endorsed multiplist or
evaluativist perspectives were more likely to make the effort to
infer and evaluate author viewpoints when they perceived conflicts
between information sources. In contrast, learners with low
endorsement of multiplist and evaluativist perspectives did not
adapt their performance. Thus, successful viewpoint comprehen-
sion was found to be related both to individuals' epistemic per-
spectives and to contextual affordances.
5.4. Limitations and future research
There are relatively few prior studies that measured author
viewpoint comprehension; Therefore we explored several assess-
ment methods that had various advantages and drawbacks. The
viewpoint identification task was an indirect assessment that did
not inquire explicitly about author viewpoints but rather examined
if readers can identify statements that express these viewpoints. It
could be argued that such a task tests memory of textual inferences
rather than comprehension of viewpoints. However, comprehen-
sion of author viewpoints is indeed based on inferences from the
text. In future studies, this task might be improved by adding dis-
tractor inferences that do not represent author viewpoints. The
viewpoint evaluation task was another indirect measure of view-
point comprehension. The questions in this task did not inquire
specifically about author viewpoints but left it up to the partici-
pants to decide whether or not to take viewpoints into account in
their reliability justifications. This task could only assess, therefore,
the inclination to seek out and interpret author viewpoints when
faced with a need to evaluate information source reliability. Finally,
the viewpoint description task directly assessed viewpoint
comprehension by asking participants to describe authors' pur-
poses and points of view (Hobbs & Frost, 2003). This task may offer
a more accurate measure of participants' interpretations of authors'
viewpoints but it does not necessarily indicate that participants
spontaneously reflected on these viewpoints while reading. The
finding that all three measures were significant indicators of the
hypothesized latent variable suggested that they assess comple-
mentary aspects of viewpoint comprehension. Future studies
should refine these measures.
Several additional limitations of the study should also be taken
into account and suggest further directions for research: (a) The
study examined the relation between epistemic perspectives and
author viewpoint comprehension in a single topic. Because
epistemic thinking has discipline and topic-specific aspects, more
studies are needed that will examine this relation in additional
contexts; (b) Some of the authenticity of Internet searching and
reading was sacrificed for the sake of tightly controlling the study
materials. The blogs were of the same length and style, lacked
visuals, and were evidently adapted. Additional studies are
needed to explore viewpoint comprehension in more naturalistic
settings and with more varied types of sources; (c) Cognitive
processing was not controlled and it may be argued that differ-
ences in viewpoint comprehension resulted from deeper pro-
cessing levels while reading. Although we cannot rule out this
objection, the corresponding lack of differences in the informa-
tion source integration suggests that processing depth might not
uniformly account for all of our results. In future studies it may be
advisable to add a processing measure as a control; (d) We did
not employ a perspective-taking measure and therefore it might
be suggested that perspective-taking ability could have accounted
for some of our findings. Furthermore, this study raises the pos-
sibility that the development of epistemic thinking may be
related to the development of perspective-taking. These are
intriguing possibilities that deserve further examination; (e)
Participants' prior beliefs about seawater desalination may have
influenced how they processed belief compatible and incompat-
ible texts. In further studies it would be interesting to check how
prior beliefs interact with epistemic perspectives in predicting
viewpoint comprehension; (f) The argument task required the
participants to present and justify a position and this might have
led some participants to refer less to opposing positions (Wolfe &
Britt, 2008). In future studies it may be worthwhile to examine
the relation between viewpoint comprehension and integration
using additional types of tasks; (g) Finally, we were limited to
conducting the study in a single session and this may have
created some effects between tasks.
5.5. Instructional implications
The findings of a single topic-specific study are obviously
insufficient to warrant instructional recommendations. Therefore,
in this section we raise tentative instructional implications that will
require future examination. The results of this study suggest that
understanding author viewpoints is critical for multiple document
comprehension, but that not all learners may be equally attentive to
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
authors' points of view. Addressing task conditions, by introducing
conflicts between sources, may be helpful for those learners who
already possess some awareness of the constructed nature of
knowledge, but might not be a sufficient instructional solution for
learners who tend to view knowledge as highly objective and
certain.
Therefore, explicit interventions might need to be designed to
address learners' understandings of the construction of knowledge
and, specifically, to foster understanding of the interplay of the
objective and subjective dimensions of knowledge construction
and justification. Tasks that call for comprehension of multiple
conflicting information sources may be highly suitable contexts for
addressing these issues. However, to be effective, such in-
terventions might need to scaffold learners' developing un-
derstandings of knowledge. That is, it may be necessary to address
not only learners' epistemic strategies but also their metacognitive
understandings of the nature of knowledge construction and
justification. This might be done, for example, by comparing
various information sources, identifying differences between them,
and then discussing how and why these differences emerge and
how they can be shaped by authors' viewpoints. When discussing
differences between author viewpoints, specific attention can be
devoted to promoting learners' understanding of the situated as-
pects of these viewpoints, to the ways in which viewpoints reflect
authors' distinct backgrounds, commitments, and motivations, and
to the need to evaluate the impact of authors' viewpoints on their
arguments. Future studies might examine whether better under-
standing of the sources and implications of authors' viewpoints can
indeed improve learners' abilities to critically integrate knowledge
from diverse information sources and their tendencies to adopt
constructivist views of knowledge.
Acknowledgments
This study was funded by a grant to Yoram Eshet-Alkalai and
Sarit Barzilai from the Open University of Israel's research fund.
Additional funding for the study was awarded to Sarit Barzilai by
the I-CORE Program of the Israel Council of Higher Education and
the Israel Science Foundation, grant 1716/12. We are grateful to
Eynav Tzadok for her assistance in data collection and analysis
and to Eyal Rabin and Haggai Kupermintz for their statistical
advice. We also thank Lucia Mason and the anonymous reviewers
for their thoughtful and constructive comments on earlier ver-
sions of this manuscript.
Appendix A. Supplementary data
Supplementary data related to this article can be found at http://
dx.doi.org/10.1016/j.learninstruc.2014.12.003.
References
Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting in-
teractions. Thosand Oaks, CA: Sage.
Alexander, P. A. (2012). Reading into the future: competence for the 21st century.
Educational Psychologist, 47, 259e280. http://dx.doi.org/10.1080/
00461520.2012.722511.
Anmarkrud, Ø., McCrudden, M., Bråten, I., & Strømsø, H. (2013). Task-oriented reading
of multiple documents: online comprehension processes and offline products.
Instructional Science, 41, 873e894. http://dx.doi.org/10.1007/s11251-013-9263-8.
Barzilai, S., & Weinstock, M. (2014). Development and validation of a scenario-based
assessment of epistemic thinking. Paper presented at the Annual Meeting of the
American Educational Research Association (AERA), Philadelphia, Pennsylvania.
Barzilai, S., & Weinstock, M. (2015). Measuring epistemic thinking within and across
topics: A scenario-based approach (Manuscript submitted for publication).
Barzilai, S., & Zohar, A. (2012). Epistemic thinking in action: evaluating and inte-
grating online sources. Cognition and Instruction, 30, 39e85. http://dx.doi.org/
10.1080/07370008.2011.636495.
101
Barzilai, S., & Zohar, A. (2014). Reconsidering personal epistemology as metacog-
nition: a multifaceted approach to the analysis of epistemic thinking. Educa-
tional Psychologist, 49, 13e35. http://dx.doi.org/10.1080/00461520.2013.863265.
Braasch, J. L. G., Rouet, J.-F., Vibert, N., & Britt, M. A. (2012). Readers' use of source
information in text comprehension. Memory & Cognition, 40, 450e465. http://
dx.doi.org/10.3758/s13421-011-0160-6.
Brand-Gruwel, S., & Stadtler, M. (2011). Solving information-based problems:
evaluating sources and information. Learning and Instruction, 21, 175e179.
http://dx.doi.org/10.1016/j.learninstruc.2010.02.008.
Bråten, I., Anmarkrud, Ø., Brandmo, C., & Strømsø, H. I. (2014). Developing and
testing a model of direct and indirect relationships between individual differ-
ences, processing, and multiple-text comprehension. Learning and Instruction,
30, 9e24. http://dx.doi.org/10.1016/j.learninstruc.2013.11.002.
Bråten, I., Britt, M. A., Strømsø, H. I., & Rouet, J.-F. (2011). The role of epistemic
beliefs in the comprehension of multiple expository texts: toward an integrated
model. Educational Psychologist, 46, 48e70. http://dx.doi.org/10.1080/
00461520.2011.538647.
Bråten, I., Ferguson, L., Anmarkrud, Ø., & Strømsø, H. (2013). Prediction of learning
and comprehension when adolescents read multiple texts: the roles of word-
level processing, strategic approach, and reading motivation. Reading and
Writing, 26, 321e348. http://dx.doi.org/10.1007/s11145-012-9371-x.
Bråten, I., Ferguson, L. E., Strømsø, H. I., & Anmarkrud, Ø. (2013). Justification beliefs
and multiple-documents comprehension. European Journal of Psychology of
Education, 28, 879e902. http://dx.doi.org/10.1007/s10212-012-0145-2.
Bråten, I., & Strømsø, H. I. (2009). Effects of task instruction and personal episte-
mology on the understanding of multiple texts about climate change. Discourse
Processes, 47, 1e31. http://dx.doi.org/10.1080/01638530902959646.
Bråten, I., & Strømsø, H. I. (2010). When law students read multiple documents
about global warming: examining the role of topic-specific beliefs about the
nature of knowledge and knowing. Instructional Science, 38, 635e657. http://
dx.doi.org/10.1007/s11251-008-9091-4.
Bråten, I., Strømsø, H. I., & Britt, M. A. (2009). Trust matters: examining the role of
source evaluation in students' construction of meaning within and across multiple
texts. Reading Research Quarterly, 44, 6e28. http://dx.doi.org/10.1598/RRQ.44.1.1.
Bråten, I., Strømsø, H. I., & Salmero n, L. (2011). Trust and mistrust when students
read multiple information sources about climate change. Learning and Instruc-
tion, 21, 180e192. http://dx.doi.org/10.1016/j.learninstruc.2010.02.002.
Bråten, I., Strømsø, H. I., & Samuelstuen, M. S. (2008). Are sophisticated students
always better? the role of topic-specific personal epistemology in the under-
standing of multiple expository texts. Contemporary Educational Psychology, 33,
814e840. http://dx.doi.org/10.1016/j.cedpsych.2008.02.001.
Brem, S. K., Russell, J., & Weems, L. (2001). Science on the web: student evaluations
of scientific arguments. Discourse Processes, 32, 191e213. http://dx.doi.org/
10.1080/0163853X.2001.9651598.
Britt, M. A., & Aglinskas, C. (2002). Improving students' ability to identify and use
source information. Cognition and Instruction, 20, 485e522. http://dx.doi.org/
10.1207/s1532690xci2004_2.
Britt, M. A., & Rouet, J.-F. (2012). Learning with multiple documents: component
skills and their acquisition. In J. R. Kirby, & M. J. Lawson (Eds.), Enhancing the
quality of learning: Dispositions, instruction, and learning processes (pp.
276e314). New York, NY: Cambridge University Press.
Britt, M. A., Rouet, J.-F., & Braasch, J. L. G. (2013). Documents as entities: extending the
situation model theory of comprehension. In M. A. Britt, S. R. Goldman, & J.-
F. Rouet (Eds.), Reading e from words to multiple texts (pp.160e179). New York, NY:
Routledge.
Bromme, R., Kienhues, D., & Stahl, E. (2008). Knowledge and epistemological be-
liefs: an intimate but complicate relationship. In M. S. Khine (Ed.), Knowing,
knowledge and beliefs: Epistemological studies across diverse cultures (pp.
423e441). New York, NY: Springer.
Brumfiel, G. (2009). Science journalism: supplanting the old media? Nature, 458,
274e277. http://dx.doi.org/10.1038/458274a.
Eastin, M. S., Yang, M.-S., & Nathanson, A. I. (2006). Children of the net: an empirical
exploration into the evaluation of internet content. Journal of Broadcasting &
Electronic Media, 50, 211e230. http://dx.doi.org/10.1207/s15506878jobem5002_3.
Efklides, A. (2008). Metacognition: defining its facets and levels of functioning in
relation to self-regulation and co-regulation. European Psychologist, 13,
277e287. http://dx.doi.org/10.1027/1016-9040.13.4.277.
Elby, A., & Hammer, D. (2001). On the substance of a sophisticated epistemology.
Science Education, 85, 554e567. http://dx.doi.org/10.1002/sce.1023.
Eshet-Alkalai, Y., & Chajut, E. (2009). Changes over time in digital literacy. CyberP-
sychology & Behavior, 12, 713e715. http://dx.doi.org/10.1089/cpb.2008.0264.
Eshet-Alkalai, Y., & Geri, N. (2007). Does the medium affect the message? The in-
fluence of text representation format on critical thinking. Human Systems
Management, 26, 269e279.
Ferguson, L. E., Bråten, I., & Strømsø, H. I. (2012). Epistemic cognition when students
read multiple documents containing conflicting scientific evidence: a think-
aloud study. Learning and Instruction, 22, 103e120. http://dx.doi.org/10.1016/
j.learninstruc.2011.08.002.
Ferguson, L. E., Bråten, I., Strømsø, H. I., & Anmarkrud, Ø. (2013). Epistemic beliefs
and comprehension in the context of reading multiple documents: examining
the role of conflict. International Journal of Educational Research, 62, 100e114.
http://dx.doi.org/10.1016/j.ijer.2013.07.001.
Feucht, F. C. (2011). The epistemic underpinnings of Mrs. M's reading lesson on
drawing conclusions. In J. Brownlee, G. Schraw, & D. Berthelsen (Eds.), Personal
epistemology and teacher education (pp. 227e245). New York, NY: Routledge.
102 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
Flanagin, A. J., & Metzger, M. J. (2007). The role of site features, user attributes, and
information verification behaviors on the perceived credibility of web-based
information. New Media & Society, 9, 319e342. http://dx.doi.org/10.1177/
1461444807075015.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: a new area of cog-
nitiveedevelopmental inquiry. American Psychologist, 34, 906e911. http://
dx.doi.org/10.1037/0003-066X.34.10.906.
Gasser, U., Cortesi, S., Malik, M., & Lee, A. (2012). Youth and digital media: From
credibility to information quality. Retrieved from Berkman Center for Internet &
Society website http://ssrn.com/abstract¼2005272.
Gil, L., Bråten, I., Vidal-Abarca, E., & Strømsø, H. I. (2010). Summary versus argument
tasks when working with multiple documents: which is better for whom?
Contemporary Educational Psychology, 35, 157e173. http://dx.doi.org/10.1016/
j.cedpsych.2009.11.002.
Goldman, S. R., Braasch, J. L. G., Wiley, J., Graesser, A. C., & Brodowinska, K. (2012).
Comprehending and learning from internet sources: processing patterns of
better and poorer learners. Reading Research Quarterly, 47, 356e381. http://
dx.doi.org/10.1002/rrq.027.
Goldman, S. R., Lawless, K., & Manning, F. (2013). Research and development of
multiple source comprehension assessment. In M. A. Britt, S. R. Goldman, & J.-
F. Rouet (Eds.), Reading e from words to multiple texts (pp. 160e179). New York,
NY: Routledge.
Goldman, S. R., & Scardamalia, M. (2013). Managing, understanding, applying, and
creating knowledge in the information age: next-generation challenges and
opportunities. Cognition and Instruction, 31, 255e269. http://dx.doi.org/10.1080/
10824669.2013.773217.
Hagen, Å. M., Braasch, J. L. G., & Bråten, I. (2014). Relationships between sponta-
neous note-taking, self-reported strategies and comprehension when reading
multiple texts in different task conditions. Journal of Research in Reading, 37,
141e157. http://dx.doi.org/10.1111/j.1467-9817.2012.01536.x.
Hair, J., Black, W., Babin, B., Anderson, R., & Tatham, R. (2006). Multivariate data
analysis (6th ed.). Upper Saddle River, NJ: Pearson.
Hobbs, R., & Frost, R. (2003). Measuring the acquisition of media-literacy skills.
Reading Research Quarterly, 38, 330e355. http://dx.doi.org/10.1598/rrq.38.3.2.
Hofer, B. K., & Bendixen, L. D. (2012). Personal epistemology: theory, research, and
future directions. In K. R. Harris, S. Graham, T. Urdan, C. B. McCormick,
G. M. Sinatra, & J. Sweller (Eds.), Theories, constructs, and critical issues: Vol. 1.
Apa educational psychology handbook (pp. 227e256). Washington, DC, US:
American Psychological Association.
Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories:
beliefs about knowledge and knowing and their relation to learning. Review of
Educational Research, 67, 88e140. http://dx.doi.org/10.3102/
00346543067001088.
Jucks, R., Becker, B.-M., & Bromme, R. (2008). Lexical entrainment in written
discourse: is experts' word use adapted to the addressee? Discourse Processes,
45, 497e518. http://dx.doi.org/10.1080/01638530802356547.
Jucks, R., & Bromme, R. (2011). Perspective taking in computer-mediated instruc-
tional communication. Journal of Media Psychology: Theories, Methods, and Ap-
plications, 23, 192e199. http://dx.doi.org/10.1027/1864-1105/A000056.
King, P. M., & Kitchener, K. S. (1994). Developing reflective judgment: Understanding
and promoting intellectual growth and critical thinking in adolescents and adults.
San Francisco, CA: Jossey-Bass Publishers.
Kintsch, W. (1988). The role of knowledge in discourse comprehension: a
construction-integration model. Psychological Review, 95, 163e182. http://
dx.doi.org/10.1037/0033-295X.95.2.163.
Kuhn, D. (1991). The skills of argument. New York, NY: Cambridge University Press.
Kuhn, D. (2001). How do people know? Psychological Science, 12, 1e8. http://
dx.doi.org/10.1111/1467-9280.00302.
Kuhn, D., Cheney, R., & Weinstock, M. (2000). The development of epistemological
understanding. Cognitive Development, 15, 309e328. http://dx.doi.org/10.1016/
s0885-2014(00)00030-7.
Kuhn, D., Iordanou, K., Pease, M., & Wirkala, C. (2008). Beyond control of variables:
what needs to develop to achieve skilled scientific thinking? Cognitive Devel-
opment, 23, 435e451. http://dx.doi.org/10.1016/j.cogdev.2008.09.006.
Kuhn, D., & Weinstock, M. (2002). What is epistemological thinking and why does it
matter? In B. K. Hofer, & P. R. Pintrich (Eds.), Personal epistemology: The psy-
chology of beliefs about knowledge and knowing (pp. 121e144) Mahwah, NJ:
Lawrence Erlbaum Associates.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for
categorical data. Biometrics, 33, 159e174. http://dx.doi.org/10.2307/2529310.
Le Bigot, L., & Rouet, J.-F. (2007). The impact of presentation format, task assign-
ment, and prior knowledge on students' comprehension of multiple online
documents. Journal of Literacy Research, 39, 445e470. http://dx.doi.org/10.1080/
10862960701675317.
Leadbeater, B., & Kuhn, D. (1989). Interpreting discrepant narratives: hermeneutics
and adult cognition. In J. D. Sinnott (Ed.), Everyday problem solving: Theory and
applications (pp. 175e190). New York, NY, England: Praeger Publishers.
Leu, D. J., Zawilinski, L., Castek, J., Banerjee, M., Housand, B., Liu, Y., et al. (2007).
What is new about the new literacies of online reading comprehension. In
L. S. Rush, A. J. Eakle, & A. Berger (Eds.), Secondary school literacy: What research
reveals for classroom practices (pp. 37e68). Urbana, IL: National Council of
Teachers of English.
Mason, L., Ariasi, N., & Boldrin, A. (2011). Epistemic beliefs in action: spontaneous
reflections about knowledge and knowing during online information searching
and their influence on learning. Learning and Instruction, 21, 137e151. http://
dx.doi.org/10.1016/j.learninstruc.2010.01.001.
Mason, L., & Boldrin, A. (2008). Epistemic metacognition in the context of infor-
mation searching on the web. In M. S. Khine (Ed.), Knowing, knowledge and
beliefs: Epistemological studies across diverse cultures (pp. 377e404). New York:
Springer.
Mason, L., Gava, M., & Boldrin, A. (2008). On warm conceptual change: the interplay
of text, epistemological beliefs, and topic interest. Journal of Educational Psy-
chology, 100, 291e309. http://dx.doi.org/10.1037/0022-0663.100.2.291.
Metzger, M. J. (2007). Making sense of credibility on the web: models for evaluating
online information and recommendations for future research. Journal of the
American Society for Information Science and Technology, 58, 2078e2091. http://
dx.doi.org/10.1002/asi.20672.
Miller, M., & Boix Mansilla, V. (2004). Thinking across perspectives and disciplines.
In Interdisciplinary studies project, project zero. Cambridge, MA: Harvard Grad-
uate School of Education.
Perfetti, C. A., Rouet, J.-F., & Britt, M. A. (1999). Toward a theory of documents
representation. In H. van Oostendorp, & S. R. Goldman (Eds.), The construction of
mental representations during reading (pp. 99e122). Mahwah, NJ: Lawrence
Erlbaum Associates.
Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion.
In L. Berkowitz (Ed.), Vol. 19. Advances in experimental social psychology (pp.
123e205). Academic Press.
Pieschl, S., Stahl, E., & Bromme, R. (2008). Epistemological beliefs and self-regulated
learning with hypertext. Metacognition and Learning, 3, 17e37. http://dx.doi.org/
10.1007/s11409-007-9008-7.
Pornpitakpan, C. (2004). The persuasiveness of source credibility: a critical review
of five decades' evidence. Journal of Applied Social Psychology, 34, 243e281.
http://dx.doi.org/10.1111/j.1559-1816.2004.tb02547.x.
Porsch, T., & Bromme, R. (2011). Effects of epistemological sensitization on source
choices. Instructional Science, 39, 805e819. http://dx.doi.org/10.1007/s11251-
010-9155-0.
Rouet, J.-F. (2006). The skills of document use: From text comprehension to web-based
learning. Mahwah, NJ: Lawrence Erlbaum Associates.
Rouet, J.-F., & Britt, M. A. (2011). Relevance processes in multiple document
comprehension. In M. T. McCrudden, J. P. Magliano, & G. Schraw (Eds.), Text
relevance and learning from text (pp. 19e52). Charlotte, NC: Information Age
Publishing.
Rouet, J.-F., Favart, M., Britt, M. A., & Perfetti, C. A. (1997). Studying and using
multiple documents in history: effects of discipline expertise. Cognition and
Instruction, 15, 85e106. http://dx.doi.org/10.1207/s1532690xci1501_3.
Rukavina, I., & Daneman, M. (1996). Integration and its effect on acquiring knowl-
edge about competing scientific theories for text. Journal of Educational Psy-
chology, 88, 272e287. http://dx.doi.org/10.1037/0022-0663.88.2.272.
Scharrer, L., Stadtler, M., & Bromme, R. (2014). You'd better ask an expert: mitigating
the comprehensibility effect on laypeople's decisions about science-based
knowledge claims. Applied Cognitive Psychology. http://dx.doi.org/10.1002/
acp.3018. n/aen/a.
Schraw, G. (2013). Conceptual integration and measurement of epistemological and
ontological beliefs in educational research. ISRN Education, 2013. http://
dx.doi.org/10.1155/2013/327680.
Schreiber, J. B. (2008). Core reporting practices in structural equation modeling.
Research in Social and Administrative Pharmacy, 4, 83e97. http://dx.doi.org/
10.1016/j.sapharm.2007.04.003.
Schwarz, B. B., Neuman, Y., Gil, J., & Ilya, M. (2003). Construction of collective and
individual knowledge in argumentative activity. Journal of the Learning Sciences,
12, 219e256. http://dx.doi.org/10.1207/S15327809jls1202_3.
Sperber, D., Clement, F., Heintz, C., Mascaro, O., Mercier, H., Origgi, G., et al. (2010).
Epistemic vigilance. Mind & Language, 25, 359e393. http://dx.doi.org/10.1111/
j.1468-0017.2010.01394.x.
Stadtler, M., & Bromme, R. (2013). Multiple document comprehension: an approach
to public understanding of science. Cognition and Instruction, 31, 122e129.
http://dx.doi.org/10.1080/07370008.2013.771106.
Stadtler, M., & Bromme, R. (2014). The contentesource integration model: a taxo-
nomic description of how readers comprehend conflicting scientific informa-
tion. In D. N. Rapp, & J. Braasch (Eds.), Processing inaccurate information:
Theoretical and applied perspectives from cognitive science and the educational
sciences (pp. 379e402). Cambridge, MA: MIT Press.
Strømsø, H. I., Bråten, I., & Britt, M. A. (2010). Reading multiple texts about climate
change: the relationship between memory for sources and text comprehension.
Learning and Instruction, 20, 192e204. http://dx.doi.org/10.1016/
j.learninstruc.2009.02.001.
Strømsø, H. I., Bråten, I., Britt, M. A., & Ferguson, L. E. (2013). Spontaneous sourcing
among students reading multiple documents. Cognition and Instruction, 31,
176e203. http://dx.doi.org/10.1080/07370008.2013.769994.
Strømsø, H. I., Bråten, I., & Samuelstuen, M. S. (2008). Dimensions of topic-specific
epistemological beliefs as predictors of multiple text understanding. Learning
and Instruction, 18, 513e527. http://dx.doi.org/10.1016/
j.learninstruc.2007.11.001.
Sundar, S. S. (1998). Effect of source attribution on perception of online news
stories. Journalism & Mass Communication Quarterly, 75, 55e68. http://
dx.doi.org/10.1177/107769909807500108.
Tabachnick, B. G., & Fidell, L. S. (2014). Using multivariate statistics (6th ed.). Essex,
England: Pearson.
 S. Barzilai, Y. Eshet-Alkalai / Learning and Instruction 36 (2015) 86e103
Tabak, I., & Weinstock, M. (2008). A sociocultural exploration of epistemological
beliefs. In M. S. Khine (Ed.), Knowing, knowledge and beliefs: Epistemological
studies across diverse cultures (pp. 177e195). New York, NY: Springer.
Veenman, M. V. J. (2011). Learning to self-monitor and self-regulate. In R. E. Mayer,
& P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp.
197e218). New York, NY: Routledge.
Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. A. (2008). Information-problem
solving: a review of problems students encounter and instructional solutions.
Computers in Human Behavior, 24, 623e648. http://dx.doi.org/10.1016/
j.chb.2007.01.030.
Weinstock, M. (2009). Relative expertise in an everyday reasoning task: epistemic
understanding, problem representation, and reasoning competence. Learning
and Individual Differences, 19, 423e434. http://dx.doi.org/10.1016/
j.lindif.2009.03.003.
Weinstock, M. (2011). Knowledge-telling and knowledge-transforming arguments
in mock jurors' verdict justifications. Thinking & Reasoning, 17, 282e314. http://
dx.doi.org/10.1080/13546783.2011.575191.
103
Weinstock,M.,&Cronin,M.A.(2003).Theeverydayproductionofknowledge:individual
differences in epistemological understanding and juror-reasoning skill. Applied
CognitivePsychology,17,161e181.http://dx.doi.org/10.1002/acp.860.
West, S. G., Aiken, L. S., & Krull, J. L. (1996). Experimental personality designs:
analyzing categorical by continuous variable interactions. Journal of Personality,
64, 1e48. http://dx.doi.org/10.1111/j.1467-6494.1996.tb00813.x.
Wiley, J., Goldman, S. R., Graesser, A. C., Sanchez, C. A., Ash, I. K., & Hemmerich, J. A.
(2009). Source evaluation, comprehension, and learning in internet science
inquiry tasks. American Educational Research Journal, 46, 1060e1106. http://
dx.doi.org/10.3102/0002831209333183.
Wineburg, S. S. (1991). Historical problem solving: a study of the cognitive pro-
cesses used in the evaluation of documentary and pictorial evidence. Journal of
Educational Psychology, 83, 73e87. http://dx.doi.org/10.1037//0022-
0663.83.1.73.
Wolfe, C. R., & Britt, M. A. (2008). The locus of the myside bias in written argu-
mentation. Thinking & Reasoning, 14, 1e27. http://dx.doi.org/10.1080/
13546780701527674.