Curator

Critical Questions at a Critical Time: Reections on the Contributions of LSIE to Museum Practices

Dennis Bartels, Robert Semper, and Bronwyn Bevan

Abstract Writing from a science museum perspective, the authors argue that the Learning Science in Informal Environments report arrives at a critical time, when growing policy interest in informal learning environments provides new opportunities for the museum eld but also introduces potential threats to autonomy, diversity, and creativity. The authors explore critical questions raised in the report, including: 1) whether and how we represent our subject matter as cultural elds of practice, as opposed to xed collections of facts and artifacts; 2) how we ensure, at a time of increasing interest and scrutiny from policymakers, that we continue to design for a variety of learning opportunities both across and within our institutions, thus sustaining rich, robust learning for more diverse and inclusive audiences; 3) how we develop better assessment questions, methodologies, and instrumentation that can more effectively address the contributions museums make to local learning ecologies. The authors conclude that, at this juncture, it is just as important for the education research community to learn from the practices of the museum eld, as it is for the museum eld to learn from the research.

For some time, practitioners and scholars have argued that our conclusions about the nature of learning have been prejudiced by the disproportionate number of education studies situated in schools (Bartels and Hein 2003; Hsi et al. 2004; Schauble, Dennis Bartels (dbartels@exploratorium.edu) is executive director of the Exploratorium in San Francisco. Robert Semper (rsemper@exploratorium.edu) is executive associate director of the Exploratorium and director of programs. Bronwyn Bevan (bronwynb@ exploratorium.edu) is director of the NSF-funded Center for Informal Learning and Schools, based at the Exploratorium.
163

164

DENNIS BARTELS ET AL CRITICAL QUESTIONS

Leinhardt, and Martin 1997). Much educational research has looked only at the immediate before, during, and after of classroom activity in order to generalize insights into how people learn. This exclusive focus, it has been argued, has impoverished our conceptions of how people learn (Moll 1990; Nasir, Rosebery, Warren, and Lee 2006). At a policy level, this emphasis has meant that museums have not been seen as important players in local educational landscapes, thus contributing to a more fragmented learning environment in which relevant resources and expertise in one eld are underutilized by educational leaders in another. Much of the research on learning in museumswhich builds on the extant knowledge base about learning and learning researchhas adapted methodologies and instruments developed for classrooms, thus privileging particular forms of knowledge (specically, verbal) that may or may not be most relevant to learning in museums. Learning Science in Informal Environments makes an invaluable contribution to the informal learning eld, and to educational scholarship, by advancing the discussion about what counts as learning. Specically, it presents a broad vision of science learning, validates it in important respects, and raises critical questions, at a critical time, about how museums (and other learning organizations) can best contribute to robust learning ecologies for a diverse and inclusive array of people. In this paper, we discuss the ways in which we think LSIE offers important guidance to the museum eld. We also share concerns and cautions about possible misapplications of the volumes ndings to practice.

A Critical Time The Learning Science in Informal Environments (LSIE) report comes at a moment of growing recognition in policy and research circles that a great deal of learningincluding the development of interest, attitudes, and crucial learning capacitieshappens outside of school hours (Gonzalez, Moll, and Amanti 2005; National Research Council 2007). In the last eight years, the National Science Foundation has made several large-scale, multiyear investments in research and leadership centers dedicated to understanding how science learning spans institutional boundaries and multiple timeframes. Among them are: the LIFE Center; the Center for Inquiry in Science Teaching and Learning; and the Center for Informal Learning and Schools. Similar efforts are occurring in the European Union and in other parts of the world (such as PENCIL, the Permanent European Resource Centre for Informal Learning, in the E.U., and comparable programs in Asia, including South Korea, Malaysia, and Singapore). At NSF, the Education and Human Resources Directorate recently reorganized and combined its programs on research, development, and evaluation of teaching and learning in formal settings with programs addressing teaching and learning in informal settings. All were gathered into one new group, called the Division of Research on Learning in Formal and Informal Settings (DRL). They also issued a report (Friedman 2008) specifying impact categories (such as awareness, knowledge, understanding, engagement, skills, and so on) that they suggested Informal Science Education projects use in

CURATOR 53/2 APRIL 2010

165

assessing impact on various audiences. All proposals to the Informal Science Education program must now show how their projects address these categories. On the positive side, these developments, taken as a whole, reect recognition of the coming of age of informal science research. But along with the car keys comes a range of requirements and expectations that have, at the very least, the potential to restrict autonomy, creativity, and even diversity of programs in informal settings. The history of K-12 schooling, in which standards and assessments have been codied and tied to funding, suggests that these developments constrain risk-taking and lead to a greater level of homogeneity. Many in the informal science eld argue that we have done too little exploratory work to lock down impact categories at this stage (see Bartels and Hein 2003), especially in the absence of coherent theoretical frameworks. We know that a person spends as little as nine percent of his or her waking hours in formal instruction over a lifespan, and less than a tenth of that time (its estimated) in science classes (National Research Council 2009). Yet many people develop strong interests in aspects of science and nature, whether its tinkering with mechanical or electrical systems, bird watching, or following media accounts of issues such as natural gas drilling or medical controversies. Beyond school, as LSIE details, there are important science learning experiences that spark, nurture, and or sustain science interests and activitiesvia TV and new media, on playgrounds, in dinner conversations, on trips to museums or zoos, during time spent at the seashore, and in after-school programs and summer camps, to name a few. LSIE reminds us that informal learning environments

Figure 1. In The Iron Science Teacher (a webcast cookoff) a wad of steel wool goes up in ames in the teachers bid to win the coveted title for cooking up the shows best science activity. Photo by Amy Snyder, 2010 Exploratorium.

166

DENNIS BARTELS ET AL CRITICAL QUESTIONS

such as museums are not representative of distinct kinds of learning per se, but rather of distinct kinds of learning opportunities that are important sources of variation within a larger learning ecology. LSIEs emphasis on ecological perspectives is a vitally important conceptual contribution to the informal education eld. Ecological perspectives conceptualize learning as occurring across multiple, nested and overlapping contexts (Bronfenbrenner 1979; Ceci and Hembrooke 1995; Lerner 2005). They also posit that learning occurs through the combination of the self, the setting, and the learning interaction (Bronfenbrenner and Morris 1998). Learning designs must therefore take into account who, where, and how, as well as before, during, and after. The renewed interest in ecological perspectives (see Bransford et al. 2006; Lee 2008) comes at an exciting moment of profound transformation of the educational landscape: a potentially radically new orientation towards learning, which is anticipated by many to unfold over the next few decades. As universities and schools begin to lose their exclusive grip on the social contract for educationthat is, as society begins to recognize and respond to the notion of the empowered life-long learner accessing a variety of resources, institutions, and experiences over time and across settingsthere is, as LSIE suggests, a pressing need to develop new paradigms and methods that can account for such learning, in ways that can guide both practice and policy.

Critical Questions LSIEs synthesis of the research provides a valuable reference point. But this is just a beginning salvo. We argue that it is now incumbent on the museum eld to engage with the research ndings, perhaps challenge some of them, and identify key questions for practice over the next decade. In this section, we address three specic points raised by the NRC volume, and what they might mean for museums: 1) the cultural and human dimensions of our subject matter, 2) the importance of varied learning opportunities within the science learning ecology, and 3) the validity and relevance of current museum research methods. 1) Cultural and Human Dimensions of Our Subject MatterLSIE emphasizes the many facets of science literacyconcepts, skills, epistemologies, and rsthand investigationsplus the social and cultural nature of science practices. The report argues for the important role that science museums playnot as archives of static knowledge, but as venues in which people encounter science as living, evolving cultural elds of practice. Many would argue that such a view of science is essential to a scientically literate person in the twenty-rst century. Yet people often leave school with a concept of science as xed truths, and they become bewildered when scientic evidence or claims contradict one another, as is common with medical research or with the current public debate about climate change. The public may also have xed or static views of history and art. Informal science environments are three-dimensional, temporal, mediated by live human beings, and frequently tactile, kinesthetic, or visual. By their nature, they have

CURATOR 53/2 APRIL 2010

167

the potential to show science in the making as a uniquely human process of invention that is fallible and open to refutation and revision. Museums can engage visitors with cultural dimensions of science through narrative, by streaming live data or interacting with scientists engaged in eld work, or by incorporating the work of artists and others who can help our audiences perceive and experience new perspectives on scientic phenomena or processes. Providing such context and experience with the nature of knowledge in a given discipline may be a special niche for informal learning environments. Engagement with current science, and science in the making, may be an especially important access route for engaging adults, who LSIE consistently promotes as a primary audience for informal science education. While work with children and youth is vital, it is clear that there are pressing societal issues that must be grappled with today. We cannot wait for the next generation of K-12 students to grow-up and begin to make the kinds of behavioral changes and quality-of-life trade-offs that respond to issues like global climate change. Developing adult audiences will require expanding beyond answer-based representations of science, to more human, cultural, and vibrant representations. Informal science institutions can play a pivotal role in this work. The closest thing we have in the U.S. to an adult science literacy delivery system is the estimated 1,500 informal science learning institutions and the multitude of science media and virtual networks (Inverness Research Associates 1996). LSIE takes us one step closer to conceiving these resources as all one system that contributes to the STEM learning ecology of adults as well as children. Just as LSIE promotes an understanding of science as a cultural practice, it also promotes a view of learning as a cultural process. How the informal learning eld comes to understand LSIEs six strands, described as a tool for organizing and assessing science learning (2009, 4), is crucial with regard to how we engage our audiences in the cultural dimensions of science through the cultural processes of learning. There is a potential for the strands to be misconstrued: isolating one or another of the strands as learning outcomes, backward-designing to achieve that outcome, and ending up with a somewhat dry and uninspiring learning landscape. For example, the only way that we can imagine designing for reecting on science as a way of knowing (strand 4) without providing opportunities to manipulate, test, explore, predict . . . (strand 3) would be to put the learner into the passive position of being told or shown what science as a way of knowing is. But such a position, at least as the totality of the experience, is antithetical to core principles of learning in informal environments, which ascribe high levels of agency to the learner. Although a close reading of the LSIE makes clear that the intention of the consensus panel was to assert that the strands are intertwined, the metaphor of individual threads is unfortunately a reductionist one. A quick or uninformed reading of the volume could lead to simplistic expectations, or efforts to achieve extremely limited learning goals, undercutting the integrated nature of disciplinary experiences in informal settings. The eld must be careful to take a deeply nuanced and sophisticated approach to the strands, always positioning them as existing in relationship to one another. Additionally, while applauding the NRC panel for including discussions of interest, identity, prior knowledge, and cultural variation in its deliberations on learning, we

168

DENNIS BARTELS ET AL CRITICAL QUESTIONS

are concerned with the ways in which identity and interest have been separated out as two strands in parallel construction to the four strands addressing STEM disciplinary knowledge and practices. This, we feel, is a category error. It positions identity and interest as design goals or outputs rather than as mediational means for learning and development. To be clear: We do not question the importance of identity and interest as critical to learning. Interest and identity are critical mediators in allnot just informalsettings. Indeed, it might be argued that a failure to recognize the role that interest, identity, prior knowledge, and cultural variations play in learning has weakened the educative potential of classrooms. Whereas an overemphasis on interest, in the absence of explicating the substance of the interest, has potentially undercut recognition of what informal experiences offer for serious and sustained learning. We see the self (dened as the learner whose identity emerges through developing interests, skills, knowledge, beliefs, cultural and historical experiences, and so on) as both shaping and being shaped by learning experiences in our (and other) settings. Our challenge is to design science learning experiences that take into account the diversity and richness of what learners bring to our museums. As LSIE suggests, we also need to consider the many facets of science and scientic literacy our programs can address. And we need to consider elements of learning designs (materials, facilitation strategies, patterns of interaction, and environmental designs, many specic to museum contexts) that shape the learning experiences. All of these elements occur within a particular historical, economic, and political context that must be taken into account. We are concerned about the inclusion of identity and interest as two strands in a set of strands that

Figure 2. Artist Bernie Lubells wooden contraption of gears, wheels, springs, and dowels captivates a visitor. Photo by Amy Snyder, 2010 Exploratorium.

CURATOR 53/2 APRIL 2010

169

otherwise exclusively address particular aspects of the nature of scientic knowledge and practice, and do not include other critical elements of the learning context such as the ones we have named above. The system of the strands paints an incomplete picture. We are therefore concerned that the strands will be used by the eld in ways that lead to either 1) a continuing overemphasis on interest per se (for example, the much heralded aha or wow factor that the eld has, as yet, largely failed to understand in relationship to a lifelong trajectory of science learning); or 2) to nativist interpretations of identity and interest, which position them as internal, possibly inborn, and that you either have or you dont. In such a view, visitors will be categorized (and possibly treated and evaluated) as either this or that. In contrast, sociocultural perspectives on learning argue that the self (ones identity, which subsumes interest as well as skills, knowledge, beliefs, and so on) is never xed (Holland, Lachicotte, Jr., Skinner, and Cain 1998; Lemke 2001; Nasir, Rosebery, Warren, and Lee, 2006). Nor does it exist independent of interactions and experiences. Earlier frameworks, such as the one developed by the Museum Learning Collaborative (MLC), more appropriately position identity as a critical facet of learning as it is mediated in social contexts (Schauble et al. 1997). The other two facets MLC examined were: 1) designed learning environments and 2) the nature of interactions afforded by the designs, taking into account the identities and motivations of the learners. MLC thus built on socio-cultural theory to position identity and interest as integral toshaping and being shaped bythe larger learning context. However, unlike LSIE, the MLC framework did not address particular facets of disciplinary knowledgesuch as epistemologies, skills, and conceptsthat can be developed in informal settings. A conceptual framework is needed that positions a) facets of, and capacities involved in, science literacy (strands 2-5) with b) developmental mechanisms (like interest, identity, affective-cognitive resources, and cultural toolkits), and c) the particular institutional affordances, including social patterns of interaction and material and cultural tools, inherent to museum settings. 2) Variation within Learning ExperiencesLSIE acknowledges the importance of preserving variation among learning experiences in the STEM learning ecology. The report makes a strong case that learning develops across varied times and settings. But, equally important, how such learning developsthrough what combinations of contingencies, structures, and other forcesis not yet clear, nor is it anticipated that any particular combination of experiences would yield consistent results for everybody. Therefore, just as ecological and cultural variation are important to maintaining rich natural and social systems (McNeely, Sterling, and Mulongoy 2008; Rogoff 2003), we should be wary of over-engineering our designs or limiting experimentation or even focusing on a narrow set of intended outcomes. Variation of goals, strategies, and outcomes within the larger learning ecology will lead to more robust learning systems. At an institutional level of decision-making, we contend that differentiating the levels of audience interaction is vital to making sense of the core ideas and recommendations of the LSIE, particularly its rst recommendation about designing informal

170

DENNIS BARTELS ET AL CRITICAL QUESTIONS

learning experiences with specic learning outcomes in mind. This recommendation, sometimes called backward design in the formal education world (Wiggins and McTighe 2005), is ostensibly straightforward, but we feel it bears further scrutiny and contextualization for the museum eld. As LSIE points out, museum audiences are not monolithic in what they know, care about, or want to do during their museum visits. It is also true that the types of interaction that we have with our audiences differ widelyvarying within a single visit experience, within a visiting group, and across repeated visits. At the Exploratorium, we have identied three levels of interaction: browsers, extended learners, and professional communities.1 The level of programmatic structures, time commitments, and assessment strategies increases across these three levels of engagement. Browsers come in the front door and direct their own learning agenda (within the array of opportunities we have designed) from start to nish. Extended learners sign up for classes or camps or other more structured experiences, which are planned, sequenced, often grouped by age, and facilitated by museum educators. Professional communities include our colleagues in informal and formal science education, museum educators and designers, artists and scientists who engage the public with science, and classroom teachers and district science leaders. These audiences participate in multi-dayoften multi-yearprograms with an express purpose of informing their professional practices. What these distinctions mean in practical terms is that a browsers experience at a given exhibit, which may or may not be facilitated and is largely dependent on the browsers prior experiences and knowledge, is quite different from an extended learners, who is brought to an exhibit by a museum facilitator to notice or experience a particular phenomenon, connecting to what has just happened before and what will soon follow afterwards. All things being equal, a particular learning outcome is more likely in the second than the rst case. A professional audience participant comes to that same exhibit with a more critical and reective eye, not only experiencing the exhibit as a learner, but reecting on the experience and its potential meaning for other learners. Identifying these different use models helps us to be vigilant about designing ecologically rich and varied informal learning environments within an institution that can encompass a wide array of uses and goals. This is the central premise of Frank Oppenheimers comparison of a museum visit to a walk in the woods (Oppenheimer 1982). Children and adults, amateurs and professionals, individuals and peer groups, rsttimers and old-timers experience the walk differently, experience different parts of the woods differently, and experience it differently over repeated visits. A guided walk in the woods, provided to extended learners, has, by denition, different goals and likely outcomes than the browsers or the professional audiences experiences. Museum designers must take all of these audiences into account. Therefore the backward design process espoused by the LSIE needs to be undertaken using a broad conception of who our audiences are, what the nature of their interactions are structured to be, and how the learning environments are used for different purposes. Too quickly embracing a backward design model in an informal environment threatens to lead to narrow or sterile learning designs. This is because, unlike in a classroom, visitors may only have a few seconds with a given concept or idea at a given exhibit. More expansive designs, we argue, can provide

CURATOR 53/2 APRIL 2010

171

greater opportunities to engage a wider array of audiences. The results of such designs, and combinations of designs, clearly become more complicated to evaluate and assess, however. Narrow or reductionist approaches to assessing expansive or holistic experiences will have to contend with signicant ecological validity issues (see next section). 3) Validity and Meaning of Current Research Methods and FindingsA major contribution of LSIE is its acknowledgment of the complexity of assessing learning in informal environments. This is an extremely important caution, especially as the larger

Figure 3. Dr. MegaVolt electries an evening event in the After Dark program, which targets young adults looking for a night out. Photo by Amy Snyder, 2010 Exploratorium.

172

DENNIS BARTELS ET AL CRITICAL QUESTIONS

educational eld begins to de-institutionalize, with increasing attention being paid to non-school learning designs and impacts. As the report points out, the eld as a whole needs to develop a better common understanding and agreement of the range of what we are assessing (interests, capacities, commitments), where we are assessing (on site, at home, online), when we are assessing (during, after, a long time after) and why we are assessing (design improvement, impact study, program design). We argue not for a narrowing of categorieswhich we believe jeopardizes the robustness of local learning ecologies, as has happened in many schoolsbut rather for a more intentional and rational articulation among the various categories. Next, we highlight three issues that we believe are important for developing better assessment tools and methodologies for the museum setting. 1. Methods, Tools, and QuestionsAs discussed above, part of the problem is that most of our methods, instruments, and even questions have been adapted from efforts to address school learning goals, strategies, and outcomes. There is a lack of good tools and techniques to measure learning in informal settings. Like the proverbial drunk who is looking under the lamppost for the keys he lost down the block because thats where the light is, much of the current informal assessment work studies a small subset of learning in informal environments because thats what can be measured. As LSIE acknowledges, measures designed to assess learning in the relative stability of the classroom setting are not applicable to the informal eld. In particular, many of the measures currently used in museums look to verbal self-reports about audience experience. Others study actions or choices during the moment of exhibit or program interaction, but do not take into account what prior experiences have contributed to these actions, or what will ensue from them. Few examine non-verbal instantiations of learning, such as visual thinking or kinesthetic memory or sensory impressions. And hardly any examine how the informal experiences develop over time and relate to other science learning opportunities. But the museum assessment problem is bigger than just needing better tools in our toolbox. The assessment task is made all the more difcult due to the relative lack of research-based knowledge about some of the key modalities of informal learning environments. Fundamentally, we argue that the learning sciences research agenda needs to be expanded if it is to shed light on the major questions that confront informal education program designers. While there is some good literature on intergenerational social learning (Leinhardt, Crowley, and Knutson 2002), age dependent concept development (Confrey 1990), and even place- and object-based learning (Bekerman, Burbules, and Silberman-Keller 2006; Paris 2002), there are fewer contemporary research ndings available concerning text, object and image interaction, the impact of design on engagement, spatial narrative development, the learning affordances of sensory learning experiences, design features of engagement, the layout of spatial learning environments, or the level of coherence appropriate between in-school and out-of-school settings. Such issues require the work of an expanded repertory of research elds, including anthropology, psychology, education, cognition, and systems theory. The Learning Sciences combine such disciplinary repertoires, and there are current efforts to support more direct

CURATOR 53/2 APRIL 2010

173

interaction between these elds of research and informal settings (see Zimmerman and Martell 2008). But there is much work to be done to help orient the research to practice, and we suggest that starting with the work and questions of practitioners may be an important step in this direction. 2. Purpose, Goals, and OutcomesWe question the ways in which assessment and evaluation are currently conceptualized in the informal learning setting. Given the appropriate questions, methodologies, and tools, formative assessment about the relationship between exhibit program design decisions and visitor use or participation patterns is clearly critical during development stages. However, the advantage of doing major summative evaluations of a given effort is less clear to us (at least using current methods). If the purpose is to measure project impact on participants, we feel that the complexity of the conditions, and the relative crudity of the tools, makes the results of some of these studies questionable. If the purpose is to generate ndings to inform project developers for future work, then the process often comes too late in the project, with reports being printed out long after exhibit designers have moved onto new concepts, modalities, or audiences. If the goal is to develop generalizable knowledge that can inuence the eld, then the questions, methods, and tools need to be oriented to allow for the great variety and contingency that characterizes the eld. Finally, although seldom acknowledged, engagement with a specic set of exhibits or programs is typically only one part of an entire ISE experience. For example, visiting an exhibit is part of visiting a science museum, watching an individual television program is part of general home science media experience, and attending an after-school class is part of the school after-school home-student continuum. Studying a component part does not get to the cumulative nature of learning, a frame which may better capture the genuine impact of informal environments on the learner. In short, we nd that summative evaluations at the project level have limited benets relative to their high cost. We suggest that a better use of resources for summative evaluations would be to invest in large scale studies not of localized project results, but of how visiting museums or engaging with media sources more generally impacts individual and group learning trajectories, as well as learning ecologies. The NSF is beginning to support such studies with its program evaluations of ITEST (NSFs Innovative Technology Experiences for Students and Teachers) and AYS (Academies for Young Scientists) programs and other impending studies in the newly restructured ISE program. Such an approach would help to develop not only an evidence base that suggests the potential of learning in museums, but it would more accurately reect ecological perspectives on learning, and thus help to develop a more coherent theoretical approach to understanding learning in informal environments, leading ultimately to better questions, tools, and methods. While this suggestion does not address the ways in which summative evaluations are used for accountability purposes, we have not seen evidence that they are in fact used for accountability. Larger impact studies can create a different kind of accountability, justifying the investment. Formative studies are needed to guide and document the decisions in which the investment is made.

174

DENNIS BARTELS ET AL CRITICAL QUESTIONS

3. Knowledge in and for PracticeAn even bigger issue is the serious disconnect between learning research, assessment, and the world of informal-experience developers. (By developers we mean the practitioner-universe of exhibit designers, media producers, program and curriculum developers, and museum educators). Unlike academic elds such as science or medicine, where research results directly or indirectly affect the practitioners, there is only a very loose coupling between the output of research and evaluation and the practitioners work in informal environments. Scientic research intentionally builds on prior work by scientists, and the eld requires peer review to advance. In medicine, the system reinforces the adoption of research ndings through extensive knowledge transfer and in-service education led by the medical profession. In these elds, the research is led by researcher-practitioners. Formal mechanisms exist to relate the results to practice. The academic model works because people in these elds read papers by their peers and challenge, integrate, and advance that knowledge through their practice. The museum world is more like a craft eld (like teaching, moviemaking, or architecture) than an academic eld. Practitioners in craft elds develop their expertise through apprenticeship, mentorship, trial and error, and peer interaction, not through reading reports, conducting formal research studies, or building frameworks. While an NSF proposal may require front end, formative, and summative evaluation, the fact remains that exhibit developers, program designers, or media producers are often not the ones doing the evaluation or research and are not always reading the papers or the evaluation studies. The coin of the realm for many of them is contained in users emotional experiences, designers instincts, and the reaction of their peers and mentors. If one is trying to inuence a eld, it is not enough to create a learning research agenda or to develop a database of evaluation outcomes if the actual developers are not involved in both the system and the process of developing questions, methods, and tools. Bridging this gap between research, evaluation, and practice requires a considerable effort, not the least because each party comes out of a very different tradition, background and training. Often the desired outcomes and the learning goals expressed by the developers are quite different than those measured by the evaluators or expected by the funders. And often the specic research ndings are not easily generalizable to the practical case at hand. The inclusion of developers and their thinking in the discussions of evaluation frameworks, research agendas, and outcome measures is critical. Developers bring intuitive insights based on years of trial and error to these discussionsa vital fund of knowledge for the eld. It is important to consciously develop research agendas, evaluation strategies, and outcome-language that are attuned to investigating areas of the developers intent. It also means understanding how ideas propagate and evolve through the practitioner community. One way to improve practice is to create opportunities for developers to become more self-reective about their work in the context of learning design (Bevan and Xanthoudaki 2008). In the K-12 world for example, the need to improve teaching practice led to the notion of teachers as researchers, as well as to the classroom-based formative assessment movement. In the museum world, the activity of exhibit prototyping and

CURATOR 53/2 APRIL 2010

175

practitioner discussion is an example of a self-reective practice for exhibit developers. As Oppenheimer noted at an AAM-sponsored evaluation workshop at the Exploratorium in 1978, it is the exhibit developers who need to be out on the exhibit oor evaluating the visitor experience if the exhibits (and the exhibit developers) are to improve in their design. Several current projectsincluding the MAPDD (Museums Afterschool: Principles, Data, and Design) project at the Exploratorium; the New York Hall of Sciences CLUSTER (Collaboration for Leadership in Urban Science Teaching, Evaluation, and Research) with City College of New York; and the nascent Museum Educators Reecting on Practice project at the Lawrence Hall of Scienceuse video documentation of museum teaching and learning interactions to promote reective practices. Such efforts need to be expanded to exhibit and environmental designers in a systematic, eld-wide attempt to build knowledge and practice in the eld, as some projects are beginning to do (within Visitors Studies Association, the Center for the Advancement of Informal Science Education, and others).

Conclusion Learning Science in Informal Environments is a valuable document for many reasons, including its demonstration of the powerful role that museums (among others) play in the educational landscape. Much of the world still thinks of schools when one talks about education. This report goes a long way, at a critical juncture in time, toward

FIgure 4. Sand drains through tiny holes in Patterns in the Sand, part of the Exploratoriums Geometry Playground exhibition. Photo by Amy Snyder, 2010 Exploratorium.

176

DENNIS BARTELS ET AL CRITICAL QUESTIONS

expanding that view. We argue that part of the elds response to LSIE should be to express our increased commitments in the following areas: To engage our audiences more consistently and fully with our subject matter as a cultural and human practice, instead of as static facts and truths. To engage adult audiences in epistemological aspects of the subject matter, in order to support understanding and decision-making in contemporary society. To provide museum visitors with holistic and integrated learning experiences that build on different aspects of the subject matter (as detailed in the strands), as opposed to narrow, one-dimensional, or bite-sized experiences because they are easier to measure. To conceptualize interest and identity as mediational means for learning in a given subject matter, and not as isolated goals or outputs per se. To maintain and expand variation of learning goals, strategies, experiences, and outcomes both across and within institutional settings, as a part of developing robust learning ecologies for a diverse and inclusive array of people. To develop better questions, methodologies, and tools for the museum setting. To consider moving from summative evaluations at the project level to comprehensive studies of learning across time and settings, showing how museum experiences relate to the larger trajectories and lifespans of learners. To root research more rmly in practice, as well as rooting practice in research. The value of the LSIE efforts to synthesize research on informal learning cannot be underestimated. Museum practitioners can nd in this volume extremely important perspectives and ndings related to our work. However, we argue that it is now equally essential for researchers and evaluators to look to the practices of informal educators and designers in order to forge the development of the next generation of research and evaluation practices for informal learning environments. Traditional learning research approaches and instruments are not adequately conceptualizing or capturing learning across settings. Yet museum practitioners are creating moving, motivating, and engaging learning spaces. This wisdom of practice must be leveraged for knowledge and understanding. Seminal works like Lave and Wengers anthropological-like perspectives on situated learning in tailor shops and AA meetings changed the very nature of the questions and methodologies that many were using to understand teaching and learning in the classroom and elsewhere (1991). We think that it is possible that studying learning design decisions, values, and participation patterns in museums, and coming to understand how they relate to the larger learning ecology, could similarly re-orient how learning scientists think about learning designs as well as learning systems and ecologies of opportunities. For the museum eld, as we strive to create compelling learning opportunities for our visitors, we need to develop compelling ways of documenting what they

CURATOR 53/2 APRIL 2010

177

really mean for people in their lives. LSIE provides an important inection point for the eld to articulate the contributions that it seeks to make, the needs it must address, and the questions it needs to ask.

Note 1. These categories reect the investment of time that the audience makes, and not any characterization about the internal state of the visitor, as for example, posited by Falk et al. (2007) in their visitor categories of explorers, facilitators, spiritual pilgrims, and so on.

References Bartels, D. and G. E. Hein. 2003. Learning in settings other than schools. Educational Researcher 32: 3843. Bekerman, Z., N. C. Burbules, and D. Silberman-Keller, eds. 2006. Learning in Places: The Informal Education Reader. New York: Peter Lang Publishing. Bevan, B., and M. Xanthoudaki. 2008. Professional development for museum educators: Unpinning the underpinnings. Journal of Museum Education 33(1): 107120. Bransford, J. D., B. Barron, R. D. Pea, A. Meltzoff, P. Kuhl, and P. Bell. 2006. Foundations and opportunities for an interdisciplinary science of learning. In The Cambridge Handbook of the Learning Sciences, R. K. Sawyer, ed., 1934. New York: Cambridge University Press. Bronfenbrenner, U. 1979. The Ecology of Human Development: Experiments by Nature and Design. Cambridge, MA: Harvard University Press. Bronfenbrenner, U. and P. A. Morris. 1998. The ecology of developmental processes. In Handbook of Child Psychology: Vol 1: Theoretical Models of Human Development, R. M. Lerner and W. Damon, eds., 9931028. Ceci, S. J. and H. A. Hembrooke. 1995. A bioecological model of intellectual development. In Examining Lives in Context: Perspectives on the Ecology of Human Development, P. Moen, G. H. Elder, Jr. and K. Luscher, eds., 303 346. Washington, DC: American Psychological Association. Confrey, J. 1990. A review of the research on student conceptions in mathematics, science, and programming. Review of Research in Education 16: 356. Eisenhart, M., E. Finkel, and S. F. Marion. 1996. Creating the conditions for scientic literacy: A re-examination. American Educational Research Journal 33(2): 261295. Falk, J. H., E. M. Reinhard, C. L. Vernon, K. Bronnenkant, N. L. Deans, and J. E. Heimlich. 2007. Why Zoos and Aquariums Matter: Assessing the Impact of a Visit. Silver Spring, MD: Association of Zoos and Aquariums.

178

DENNIS BARTELS ET AL CRITICAL QUESTIONS

Friedman, A. J. E. ed. 2008. Framework for Evaluating Impacts of Informal Science Education Projects. Washington DC: National Science Foundation. Gonzalez, N., L. C. Moll, and C. Amanti. 2005. Funds of Knowledge: Theorizing Practices in Households and Classrooms. Mahwah, NJ: Lawrence Erlbaum Associates. Holland, D., W. Lachicotte Jr., D. Skinner, and C. Cain. 1998. Identity and Agency in Cultural Worlds. Cambridge: Harvard University Press. Inverness Research Associates. 1996. An Invisible Infrastructure: Institutions of Informal Science Education. Washington, DC: Association of Science-Technology Centers. Lave, J. and E. Wenger. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press. Lee, C. D. 2008. The centrality of culture to the scientic study of learning and development: How an ecological framework in education research facilitates civic responsibility. Education Researcher 37(5): 267279. Leinhardt, G., K. Crowley, and K. Knutson, eds. 2002. Learning Conversations in Museums. Mahwah, NJ: Lawrence Erlbaum Associates. Leinhardt, G., L. Martin, and L. Schauble. 1997. A framework for organizing a cumulative research agenda in informal learning contexts. Journal of Museum Education 22(2 and 3): 38. Lemke, J. L. 2001. Articulating communities: Sociocultural perspectives on science education. Journal of Research in Science Teaching 38(3): 296316. Lerner, R. M. 2005. Promoting positive youth development through community and after-school programs. In Organized Activities as Contexts of Development: Extracurricular Activities, After-school and Community Programs, J. L. Mahoney, R. W. Larson, and J. S. Eccles, eds. Mahwah, NJ: Lawrence Erlbaum Associates. McNeely, J., E. J. Sterling, and J. Mulongoy. 2008. What each of us can do to save biodiversity. In Sustaining Life: How Human Health Depends on Biodiversity, E. Chivian and A. Bernstein, eds., 3750. New York: Oxford University Press. Moll, L. C. and J. B. Greenberg. 1990. Creating zones of possibilities: Combining social contexts for instruction. In Vygotsky and Education: Instructional Implications and Applications of Sociohistorical Psychology, L. C. Moll, ed., 319348. Cambridge: Cambridge University Press. Nasir, N. S., A. S. Rosebery, B. Warren, and C. D. Lee. 2006. Learning as a cultural process: Achieving equity through diversity. In The Cambridge Handbook of the Learning Sciences, R. K. Sawyer, ed., 567580. New York: Cambridge University Press. National Research Council. 2007. Taking Science to School: Learning and Teaching Science in Grades K-8, R. A. Duschl, H. A. Schweingruber, and A. W. Shouse, eds. Committee on Science Learning, Kindergarten through Eighth Grade. Board on Science Education, Center for Education. Division of Behavioral

CURATOR 53/2 APRIL 2010

179

and Social Sciences and Education. Washington, DC: The National Academies Press. . 2009. Learning Science in Informal Environments: People, Places, and Pursuits, P. Bell, B. Lewenstein, A. W. Shouse, and M. A. Feder, eds. Committee on Learning Science in Informal Environments. Board on Science Education, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Oppenheimer, F. 1982. Exploration and Discovery: Acceptance Speech for the AAM Distinguished Service Award. Paper presented at the American Association of Museums. Paris, S. G. ed. 2002. Perspectives on Object-centered Learning in Museums. Mahwah, NJ: Lawrence Erlbaum Associates. Rogoff, B. 2003. The Cultural Nature of Human Development. New York: Oxford University Press. Wiggins, G. and J. McTighe. 2005. Understanding by Design. (Second edition.) Alexandria, VA: Association for Supervision and Curriculum Development. Zimmerman, H. T. and S. T. Martell. 2008. The intersection of museum education and the learning sciences. Journal of Museum Education 33(1): 101106.