Vous êtes sur la page 1sur 165

Designing for Model Progression to Facilitate

Students Science Learning


a,b.

Daner SUN a*, Chee-Kit LOOI b


Learning Sciences Laboratory, National Institute of Education, Singapore
*daner.sun@nie.edu.sg

Abstract: WiMVT (Web-based inquirer with Modeling and Visualization Technology) is


designed as a learning system combining guided inquiry, modeling and visualization with
the social interaction. In the paper, we first present the design rationale of the system, briefly
describe the main functions, then discuss the features supporting model progression in
science learning. Following it, we describe a pilot study of WiMVT implementation in the
secondary science class. The data analysis demonstrates the pedagogical value of WiMVT
on students conceptual understanding, and indicates that appropriate peer feedback can
promote students model elaboration in the modeling activities.
Keywords: WiMVT, model progression, collaboration, peer feedback

Introduction
Due to the learning effectiveness of the models and modeling tools for science education, in
particular, the computer-supported Model-based Science Learning Environment (MbSLE),
a number of researchers have invested great effort in building and implementing MbSLEs in
science learning (e.g. Model-it, ModelingSpace, NetLogo, WISE). Besides modeling tools,
these applications may have design elements such as curriculum materials, proposed
pedagogy (e.g. inquiry, CSCL, model progression) and communicative tools. However, few
of them have been designed with the integration of all these design elements. For example,
most of them are unable to support online collaborative modeling; some of them do not
allow for importing of multiple visual representations; and some of them do not facilitate
model progression in pupils. With the intention of creating an innovative application for
secondary school students to acquire sophisticated understanding of scientific conceptions,
develop critical learning skills, we have developed a web-based science learning
environment named WiMVT (Web-based inquirer with Modeling and Visualization
Technology, http://www.sstlsl-wimvt.sg/wimvt). It is designed as a system in which guided
inquiry, modeling and visualization, and social interaction, are integrated - which is unique
among existing science learning environments (Sun & Looi, 2012).
The work reported in this paper focuses on one of core features of the system: model
progression. In this paper, we first introduce the design rationale and the basic functions of
the system. Then we emphasize the feature of model progression in the system. Finally, we
present some results from a pilot study to illustrate the educational value of the model
progression with WiMVT system. The study is used to answer the research questions below:
1) How to integrate a science learning environment featuring model progression into a real
learning context?
2) What is the learning efficacy of the model progression for students conceptual
understanding in science?
3) Does the peer feedback promote students modeling progression in the WiMVT lessons?

1. The Design Rationale of WiMVT System


The learning efficacy of the model-based inquiry in science education has been
demonstrated in many studies (Schwarz & Gwekwerere, 2007). Its generic learning pattern
can be summarized as question, hypothesis, plan, investigation, model, and conclusion (Bell,
et al., 2009). The pattern serves as the guide for the form of WiMVT inquiry cycle.
Additionally, informed by the principle of POE (Predict-Observe-Explain) adopted in
science class in some Singapore schools (White & Gunstone, 1992), we propose a phase
called Pre-model (to explicate students initial ideas) with the corresponding phase: Model
(to explicate students post ideas) in the inquiry. The design guides students to present their
predictions of the science phenomena before investigation, and then to verify their
predictions through investigation, thus ultimately improving their understanding of the
science phenomena. Hence, the main purpose of embedding model progression by
including Pre-model and Model into guided inquiry is to help students elicit and expose
their prior knowledge through pre-models and to elaborate their models in Model phase
after a series of activities. Afterward students refine their understanding and seek validation
them in the Reflect and Apply phrases. Finally, a revised model-based inquiry cycle
incorporating eight phases is created: Contextualize, Question & Hypothesize (Q&H),
Pre-model, Plan, Investigate, Model, Reflect, and Apply (WiMVT inquiry cycle).
WiMVT is a complicated system, so we employ a standard approach: the Rapid
Application Development (RAD) for the system development in consideration of the
research condition. The development process consists of five short development cycles
which mostly involve: design discussion adopted features development discussion
& usability test redesign. At each stage, consultants and collaborators from different
research including science education, computer technology, and educational technology
areas are invited to give feedback and comments on the design of the system. Subsequent
revisions are made based on the feedback. During the whole process, to verify the validity of
the system at each stage, usability testing is conducted to collect data for revising and
improving the design and development. Up to date, we have finished several usability tests
and two pilot studies. The existing WiMVT system has been revised and improved based on
the usability report and feedback from trial implementation in the pilot school.

2. The Introduction of the WiMVT System


The WiMVT system operates via the Internet and is accessible through a general web
browser. Figure 1shows the work flow of the system as a pedagogical scenario. The number
tags (A-D) corresponds to the four operational procedures of the system, see discussion
below.
A. Establishing the project: the authoring tool in the teacher module supporting the
design of the project1: 1) Present brief project description, learning objectives, and tasks for
Home; 2) Edit content accompanying various types of information (e.g. videos, images,
simulations) for Contextualize; 3) Design questions for Q&H, and assign tasks for Plan,
Pre-model and Model, Reflect, and Apply. 4) Insert simulations together with guided
questions into the Investigate tab where students are required to do certain virtual
experiments. Finally, the teacher configures the students groups in the Group Management
section and assigns the projects to the students.

1
The teacher module consists of six sections: My Profile, My Subjects, Project Management, Solutions Review,
Simulation Library, My Mailbox. The section of Project Management provides an authoring tool for teachers to design
and edit the instructional content.

Figure 1. The work flow of WiMVT system


B. Logging onto account: the student module comprises four functional components:
My profile, My project, Group Management and Mailbox. The general information of the
assigned project can be retrieved in My Project section after the students log into the system
with their accounts and passwords.
C. Conducting inquiry activities: The typical work surface in student module is
illustrated in Figure 2. It is split into four panes: shared workspace which holds the textual
information or tools associated with each phase, status of group members (online students
username is visible at all times), name list of group members, and a chat box.
Inquiry process
Online window

Chat box

Group name list

Shared workspace

Figure 2. The interface of students work session


The process of WiMVT inquiry can be briefly described as below: After accessing the
Home tab, the students are guided to engage in a series of learning activities: students
formalize their hypothesis of the questions in Q&H; create pre-models of scientific concepts
they will learn based on their prior knowledge when in Pre-Model; design the plan in the
Plan and then collect and discuss the data in Investigate. Additionally, they can also engage
in the manipulation of several simulations to do virtual experiments, as well as answer the
guided questions. They are then asked to revise their pre-models through peer review and
discussion in Model, and to reflect upon artefacts being built when getting into the Reflect
phase.
D. Reviewing and commenting on artefacts: The teacher thereby can access the
artefacts and interactions generated by students while navigating in Solutions Review, and
comment on each student or each groups hypotheses, plan, investigation report, pre-models
and models, reflective content, as well as their responses in Apply if any. Thus, the system
supports both formative and summative assessments.

3. Model Progression in the WiMVT System

The model progression is a way to present models in increasing complexity gradually


through expanding the number of components or the levels of relations among variables of
models (Swaak, et al., 1998; Mulder,et al., 2011). As indicated earlier, the design elements
of the system support model progression. Specifically, a sketch tool serves as a
drawing-based modeling tool is designed to assist low-ability students creation of the
models both Pre-model and Model (Lerner, 2007). Compared to the drawing-based tool, the
qualitative modeling tool and quantitative modeling tools in Model provide more
opportunities for students to construct high level scientific models. In the system, when
students define objects and establish relations between variables of a qualitative model, the
modeling functions were mainly executed as an invisible simulation engine for processing
relationships which are specified in the form: If A increases, B increases (Avouris, et al.,
2005). In the quantitative modeling scenario, the relations are established via precise
mathematical forms involving variables. In this way, modeling thereby can be progressive
because the students can start from simple (novice) models to complex (expert) models
using the sketch tool. Otherwise, they can work from more qualitative modeling without
defining formulas and then get into the stage of more quantitative modeling when figuring
out the formulas finally. Moreover, synchronous collaboration in Pre-model and Model is
facilitated via the shared workspace and a chatting tool. It means that students can
co-construct a model in real-time, and then modify and elaborate it through online peer
discussion.

4. Research Design and Methods


4.1 Participants
In this study, 46 students from two classes were randomly selected from a junior secondary
school in Singapore. A female physics teacher with 9 - year teaching experience conducted
the class. She had participated in a series of teacher-researcher working sessions of WiMVT
project, and thus had some good understanding of the system. The computer facilities in the
school were excellent, and each student owned and used a Macbook for daily lessons in the
various subjects.
4.2 The design of WiMVT lessons
The WiMVT lessons were co-designed by WiMVT team, science teachers and collaborators.
The classes studied Current Electricity and D.C. Circuit. The topic was divided into 8
50-minute lessons, in which four lessons were incorporated by the system. Table 1
summarizes the lesson flow and relevant information.
The students drew a model of a circuit needed to run a quiz show for 3 teams of
participants, and to point out the direction of the current flow as well in lesson 1. The teacher
reviewed the pre-models and identified the major misconceptions amongst the students.
Students initial ideas of simple circuits then further explored and elicited in lesson 2
through doing hands-on experiments of connecting possible circuits. During lesson 3 and 4,
the students interacted with three levels of PhET simulations, as well as answered guided
questions individually. After obtaining new understanding through investigation, they were
guided to Model phase to elaborate initial models drawn in lesson 1, and to reflect on their
conceptual changes in Reflect. In this pilot study, the qualitative and quantitative modeling
tools are not incorporated into the system. The students mainly worked with the sketch tool
to create models. They were encouraged to provide online peer feedback for models

creation and elaboration within the group members. Before class time, the students were
asked to log onto WiMVT at home to sufficiently familiarize themselves with the system.
The teachers integrated the instructional content in the system and managed the grouping of
the students. As 23 students in each class were divided into 8 groups with heterogeneous,
they mainly worked in triads (one group worked in dyads each class).
4.3 Data source and data analysis
The study aimed to examine the pedagogical value of model progression of the system in
students science learning, as well as the impact of peer feedback on students modeling
performance. We used software to capture the screen activities with the intention of
validating data analysis. Videotaped recordings of the teacher and students interactions
were used to identify patterns of change for triangulation purposes. One audio recorder was
directed at each of 8 groups in both of classes. After all sessions, we interviewed the teacher
and 16 students using a semi-structured interview protocol for approximate 20 minutes. In
data analysis, all videotapes and audios were transcribed to examine students performance
in WiMVT activities. The students peer feedback generated in chatting log during the
modeling process was also saved, identified and transcribed. It was used to investigate the
relation between students model quality and their peer feedback. The results were
subsequently verified by cross-referencing collected data.

5. Results
5.1 The progression of the models quality
We used the quality of models as the indicator for evaluating students modeling
performance. Based on the literature review, we classified the quality of models into three
levels: 1) High Quality Models (HQMs) are the model representations reflecting appropriate
descriptions of science conceptions that involve components with basic properties, and
depicting interactions between variables of components. 2) Medium Quality Models
(MQMs) are the model representing partially accurate descriptions of scientific conceptions,
in which some of appropriate components of models are included in the models. 3) Low
Quality Models (LQMs) refer to the model representations which contain inaccurate
descriptions of all models components, they are usually at the level of the scribble drawing
(Grosslight, et al., 1991). In this study, we collected a total of 11 models in Pre-model phase
and 14 models in Model phase. The outcomes of the evaluation of students models in
Pre-model and Model phases are depicted in Table 12.
Table 1. The number of different models in the Pre-model and Model phases
Quality of models
LQM
MQM
HQM

Pre-model stage
1
5(without current direction)
3(inaccurate current direction)
1( short circuit)
1

Model stage
0
6 (without current direction)
1 (inaccurate current direction)
3 (broken circuit with current directions)
4

Overall, the quality of students pre-models and models were at the level of MQMs,
around 82% and 71% respectively. Specifically, 45% of models in Pre-model phase
presented right representations with components of bulbs, switches and batteries, but failed
2

As the models were the products of groups work, the number of models equals the number of groups.

to define the current flow direction, while the rate decreased to 42% in Model phase. In
comparison with the 3 groups who drew the incorrect current flow direction, only one group
exhibited the same mistake in Model phase. In Model phase, 3 groups defined the right
components of models although they drew the current in the broken circuit, while the
models of 4 groups achieved the level of HQMs. These findings indicated students better
performance as reflected by the better quality of models generated in the Model phase. The
increase in HQMs further demonstrated students progress in the understanding of core
concepts and modeling skills. The students responses on the process of understanding the
circuit in the Reflect phase demonstrated their progressions as well:

Students A: I used to think that short circuits are very complicated, but they are not. In
addition, I thought that parallel circuits have different current for each bulb. But now, I
think that for parallel circuits, the bulbs have the same brightness as the same amount
of current is being flowed through it. Only, when the switch is closed, then the
electrons can starts flowing. Bulbs in series circuits have lower brightness than bulbs in
parallel connection.

Student B: I feel that our design is correct as it is in parallel connection of the bulbs
with a switch connected to it. Closing one switch will cause its corresponding light
bulb to light up.
5.2 The correlation of models quality and peer feedback
Online peer feedback is particularly advantageous, due to the possibility of a less stressful
and intimidating working environment from the lack of face-to-face interaction, which may
promote students to be adventurous and be more involved (Guardado & Shi, 2007). In the
study, students were encouraged to build, revise and elaborate their models through
receiving peer feedback from their team members both in the Pre-Model and Model phases.
The peer feedback coding instrument was developed based on the principles of good
feedback theory and practice (Nicol & Macfarlane-Dick, 2006). It consisted of five
categories: A. task-oriented (clarifies the task specificities), B. knowledge-oriented
(provides necessary information on how to solve a problem), C. strategy-oriented (provides
strategic plans to derive answers in the best way), D. assessment-oriented (provides
constructive comments on the work produced) feedback, and E. affection-oriented
(provides comments with intentions to improve motivations). Here are some examples of
the peer feedback from the transcription of the discourse of a group doing the modeling,
with their coding:

Category A +Category B: you press undo and draw the bulb.

Category B: Just put more batteries to make it (electromagnet) stronger.

Category C: Let us first draw and then think it.

Category D: Actually it is possible. But maybe need more batteries.

Category E + Category D: Nice drawing. I will draw the line.


In the data analysis, the Pearson coefficient was computed to assess the relationship
between model scores and the students peer feedback. Thus, we scored 25 models from the
range of 0 -100 according to the components of models and its relations. The LQM score
was less than 60, the MQM score was between 60 and 80, and the HQM score was between
80 and 100. We calculated the quantity of peer feedback that happened at each group, as
well as the number of each type of peer feedback. The results indicated that there appeared
to be an upward trend, namely, as the amount of peer feedback increases, the higher the
scores of the models drawn. The statistical analysis with the Pearsons r=0.972, p=0 (at the
level of 0.01) reflected a strong positive correlations between students model scores and the

quantity of feedback. Table 2 presents the respective correlation for the five categories of
peer feedback and the model scores.
Table 2. The correlations between peer feedback and models score
Measure
A: Task
B: Knowledge
Pearson Correlations 0.839
0.280
Sig.(2-tailed)
.000
.158
N (Model)
25
25
Correlation is significant at the 0.01 level (2-tailed)

C:Strategy
0.574
.253
25

D:Assessment
0.941
.000
25

E:Affection
-0.739
.261
25

The findings suggested that category A, B, C, D were positively correlated with the
model scores. Significant correlation existed between category D and model scores, r=0.941,
p =.000. Thus, the higher the quantity of category D, the higher the models scores, or vice
versa. There was also significant correlation between category A and the model scores
(r=0.839, p=000). As for knowledge and strategy-oriented peer feedbacks, the correlations
(r=0.28, p = .158; r=0.574, p=.253) for both measures with the model scores are not
significant and they are weakly correlated. This could imply that knowledge and
strategy-oriented feedbacks may be less related to the scoring of the model. Also, the
correlation between affection-oriented peer feedback and scores is negative, r(2) = -.739, p
= .261. We would like to explore these further in future empirical studies. Thus, when
students are working with the modeling tool in the system, it is suggested
assessment-oriented peer feedback and task-oriented feedback be provided by team
members or otherwise, using words like:

Please dont forget that the switch is one of the objects.

There appears to be a problem in that part, do you mind if you check it again?

I think the two objects that you linked up may be incorrect.


5.3 Voices of the teacher and students
The teacher and students expressed an overall positive attitude toward the WiMVT
implementation in the science class. The teacher had a better understanding of the lessons
which could be designed to leverage on the affordances for WiMVT inquiry. She concluded
that 1) lesson plans should be adopted for best fit with WiMVT inquiry and the instruction
should optimize the core features of the system; 2) The explicit inquiry mode was a good
scaffold to guide students learning activities; 3) Students were suggested to do individual
modeling in the Pre-model phase, because they had different initial ideas; 4) In the Model
phase, students were encouraged to converge to a solid understanding whereby they could
present in one consensus model through the co-constructive way. The students agreed on
that the WiMVT learning activities were more interesting and engaging compared what they
had used previously. They pointed out that the small groups collaboration provided more
opportunities to do tasks in the system, and the synchronized collaboration could help them
finish the task faster. The modeling process directly within the system could make drawing
more convenient and less time consuming. They thought that they enhanced their
understanding of electrical circuits bring taught in the lesson through the comparison of
pre-models and models, as well as a reflection phase to concretize the thinking process.

Conclusions
This pilot study on design and implementation of WiMVT lessons addresses the research
question on how to design the WiMVT lessons, and the results demonstrated that WiMVT

exhibited some value in aiding students conceptual understanding. Specifically, the


students model progression can be achieved through the design elements of the
combination of Pre-model and Model phrases in the inquiry. The preliminary finding that
the quantity of peer feedback varies with the quality of the models can help inform the
design of collaboration into such an environment. The students are particularly encouraged
to heed assessment-oriented feedback in the collaborative activities. The teacher and
students voices suggested that more engagement needs more appropriate instructional
support such as guiding students collaboration, scaffolding students modeling process. In
summary, we provide an illustration of the WiMVT system that supports flexible
collaborative students model-based inquiry. We believed that the inquiry with WiMVT
will create unique educational opportunities for students science learning. In the future
work, the investigation of students conceptual understanding, collaborative skills, inquiry
skills and reflective thinking skills will be the main avenues we will pursue with longer-term
and larger scale use of the WiMVT system.

Acknowledgements
This research is supported by the project: Fostering Collaborative Inquiry Modeling and
Visualization Practices in Secondary Science Learning (WiMVT) funded by National
research Foundation in Singapore (Project #: NRF2009-IDM001-MOE-019 project).We
thank Baohui Zhang for his initial conception of and contributions to this work. We would
also like to thank WiMVT team members and our collaborators: Gao Shan, Charles Chen,
Fu Weikai, Evelyn Teo, Karel Mous, Pey Tee Oon, Sze Ghee Tan, and Kin Chuah Chan for
working with us on the project.

References
[1]Avouris, N., Margaritis, M., Komis, V., Saez, A., & Melndez, R. (2005). Modelingspace: interaction
design and architecture of a collaborative modeling enviroment. Presentation for Comodeseminar,
Patras, December 2005.
[2]Bell, T., Urhahne, D., Schanze, S., & Ploetzner, R. (2009). Collaborative inquiry learning: Models, tools,
and challenges. International Journal of Science Education, 32(3), 349-377.
[3]Grosslight, L., Unger, C., Jay, E., & Smith, C., L. (1991). Understanding models and their use in science:
Conceptions of middle and high school students and experts. Journal of Research in Science Teaching,
28(9), 799-822.
[4]Guardado, M., & Shi, L. (2007). ESL students experiences of online peer feedback. Computers and
Composition, 24, 443-461. doi:10.1016/j.compcom. 2007.03.002
[5Lerner, N. (2007). Drawing to learn science: Legacise of agassiz. Journal of Technical Writing and
Communication, 37(4), 379-394.
[6]Mulder, Y. G., Lazonder, A. W., & Jong, T. d. (2011). Comparing two types of model progression in an
inquiry learning environment with modelling facilities. Learning and Instruction, 21(5), 614-624.
[7]Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model
and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
[8]Schwarz, C. V., & Gwekwerere, Y. N. (2007). Using a guided inquiry and modeling instructional
framework (EIMA) to support preservice K-8 science teaching. Science Education, 91(1), 158-186.
[9]Swaak, J., Joolingen, W. R. V., & Jong, T. D. (1998). Supporting simulation-based learning:the effects of
model progression and assignments on definational and intutive Knowledge. Learning and Instruction,
8(3), 235-252.
[10]Sun, D.E, & Looi, C.K. (2012). Designing a web-based science learning environment for model-based
collaborative inquiry. Journal of Science Education and Technology, doi: 10.1007/s10956-012-9377-9.
[11]White, R., & Gunstone, R. (Eds.). (1992). Probing Understanding. London and New York: The Falmer
Press.

Evolution of Literacy in Software Functions by


Creation of Storytelling
a

Isao MIYAJIa*
Faculty of Informatics, Okayama University of Science, Japan
*miyaji@mis.ous.ac.jp

Abstract: "Creation of storytelling" using PowerPoint was conducted so as to raise


computer literacy and to foster the students' self-understanding. They were required to draw
a figure using Excel and stick it on a report. They were expected to write their reports using
Word. The pre and post literacy in functions of the software of three kinds was investigated
to know the computer literacy which students were able to raise by "creating the
storytelling". As a result, the pre literacy in functions of PowerPoint was lower than those of
Word and Excel. After this practice, the post literacy in the functions of PowerPoint
significantly became higher. This paper will report that students became able to utilize the
functions of PowerPoint as same as those of Word and Excel.
Keywords: Storytelling; slide-show storyboards; literacy in software; peer assessment;
self-understanding

Introduction
Creative activities that produce works that inform and entertain people by describing real
and imaginary events, using graphics, narration, and music are called storytelling [1]. In
digital storytelling, still pictures such as photographs, figures, and drawn pictures are
displayed sequentially to create a storytelling and narrated. Digital stories can be easily
reconstructed, and producers can distribute a story to many people through the Internet.
A practice method or a purpose is reported a lot in until now about storytelling [2], [3],
[4], [5], [6]. Sadik calls digital storytelling a meaningful technology-integrated approach
[2]. Effects of creating digital storytelling are reported as follows. Still pictures are easy to
handle for the producers of such assignments, and students can reflect upon memories or
what they have learned through reviewing still pictures. It has been reported that the
practical class improved the quality of their text narratives through the practice of digital
storytelling [7]. It has been reported that storytelling task showed effect of similar patterns
to the other tests in written language comprehension [8].
When persons used WBT as a supplementary material, Hirose et al. have reported that
the person who has weak point awareness toward PC operation used the WBT for the
number of fewer times as a study on the software literacy [9]. In addition, Yamagishi et al.
reported that achievement degree of operation for the applied software is low for persons
who have uneasiness about PC utilization [10].
The literacy of word processor, spreadsheet and presentation becomes required as a
member of society. The literacy of presentation is inferior to the literacy of word processor
and spreadsheet about students in our department. We made a study on designing and
devising a class so that these three kinds of literacy became the same degree. We adopted
evaluation and modified activities through creating storytelling using PowerPoint. We told
meaning to revise it and method to convey oneself idea to the other persons. As a summary

of the classes, students were required to write the report which included consideration for
the contents of the work, self-evaluation, peer-evaluation after filling in them. It was
reported that the blended class which utilized e-learning inside and outside the class was
effective [11]. We conducted the blended class which utilized e-learning under such
contents. We investigated the literacy in the software function in pre and post time to know
the literacy in a computer as an effect of this class. In this paper, we will analyze them and
report knowledge obtained from them.

1.

Instructional Design and Method

The target subject in this study is called an information science experiment, consisting of
three hours per week as one of the compulsory subjects in the second semester at the
department of information science in a university. The content which the author was in
charge of was creating digital storytelling. Students are separated into three groups. Each
group creates the storytelling for four weeks. Each class session was 180 minutes long, and
the class proceeded according to the plan shown in Table 1. The themes of each group are
different as shown in Table 1. Each group carried out the class by four weeks. The first,
second and third groups created each theme of storytelling from 1 to 4, from 5 to 8, and from
9 to 12 weeks respectively. The class was conducted by a teacher and a teaching assistant.
After explaining the activity contents of the day at the beginning of each class, the teachers
walked around the classroom and responded to questions as needed. The number of students
attending a lecture of three groups was 63 persons in total, that is, 21, 22 and 20 persons
respectively.

Week

Table 1 Lesson plans


f i r st
gr oup

Theme
s econd
gr oup

t hi r d
gr oup

Sel f Chi l dr en
under s t a
' s s t or y
3
ndi ng

Fut ur e
cour s e

Exper i ment cont ent s

Di s t r i but ed s ur vey s heet and


del i ver abl e i n t he s es s i on

Submi t t i ng a s t or y,
At t i t ude r el at ed t o abi l i t i es
Cr eat i ng s t or y s l i des
( pr e)
Cr eat i ng s t or y s l i de,
Compl et e s t or y s l i de, Sel f Nar at i ng a s t or y, Sel f eval uat i on 1
eval uat i on
Vi ewi ng 1, Peer
Peer eval uat i on 1, Modi f i ed
eval uat i on 1, Modi f yi ng
s l i des , Sel f -eval uat i on 2,
s l i des , Sel f -eval uat i on 2 Repor t 1
Vi ewi ng 2, Peer
Peer eval uat i on 2, Sel f eval uat i on 2, Sel f eval uat i on 3, Repor t 2, At t i t ude
eval uat i on 3
r el at ed t o abi l i t i es ( pos t )

1.1 Purpose of Classes


One of the purposes of this class is to heighten students' literacy in computer and
self-expression through creative activities. The literacy in computer means that the PC can
be smoothly operated using functions of Word, Excel, Power Point and so on.
Self-understanding and self-analysis are important when searching for jobs. Another
purpose is to deepen self-understanding and a meaning to work through creating a
storytelling on the assigned theme.
1.2 Lesson Plans
The teacher recruited three groups for a class some days ago. Then he distributed an
experiment description document (A4 paper, 31 pages) to the students and explained the

outline of class method. Based on the experiment description document, the teacher
explained the experiments purpose, contents, plan, slide creation method, experimental
method, and related details. He distributed one A4 sheet on which six pictures and the
narrative stories could be entered. Students were assigned to write a story on the right-hand
page and to draw a related picture within the square frame on the left-hand page before the
first session. The illustration and character for an animation are drawn outside of the square
frame. The students were instructed to paint handwritten picture with colored pencils.
The students were instructed to download the story slide of the My course in the
future as an example for their assignment. The teacher explained how to animate slides in
the first session. They subsequently scanned the images of their sheet with an image
scanner. Then they imported the images to Paint software, which were installed as part of
Windows XP Microsoft? suite of files. The students cut the images on Paint software and
pasted them on slides with PowerPoint. The teacher instructed a student who completes the
work and has room at time so as to attach an animation using a function of PowerPoint.
At the beginning of the second session, the teacher explained how to write reports.
They were required to enter the final image works into PowerPoint to create their slides.
They attached an animation to illustrations and characters for deeper understanding for their
stories. Students then recorded their storytelling narrations using a microphone while
viewing their slide show after their works were completed. At the end of the second session,
the students required to submit the file of the storytelling slide.
Students themselves rated their works on an evaluation sheet after completing the
works. At the start of the third session, the teacher printed and distributed a peer evaluation
sheet and a handout in which all works by the students were printed. The slide shows of all
members in the class were sequentially projected on the screen and viewed. Then they were
required to evaluate them for one minute. After the students evaluated each story, the
evaluations were entered into the peer evaluation sheet. After viewing all the story slide
shows, the students entered the rating value and comment to spreadsheet in Excel, and the
files were submitted using the Internet. The teacher gathered and summarized the
evaluations in each students file, then gave each student access to an e-learning portal so
that they could download the peer evaluations. The teacher also pointed out the points that
should be revised in the printed work and distributed instructor feedback to each student.
The students were then required to modify their digital stories and slides by referring to the
peer evaluations and the instructor feedback provided. Students performed the second
self-assessments after modification in the third session. After the correction, they submitted
the file of the story slide. Students themselves rated their works again.
The modified story slides were viewed again in the same manner as during the third
session, and then evaluated once again in the fourth session. The second peer evaluation was
entered into an assessment spreadsheet, and the files were also submitted.
The teacher gathered and summarized the evaluations in each students file, then gave
each student access to an e-learning portal in the same manner as in the third session.
Students then pasted the second peer assessment for themselves on an assessment
spreadsheet. By comparing the first and second evaluations, students could learn from and
interpret the appropriateness of the corrected elements. Students performed the third
self-assessments after the final evaluation in the fourth session.
1.3 Theme of Storytelling
The storytelling of the theme directed was created in this class. Students draw six pieces of
pictures matched with the scene of the story of the theme and create a slide of PowerPoint in
accordance with it. We require students to consider about oneself, to view a work to another
persons, to know the reaction, and to revise a work through creating digital storytelling. A

student talks about oneself creating a slide work, after writing its scenario so that a person
can understand it.
As expression unlike the sentence, a student was made to consider how to draw and
express a picture to supplement the sentence in the story. A student was made to tell one's
thought to a person utilizing a PC, using characteristic of a picture and the narration, and
being made a story plain. A digital picture book completes when a story was narrated using
a microphone so as to promote the understanding of the story contents attaching animation
and to reach the feeling of the contents of the story.
We made the first group create a work according to theme of "self-understanding", the
second group create a work according to theme of "a children's story", and the third group
create a work according to theme of "a future course" as a theme of storytelling. Each group
creates it in four weeks respectively.
A story related to autobiographical topics was made to deepen self-understanding as
the first theme. My hobby, my memories, a childhood dream and so on are shown as
an example. A student was required to create a story which reflects and tells oneself about
contents related to oneself.
A student was required to create the story for children so as to make a student
understand to act for a person as the second theme. "Fantasyland," "a dogs adventure," "an
insects life" and so on are shown as an example. Students were told to create a story that a
child would interest in or that is educational and useful for a child.
A student is made to create a story about one's future to make a student have
consciousness to work after having done a self-analysis enough as the third theme. A job
that I would like to get, workplace where I want to work, my dream job and so on are
shown as an example. The students were requested to think about a future course and to
create the work with an attitude towards work based upon this theme.

2.

Analysis Results and Discussion

Literacy in software functions was investigated before and after the course. Based on the
assessed changes in literacy, the degree of achievement of the objectives of this course was
estimated. In this study, the term literacy represents the skill and ability of use of software
functions shown in Table 5. Hereinafter the existence of significant difference is inferred
using a criterion of a level of significance of 5%.
2.1 Description of Computer Functions in which Literacy is Acquired
Students were advised to describe in a report what they have understood in the course about
the use of a personal computer. They stated that they have learned to use either "narration,
PowerPoint, personal computers, Word, Paint, animation, image scanners, or Excel", as
presented in Table 2. In all, 102 descriptions and an average of 1.65 per person were
obtained. All students described that they gained literacy in at least one function.
2.2 Computer Literacy Investigation Method
Literacy in software functions was investigated as shown in Table 5, to measure computer
literacy more quantitatively than the descriptions written in the report and to ascertain the
details of its improvement. The survey was administered twice, before (at the beginning of
the first lesson) and after (at the closing of the fourth lesson) the course. Students were
advised to fill out a questionnaire that surveys literacy level in the functions of three
application software: PowerPoint, Word, and Excel. Nakamura et al. proposed

measurement of basic knowledge about information technology using questionnaire on


computer technical terms and reported its application to evaluation of information education
that we could know enough the understanding degree easily and in a short time [12].
The questionnaire presented a total of 60 items, respectively including 25, 17, and 18
items on the functions of PowerPoint, Word, and Excel. Responses of 62 students who
answered both before and after the course were used. Literacy levels were classified as "1.
do not know, 2. know the name but cannot use, or 3. can use." Students were asked to assign
an appropriate number from 13 to each item. This rating suggests that the present survey is
based on students' personal assessments.
2.3 Comparison of Literacy for Three Software Applications
Table 3 presents the variance analysis results of average literacy in all items for each of three
software applications, where m, SD, F, p, Cond, Error, and df respectively signify the
average, standard deviation, F value, significance probability, between-groups,
within-groups, and degrees of freedom. The factor of conditions was significant, as evident
in Table 3 (F (5,114) = 34.3, p < 0.001).
Multiple comparisons were conducted according to Tukey's method. The result is
presented in Table 4, where n. s. represents no significant difference, which indicates that
literacy in PowerPoint was significantly lower before the course than that in Word or Excel.
However, literacy in PowerPoint was significantly improved after the course, up to a level at
which no significant difference was found from either Word or Excel. No literacy difference
was observed in Word, although literacy tended to be improved significantly in Excel after
the course compared with before. Consequently, results suggest that this practice brought
about a good effect on literacy not only in PowerPoint but also in Excel.
Table 2 Personal computer
functions for which literacy
is acquired
Literacy contents
N arration
Pow erPoint
PC
W ord
Paint
Anim ation
Im age scanner
Excel
Sum

N o. of
students
29
26
17
8
8
6
6
2
102

Table 3 Result of variance analysis of literacy


in three application software
Pow erP oint
pre
post

SD

SD

W ord

Excel

pre

post
SD

SD

pre

post
SD

SD

2.33
0.67 2.92 0.23 2.82 0.45 2.96 0.15 2.58 0.30 2.93 0.10
S um of square
df
M ean square F
p
C ond Error C ond Error C ond Error value
6.37
4.23
5 114 1.27 0.04 34.3 ***
*** p<.001

Table 4 Result of multiple comparison of literacy for three software applications.


T im e

P re

P ow er
T im e S oftw are
W ord Excel
P oint
P ow erPoint *** ***
***
P re W ord

n.s. n.s.
Excel
n.s.
**
P ost P ow erPoint n.s. **

*** p<.001, ** p<.01, + p<.1

Post
W ord Excel
***
n.s.
**
n.s.

***
n.s.
+
n.s.

2.4 Applications Comparison of Literacy in Each Software Function


Wilcoxons signed-rank test was performed to rank values obtained before and after the
course for each function of the three software applications. The result is shown in Table 5,

where z denotes the Wilcoxon test statistic. A significant difference was observed in 23 of
25 items for PowerPoint. Many students were surprised that a story was presentable in a
slide show with voice narration. Presumably, this is the reason why "11 Record narrations"
is greatly strengthened. Moreover, literacy in items 1525 related to animation was
remarkably improved compared with other items. Students were instructed to assign a
motion to characters, personae, etc. in all six pages. This is regarded as having brought
about a good effect.
No significant difference was found in items "1 Launch PowerPoint" and "8 Execute a
slide show" for PowerPoint. Their respective rankings had been 2.92 and 2.90 before the
course. Improvement was not found after the course because there was little room for
improvement.
Significant difference was found in 10 of 17 items for Word for the reasons given
below. Items "26 Role of windows in Word" and "28 Page layout" were improved because
students were asked to use Word to write a report. Because they were instructed to insert a
table in their reports, items "34 Draw a ruled line" and "35 Erase a ruled line" were
enhanced. Moreover, because they put the graphs for self-evaluation and peer-evaluation or
slides for storytelling in their reports, item "37 Insert an image" was improved. Item "42
Count text characters" cannot be used fully yet, although significant difference was found.
Students were advised to include the number of characters and graphs into a table in their
reports when they were asked to summarize the story contents. Therefore character counting
was explained by necessity. Nevertheless, the story was so short that characters were easily
countable without using the function "Count text characters." Therefore, not all the students
used the function. Significantly different tendencies were found in each of items 32 and 38.
No significant difference was found in five items for Word: items "30 Change font",
"31 Character style", "36 Input characters", "39 Print previews", and "40 Print." They had
already been well understood (ranked as much as almost 3 before the course), so there was
little room for improvement. No significant difference was recognized.
Significant difference was observed in 14 of 18 items for Excel. The reasons are
considered below. Items "48 Arithmetic operation", "49 Copy a formula", and "60
Function" were improved because students were ordered to execute arithmetic operations
and to copy the results to some cells when calculating the average of self-evaluation or
evaluation of others. Items "45 Make a table", "46 Save a table", and "50 Edit a table" were
improved by creating a table with Excel. Item "53 Change the number of digits" was
enhanced because students were ordered to arrange the place after the decimal point when
writing average ranks in a table in the report. When making a table, students were advised to
narrow or extend column width and to center numbers in the case of a numerical column, if
needed, for better appearance. These operations strengthened items "51 Width change of
rows and columns", "52 Centering", and "54 Draw a ruled line." Students used Excel in
various scenarios and tasks, such as placing data into an evaluation sheet, creating a table,
and drawing a graph. For this reason, item "44 Role of window in Excel" was enhanced.
Although there were few occasions to print something using Excel before this practice,
one required part was exclusively printed with a print range specified in this course. This
improved item "55 Designate print range" and "57 Print." When making a report and an
evaluation sheet, students were instructed to perform operations so that a table might be well
understood by everyone. The items strengthened as above suggest that this instruction
enhanced effectiveness. Marginally significant difference was found in two items 47 and 59
for Excel. No significant difference was found in two items for Excel. "43 Launch Excel"
and "58 Make a graph" had already been well understood, ranked as much as almost 3
before the course. Therefore, there was little room for improvement. No significant
difference was found.

Table 5 Survey results of literacy in software functions

Excel

W ord

P ow erPoint

Item s of evaluation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

Launch P ow erP oint


R ole of w indow s in P ow erP oint
Layout of slide
S ize of place holder and change of position
Input of title and change of font size
Input a text
S ave a slide
P ractice of slide show
Interrupt slide show
S et background
R ecord narrations
P rint distributed docum ent
S et slide show
S et effect to change a screen
S et anim ation
M ethod to play back repeatedly
S et special effects in a text
S et special effects to an object
C hange anim ation
P ractice anim ation
C hange a kind of anim ation
C hange the order of anim ations
C hange to distribute tim e of anim ation
D elete anim ation
S et the trace of anim ation
R ole of w indow s in W ord
S ave a docum ent
P age layout
S et letter form at
C hange font
C haracter style, a size, underline
M ake a table
Insert a table
D raw a ruled line
Erase a ruled line
Input characters
Insert an im age
P rint docum ent
P rint preview s
P rint
P ast up of letter
C ount text characters
Launch Excel
R ole of w indow s in Excel
M ake a table
S ave a table
O pen a saved table
A rithm etic operation
C opy a form ula
Edit a table
W idth change of row s and colom ns
C entering
C hange the num ber of digits
D raw a ruled line
D esignate print range
P rint preview
P rint
M ake a graph
Editi a graph
Function

*** p<.001, ** p<.01, * p<.05, + p<.1

Pre
SD

Post
SD

2.9
2.5
2.7
2.1
2.7
2.8
2.9
2.9
2.8
2.6
1.6
2.4
2.5
2.2
2.3
2.0
2.1
2.0
2.2
2.3
2.2
2.2
2.0
2.3
2.0
2.6
2.9
2.7
2.9
2.9
2.9
2.8
2.7
2.8
2.8
2.9
2.8
2.9
2.9
3.0
2.8
2.6
3.0
2.7
2.9
2.9
2.8
2.5
2.5
2.6
2.7
2.8
2.5
2.7
2.7
2.8
2.9
2.7
2.7
2.4

3.0
2.9
3.0
2.5
3.0
3.0
3.0
3.0
2.9
3.0
3.0
2.9
3.0
2.9
3.0
2.8
2.9
2.8
3.0
3.0
3.0
3.0
2.9
3.0
3.0
2.9
3.0
3.0
3.0
3.0
3.0
2.9
2.9
3.0
2.9
3.0
3.0
3.0
3.0
3.0
3.0
2.8
3.0
2.9
3.0
3.0
2.9
2.8
2.9
2.9
2.9
3.0
2.9
2.9
2.9
3.0
3.0
2.8
2.8
2.7

0.4
0.7
0.6
0.8
0.5
0.5
0.4
0.4
0.5
0.6
0.7
0.7
0.6
0.7
0.7
0.7
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.7
0.5
0.5
0.4
0.4
0.3
0.5
0.5
0.5
0.5
0.4
0.5
0.4
0.4
0.3
0.5
0.6
0.3
0.6
0.4
0.4
0.5
0.6
0.6
0.6
0.6
0.5
0.6
0.6
0.6
0.5
0.4
0.6
0.6
0.6

0.0
0.3
0.2
0.7
0.1
0.1
0.1
0.2
0.2
0.2
0.0
0.2
0.2
0.3
0.2
0.5
0.4
0.5
0.1
0.1
0.1
0.1
0.3
0.0
0.3
0.4
0.0
0.2
0.1
0.0
0.2
0.3
0.2
0.2
0.2
0.0
0.1
0.0
0.0
0.0
0.0
0.5
0.0
0.3
0.2
0.2
0.2
0.4
0.4
0.3
0.3
0.0
0.4
0.3
0.4
0.2
0.0
0.4
0.4
0.5

Test

1.6
3.4
3.5
3.1
3.3
3.1
2.0
1.2
2.1
4.4
6.7
4.6
4.9
5.4
5.2
5.4
4.9
4.9
5.5
5.1
5.5
5.4
5.4
4.9
5.6
3.1
2.1
3.2
2.4
1.6
1.2
1.8
2.9
2.6
2.5
1.6
2.6
1.9
1.6
1.0
2.6
2.6
1.0
2.8
2.0
2.2
1.8
4.2
4.2
3.7
2.5
2.4
4.2
3.2
2.7
2.2
2.1
1.2
1.8
2.9

***
***
**
***
**
*

*
***
***
***
***
***
***
***
***
***
***
***
***
***
***
***
***
**
*
**
*

+
**
*
*

**
+

**
*

**
*
*
+
***
***
***
*
*
***
**
**
*
*

+
**

3.

Conclusion

Students were instructed to create slides about storytelling of contents related to their
self-understanding and self-analysis for their career decision, using PowerPoint to
strengthen computer literacy. This article described the practice method and measured
literacy for computer functions before and after the course. Results were reported herein.
Literacy in functions of PowerPoint was lower than that in either Word or Excel before
the course. However, through practice, literacy in functions of PowerPoint was enhanced
significantly. Students reported that they had learned to use PowerPoint as they had Word
and Excel, which suggests that the functions that are necessary for a presentation were more
useful after the practice than before the practice. When making slides, reports, and
evaluation sheets during the creation of storytelling, students were instructed to produce a
work so that it might be well understood by everyone, which seems to have enhanced
effectiveness. Our future plans include analysis of results obtained from an attitude survey
to clarify differences in the effects anticipated from three themes in storytelling.

Acknowledgements
The author appreciates the support of the Grant-in-Aid for Scientific Research, foundation
study (C22500949) provided by the Ministry of Education, Culture, Sports, Science and
Technology, Japan for this research.

References
[1] Susono, H., Simomura, T., Oda, K., Ono, E. (2006). Production of digital storytelling by K-12 and college
students, Proceedings of the 22nd Annual Conference of JSET, pp.911-912.
[2] Sadik, A. (2008). "Digital storytelling: A meaningful technology-integrated approach for engaged student
learning." Educational Technology Research and Development, Vol.56, No.4, pp.487-506.
[3] Zipes, J. (1995). "Creative storytelling: building community, changing lives," New York & London,
Routledge.
[4] Tsou, W., Wang, W., Tzeng, Y. (2006). "Applying a multimedia storytelling website in foreign language
learning", Computers & Education, Vol.47, No.1, pp.17-28.
[5] Kelleher, C., Pausch, R. (2007). "Using storytelling to motivating programming", Communications of the
ACM, Vol.50, No.7, pp.59-64.
[6] Figg, C., McCartney, R., Gonsoulin, W. (2010). "Impacting academic achievement with student learners
teaching digital storytelling to others: The TTTCSE digital video project", Contemporary Issues in
Technology and Teacher Education (CITE Journal), Vol.10, No.1, pp.38-79.
[7] Ballast, K.H., Stephens, L., Radcliffe, R. (2008). "The effect of digital storytelling on sixth grade students'
writing and their attitudes about writing," Proceedings of Society for Information Technology & Teacher
Education International Conference 2008, pp.875-879.
[8] Ku, Y., Chen, Y., Chang, W. (2012). "Development of early literacy skills: The nature and relationalships
between oral and written language comprehension," Proceedings of the 10th Hawaii International
Conference on Education, pp.1-7.
[9] Hirose, H., Yamamoto, Y. (2004). "Relation between individual characteristics concerning personal
computer operation and use frequency of WBT as a supplementary teaching material," Japan Journal of
Educational Technology, Vol.28, No.2, pp.141-147.
[10] Yamagishi, M., Kimoto, M., Meguro, K., et al. (2009). "Consideration of the application software
operation achievement by the change of the PC user's anxiety," Research Report of JSET Conferences,
JSET09-1, pp.347-352.
[11] Miyaji, I. (2011). "Comparison between Effects in Two Blended Classes Which E-learning is Used inside
and outside Classroom," US-China Education Review, USA, Vol.8, No.4, pp.468-481.
[12] Nakamura, Y., Kojima, H., Kikugawa, M. (2003).Measurement of basic knowledge about information
technology using questionnaire on computer technical terms, and its application to evaluation of
information education, Information Processing Society of Japan Journal, Vol.45, No.4, pp.1222-1231.

Comparing Enactments of a Collaborative


Writing Activity in a Networked Language
Learning Classroom
Yun Wen*, Wenli Chen, Chee-Kit Looi
Learning Sciences Lab, National Institute of Education, Singapore
*yun.wen@nie.edu.sg

Abstract: Good learning activity designs do not guarantee effective classroom orchestration
by the teacher. Enactments of the same learning activity design may vary greatly among
different teachers. This study compares two teachers enactments of a collaborative learning
activity in a L2 writing classroom supported by a networked technology called Group
Scribbles (GS). Plausible factors of teachers moves and actions that impact the different
enactments are identified and discussed, including articulating the objective of activity
explicitly, providing improvised formative assessment and scaffolding to support students
work on an ongoing basis, and controlling the tempo of the activity and maintaining
students enthusiasm.
Keywords: Enactment; Collaborative writing; Networked language classroom

1. Introduction
In a computer-supported collaborative learning (CSCL) environment, teachers are required
to shift their role from being a dominator to being facilitator, guiding and helping students
with their learning. Despite the realization that the way teachers enact the instructional
practices is essential for the success of collaborative learning, much research focuses on the
interactions among students when they are engaged in collaborative learning [2, 9].
Relatively less attention is paid to teacher practices in the networked classrooms where
collaborative learning tasks place. A small group of researchers seek to analyze how
teachers create opportunities for student interaction through analyzing teacher discourse [7,
8]. These studies investigate how teacher-led discussions have affected classroom
interactions and identified successful strategies that teachers used in an inquiry classroom
[11]. In addition, findings reveal that enactment styles vary amongst teachers with different
beliefs, pedagogy and content knowledge [3]. Puntambekar et al. [6] claim that few studies
have investigated how difference in enactments of collaborative activities might impact
students learning outcomes (p.82). Enactments may vary greatly amongst different
teachers, even though they address the same activity design. Puntambekar et al. [6] compare
classroom enactments of an inquiry science curriculum by two teachers and suggest the
importance of teachers in helping students make connections between activities such as
brainstorming, generating questions, finding and applying information in an inquiry unit.
The findings of studies in the context of science and mathematics may not be applicable for
language learning. Research on teachers enactment of pedagogy design in productive
collaborative learning for language classrooms is still lacking.
Collaborative activities have been widely used in language learning [1]. A growing
number of researchers work on developing technological environments to provide explicit

scaffoldings for language learners, as well as visualization of and feedback on group work
process. These studies focus on technology design instead of CSCL pedagogical design and
enactment in authentic classroom environments. This paper uses a comparative study
approach to examining the differences of teacher enactment of the same collaborative
writing activity, seeking to explain these differences, and how they might impact language
learners learning outcomes. It is a collaborative second language (L2) writing lesson in a
networked classroom supported by a collaborative technology called Group Scribbles (GS).
The findings shed light on how to address the teachers role and challenges in enacting
well-designed CSCL activities successfully in real classroom settings.

2. Context of Study
The study described here is part of a 3-year project introduce Rapid Collaborative
Knowledge Improvement (RCKI) to language learning classrooms in a secondary school of
Singapore [5]. The school provides a technology-rich environment for students. Each
student is equipped with a laptop. In Singapore schools, English as the first language is the
main teaching language in schools, whereas Chinese is taught as a second language (L2) for
the Chinese ethnic students. This paper focuses on the enactment of collaborative L2
argumentative writing lessons in secondary grade 2 (14-16 year old) higher Chinese classes.
The two experimental classes are selected because the students writing ability of the
two classes is at a similar level in term of the scores of writing in schools examination on
Chinese subjects (t=-0.265, p>0.05). One of them, Class E1 (N=20) is taught by Chin, who
is a female teacher with approximately 10 years of teaching experiences. The other one is
Class E2 (N=16) taught by Judy, who has about 5 years of teaching experiences. Both Chin
and Judy have had experiences of studying Chinese language abroad (Chin in Taiwan for 4
years and Judy in mainland China for 3 years). Compared with other local Chinese language
teachers without overseas educational backgrounds, these two teachers are fully aware of
the necessity and importance of Chinese ethnic students in mastering the Chinese language
well. They are willing to try new teaching approaches to arouse students interests in
Chinese language learning. Both of them fully believe that every student has potential, and
what teachers need to do is to assist students to reach their potential.

3. Intervention
GS is a software platform designed for supporting students to create lightweight multimodal
representations for mediating collaborative activities. Its workspace is divided into private
and public spaces presented in a two-paned window (Figure 1). The lower pane of the GS is
the users personal workspace or private board whereas the upper pane is the public board or
public board. The private workspace was provided with a virtual pad of fresh scribble sheets
on which the user could draw or type. The students can share the scribbles sheets by
dragging them from private space to public space. A student can select any group board by
clicking the board number on the right-top, and browse all other groups postings posted on
the public board. GS hence promotes and facilitates intra- and inter-group interactions.
When exploring the affordances of GS for Chinese language learning, we have
proposed the rapid collaborative knowledge improvement (RCKI) concept and related 9
principles. The concept of RCKI refers to the notion of democratizing participation and idea
refinement in the context of live dynamic classroom settings, that is, face-to-face (FTF)
collaborative knowledge construction and improvement over the duration of a class session,
and supported by certain technologies for lightweight instant interaction (see [9, 12] ).

Before implementing the GS-based collaborative writing classes, both teachers and
students had been familiar with its function. A series of professional development sessions
(1 hour per week, 5 weeks) were held by two researchers to ensure the teachers belief and
understanding about GS-based language learning and RCKI principles. After that, a
GS-based Chinese writing lesson (60 minutes) was co-designed by the teachers and
researchers, guided by RCKI principles and argumentative writing strategies.
The main learning objective of the lesson was to help students understand that an
argumentative essay can be written from discussing a phenomenon followed by finding
effects and providing solutions. The topic for the writing was Guilty? Plastic survey. A
template (Figure 2) was uploaded as the background of each GS group board for assisting
learners to perform tasks. Chin and Judy enacted the same lesson plan for their classes.
Table 1 presents the main phases of the designed collaborative writing activity. Both classes
of students were heterogeneously organized into groups of 4 members each.

Figure1. The user interface of GS

Figure2. A graphic organizer for the activity

Table 1: Overview and main phases of the activity


Phases
P1
P2
P3
P4
P5

Teacher-directed moves
Introduce to students the main purpose of the GS activity; help them recap strategies for argumentative
writing; encourage them to think of phenomenon around the topic and brainstorm reasons for the phenomenon.
Facilitate students to perform the task (providing results based on given reasons); ask them to explain the
results and give comments for group artifacts.
Facilitate students to perform the task (thinking about solutions based on existing reasons and results);
ask them to explain the results and give comments for group artifacts.
Encourage students to do FTF discussion to improve their group artifacts, synthesize and extract big ideas
about their group writing.
Facilitate and ask students to present group final artifacts, provide comments and summarize the whole lesson.

4. Method
This study compares two teachers enactment of the lesson plan and explores its relation
with students learning outcomes. There are three main sources of data: 1) results of
students subsequent individual writing; 2) individual student feedback; 3) teachers
instructional discourse in the classroom. The quantitative results of subsequent individual
writing and students qualitative feedback together help ascertain any difference in learning
outcomes between Chin and Judys class. The data regarding the two teachers instructional
discourses are analyzed to find out if and how the two teachers enactments differ, in an
effort to understand what might have affected the effects of collaborative writing enactment.
When collecting data on teachers momentary instructional practices, two researchers
observed Chin and Judys GS lessons, took notes and captured the whole class process by
video cameras. The researchers used a chronological representation tool called Studio Code
to provide visual patterns of their instructional discourse. First, for the sake of consistency,
two researchers watched and transcribed all the video data about teacher discourse in two
classes. Next, the transcribed data was segmented into units of theme by using semantic
features such as ideas, discussion topic, or by regulative actions such as asking for an

explanation or explaining on specific point. Finally, the teacher instructional discourse was
coded again on Studio Code to present the patterns of the teacher practices visually.
The coding scheme was designed to capture five aspects of enactment. It was
developed through an iterative process of creating codes, coding, modifying and refining
codes, and recording consistent with Miles and Hubermans [4] recommendations for
rigorous and meaningful qualitative data analysis. We did open coding as many existing
coding schemes are mainly about inquiry-based learning and thus they are not applicable for
language learning. For instance, we added the item Relating to language borrowed from
Swain and Lapkin [10] who coined language-related episodes when they studied L2
learners language use in collaborative dialogues. Parts of the coding categories of the study
came from research on enactments of inquiry lessons [6]. For example, Relating to topic
referred to the way in which students were encouraged to think back about the topics they
had already learned, and to connect that learning to the new topic that was being introduced.
In this study, it referred to relating knowledge and strategies of argumentative writing to the
current topic. Focus on goals referred to the speech that teacher expressed the aim of
lesson or the activity design explicitly to students. The details are listed in Table 2.
Table 2: Coding schemes with examples
Categories
1: Ensure the
activities can be
completed as
designed

2: Ensure the
activities can be
completed with
a high quality

Items
Task
introduction
Time
management
Providing
encouragement
Protocol
FTF discussion
Ideas
HOT

3: Ensure
students can
understand the
significance of
the activity
design

Relating to
topic

4:Help students
master
language
knowledge
5:
Help students
improve
cognitive skills

Relating to
language

Focus on goals

Asking for
further think
Explaining
specific point
Assessing or
summarizing
students' work

5. Results and Discussion

5.1 Students Learning

Interpretations
Introducing how to complete the
tasks
Reminding students to pay
attention to time
Encouraging or giving praise to
students performance
Rules for group task complement
in GS-based learning environment
Encouraging students to do FTF
discussion
Encourage students to share ideas
and improve them consistently
Encouraging skills like analysis,
synthesis, categorizing, evaluation
Help students think back about the
topics that they had learned, and
connect that learning to the new
topic that was being introduced.
Helping student keep the overall
goal of the challenge in mind
while being engaged in activities.
Episodes in which the teacher
deliberate over lexical or
grammatical choices
Opportunities for students to
explain their thinking
Comment and elaborate on student
ideas
Providing evaluative response to
students group artifacts

Examples
In the task 2, your group should give effects
based on reasons that have been posted.
Last 2 minutes. Seize the time to complete
your groups task.
Group 2 did quite well.
Each group leader help teacher to monitor the
procedure of your group work.
Communication. Let me hear your voice of
the group verbal discussion.
After visiting other groups board, you need
to improve own existing postings
If you disagree with the comments from your
peer group, please give your reasons.
We have learnt about the Five Fingers
which taught us that an argumentative essay
could be written from: individual, family,
friend, county and society. You can give
reasons from these 5 perspectives.
Today, the ultimate goal of our study is to
mater the RES model for your argumentative
essay writing.
The idiom should be Ru Huo Ru Tu. The
last character should be pronounced as Tu.

Here, what does others refer to?


Social influence here might be understood
as plastic surgery rampant in the community
Group 5 provides reasons for the
phenomenon of plastic surgery mainly from
individual and society these two aspects

We started our analysis of student learning by examining whether the two classes performed
differently in the subsequent individual writing. Each student took 50 minutes to complete
the writing with the same topic in the following Chinese language class. Their compositions
were marked according to writing rubric from Ministry of Education of Singapore, which
has 5 parameters: 1) Solid writing material; 2) In line with topic and the goals; 3)
Consistence of reasoning and focus 4) Diversity and innovation of ideas 5) Creative
imagination. The maximum score for a composition is 70. All compositions from both
classes were marked by two teachers. The strength of association between scores marked by
them was high (r=0.727). The result of t-test shows a significant difference of students
subsequent writing scores between two classes (t=3.153, p<0.01). The mean score of Chins
class is 47.53, higher than 41.88 of Judys class. It indicates that the writing score of Chins
class after GS-based collaborative activity is significantly higher than that of Judys class.
In addition to academic performance, students were required to reflect on their learning
experiences which help researchers to explain why the results of sequential writing differ
between Chin and Judys classes. Two semi-constructed questions were provided: Q1)
Collaborative writing and individual writing, which one do you prefer? Why? Q2) What did
you learn from the last GS lesson? Towards Q1 Almost all of them were positive about
collaborative writing. Two students from Chins class and three students from Judys class
stated that they prefer individual writing to collaborative writing. Students from different
classes provided different reasons. Students from Chins class emphasized that they enjoyed
the process of creating alone more, compared with completing a collaborative writing task.
Students from Judys class merely emphasized that they felt it was easier to do individual
writing. Of those who preferred to collaborative writing, the predominant reason given (by
12 students from Chins class and 9 students from Judys class) was that it provided them
with an opportunity to compare and exchange ideas with each other. For example, students
from Judys class pointed out that:
I prefer to writing together with my group members, as we can share our ideas, help each other to finish
our writing. I like group writing as we come to know others views towards the topic, and we can
choose the most suitable opinions through discussions.

Chins students however, provided wider and deeper reasons to explain why they like
collaborative writing. In addition to sharing ideas, they noted that:
I like to write together in a group, because we can discuss with each other and provide ideas and
suggestions for each other. My own idea might not be the best, but we can keep improving these ideas
in group work. students own different ideas toward the same topic, thus everyone will be
involved in intensive discussion on how to write an article. During this process, we can learn from
each other, and in this way, our abilities of critical thinking and collaborative learning get
opportunities to be improved.

The most obvious difference is from two class students answers for the second
questions. All 20 students from Chins class noted that they learned that reasons, effects
and consequences (RES) are three essential parts of an argumentative essay. Just like one
student expressed that:
RES indeed help us better understand the process and the theme of essay writing. It portrays the
whole thinking process and the way to find out the answers. Making use of this model, students can
get the right way of writing with guidance.

However, no student in Judys class mentioned RES. Their feedback focused on how
to write together or how to complete a task with group members within a short period of
time. It seems that students in Chins class could better understand the intent of teachers
activity design. This may explain the differences on students academic performance
between two classes, although in both classes, students hold a positive opinion of their
group work. It also seems that the students in Judys class were highly engaged but did not
learn as much as students in Chins class. What might account for this difference? The
researchers try to further find out if there is a difference between teachers enactment that
may cause the differences in students learning

5.2 Comparing Teacher Discourse through Chronological Representation


We used Studio Code to represent teacher discourse chronologically to understand how they
facilitated GS activities. This method enables a graphic representation of the chronology of
discourse, allowing an understanding of how it changed over time [7]. Figure 3 and Figure 4
describes Chin and Judys discourse as they occurred respectively. The top line of each
figure indicates the time period of every teacher-directed phase (Table 1). The line of the
figure depicts a single category with the incidence of teacher discourse in that category
represented along the horizontal line. Each action is represented using a bar code.

Figure 3. Chronological representation of Chins discourse

Figure 4. Chronological representation of Judys discourse


Generally speaking, both Chin and Judy did enact the lesson plan with some fidelity.
They spent almost the same time at the beginning phase of orientation and introduction of
the activity (P1), and the last phase of evaluation and students presentation (P5). But the
patterns of two teachers orchestrating the activity are different. At the orientation phase
(P1), Chin made the goals of the GS-based activity explicit. Rather than assigning the task
directly, she spent 5 minutes in introducing the RES and helping students to recap other
related skills for argumentative essay writing. Judy, however, spent about 7 minutes to talk
about the existing phenomenon of plastic surgery and to give examples in daily life. Judy
spent a lot of time seeking to arouse the students interests of the topic discussion, but
without making the goals of the activity clear to students.
Figures 3 and 4 indicate that Chin spent more time than Judy at phase 2 of activity (P2).
When students were brainstorming reasons for the phenomenon of the popularity of plastic
surgery, Chin said If your group has posted sufficient reasons, now you need to group
these given reasons. Do remember to think of the five perspectives mentioned in Five
Fingers that we have learnt. After the students had completed the first phase of the
activity, Chin selected some of the group works to provide comments and further
explanations (see Figure 3). However, Judy did not asked her students to categorize their
brainstormed ideas and she seldom provided improvised feedback at the class level, though,
like Chin, she kept passing between groups and monitoring the state of group work as well.
Below figure 5 visualizes the differences between the number of instances of Chin and
Judys discourse occurring in each category. It is observed that Judy spent more time on task
introduction than Chin. In other words, both teachers gave their task instruction clearly to
ensure students follow the designed procedures, but Chins introduction was more concise.
Compared with Judy, Chin managed class time more strictly, and she praised and
encouraged her students more frequently. The similarity is that both encouraged students to

do FTF discussion to improve their ideas consistently, and to give comments/suggestions


for others at the group or class level. Except for the number of providing protocol, there is
no obvious difference in category 2. Judy liked to provide more specific instructions
regarding to the GS group work protocol. For example, she required each group to pick a
different color to represent their group. She thought this might help proceeding round robin
smoothly. This could potentially cause students losing their individual identity. In contract,
Chin preferred that group students could generate their own group protocol.
The most significant difference between Chin and Judys enactment discourse can be
found in category 3. Chin helped students make clear the learning objective and make
explicit the connection between the meaning of collaborative writing activity design and
learning content, rather than merely gave commands for students to follow procedures
automatically. This sort of discourse often happened at the beginning of the activity in
Chins class (figure 3). She used this approach to help students to complete subsequent
phases of the activity effectively. This result is consistent with findings drawn from
students feedback, that students from Chins class had a better understanding on the
objective of the designed collaborative writing activity. The difference between two
teachers enactments in this category is probably one of the main reasons leading to the
difference of students performance in the subsequent writing between two classes.
As for category 4 & 5: Help students master language knowledge & improve cognitive
skills, there is not much difference on the discourse frequency between two teachers. Both
teachers instructional discourse did not focus on correcting grammatical or syntactical
errors. They sought to scaffold students cognitive development and problem-solving
approach. The difference exists in the timing of providing scaffoldings. Chin monitored the
progress and quality of group work and reacted immediately, whereas Judy concentrated her
explanation and assessment when students had completed the activity thoroughly.
C1: Ensure the activities can be completed as designed

C 2 : Ensure the activities can be completed with a high quality


16

70
60

Chin

14

Chin

50

Judy

12

Judy

40

10

30

20

6
4

10

0
Task introduction

Time management

Providing
encouragement

C3 : Ensure students can understand the significance of the


activity design

0
Protocol

FTF discussion

Ideas

HOT

C4 & 5 : Help students master language knowledge & improve


cognitive skills
14

18
16

Chin

14

Judy

12

Chin

10

Judy

12

10

2
0

Relating
language

2
0
Relating topics

Focus on goals

Asking
students to
explain or
further think

Explaining on Summarizing or
specific point
assessing
students' work

Figure 5. Frequency chart comparing Chin and Judys discourses

6. Conclusion and Limitations of This Study


This study pointed out three major differences in the enactments of lessons of two language
teachers. Firstly, the learning objectives and the connection between the purpose of writing

activity design and learning content, were made explicit in Chins class, whereas it was not
evident in Judys class. Secondly, Chin was able to improvise teaching in the light of
dynamic formative feedback from students group works at each phase of the activity. Judy,
however, commented students group work only at the end of the activity. Thirdly, Chin
strictly controlled the time of each segmented activity and consistently encouraged and
praised students. Although Judy also did well at the beginning trying to arouse students
interests in participating in the activity, she neglected to maintain this enthusiasm. The
results suggest that language teachers should 1) articulate the objective of the collaborative
activity on language learning explicitly; 2) provide improvised formative assessment and
scaffolding to support students cognitive development and problem-solving in student
working process, rather than being entangled on correcting grammatical or syntactical
errors; 3) control the tempo of the activity and maintain students enthusiasm.
There are limitations in this study that need to be addressed in further research. For
example, this study focused on analyzing the differences of enactments between two
teachers. The commonalities between them are not been taken into consideration, though
they may influence the effects of the enactment. The present comparative study is conducted
under the assumption that two teachers have the similar beliefs about student language
learning after a series of professional development sessions. We acknowledge that the
teachers perceptions about the students as well as their knowledge and beliefs about
learning influence their enactments. Thirdly, due to the page limit, we are not able to present
the details about the process of students collaborative learning in relation to the teachers
instructional discourse.

References
[1]

Blake, R., Wilson, N. L., Cetto, M., & Ballester, C. P. (2008). Measuring oral proficiency in distance,
face-to-face, and blended classrooms. Language Learning & Technology, 12(3), 114-127.
[2] Cakir, M. P., Zemel, A., & Stahl, G. (2009). The joint organization of interaction within a multimodal
CSCL medium. Computer-Supported Collaborative Learning, 4, 115-149.
[3] Dimitriadis, Y. A., (2010). Supporting teachers in orchestrating CSCL classrooms. In A. Jimoyiannis
(Ed.), 7th Pan-Hellenic Conference with International Participation ICT in Education (vol.I,
pp.33-40). Korinthos, Greece.
[4] Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.)
Thousand Oaks, CA: Sage Publications.
[5] Looi, C. K., Chen, W., & Patton, C. (2010). Principles and enactment of rapid collaborative knowledge
building in classrooms. Educational Technology Magazine, September.
[6] Puntambekar, S., Stylianou, A., & Goldstein, J. (2007). Comparing classroom enactments of an inquiry
curriculum: Lessons learned from two teachers. The Journal of the Learning Sciences, 16(1), 81-130.
[7] Sandoval, W. A., & Daniszewski, K. (2004). Mapping trade-offs in teachers' integration of
technology-supported inquiry in high school science classes. Journal of Science Education and
Technology, 13(2), 161-178.
[8] Schneider, R. M., Krajcik, J., & Blumenfeld, P. (2005). Enacting reform-based science materials: The
range of teacher enactments in reform classrooms. Journal of Research in Science Teaching, 42,
283312.
[9] Suthers, D. D. (2006). A Qualitative analysis of collaborative knowledge construction through shared
representations. Research and Practice in Technology Enhanced Learning, 1(2), 115-142.
[10] Swain, M., & Lapkin, S. (1998). Interaction and second language learning: Two adolescent French
immersion students working together. The Modern Language Journal, 82(iii), 320-338.
[11] Tabak, I., & Reiser, B. (1997). Complementary roles of software-based scaffolding and teacher-student
interactions in inquiry learning. In R. Hall, N. Miyake, & N. Enyedy (Eds.), Proceedings of the 2nd
International Conference on Computer Support for Collaborative Learning (pp.289298). Toronto,
Ontario, Canada: AACE.
[12] Wen, Y., Looi, C.K., & Chen, W. (2011). Towards a model for rapid collaborative knowledge
improvement in classroom language learning. In Hans, S., Gerry, S., Naomi, M., Nancy, L. (Eds.)
Connecting Research to Policy and Practice: Proceedings of CSCL 2011 (pp. 836-851). Hong Kong.

Do students and lecturers actively use


collaboration tools in learning management
systems?
Nastaran ZANJANI, Shaun NYKVIST* & Shlomo GEVA
Queensland University of Technology, Australia
*s.nykvist@qut.edu.au

Abstract: In recent years there has been a large emphasis placed on the need to use Learning
Management Systems (LMS) in the field of higher education, with many universities
mandating their use. An important aspect of these systems is their ability to offer
collaboration tools to build a community of learners. This paper reports on a study of the
effectiveness of an LMS (Blackboard) in a higher education setting and whether both
lecturers and students voluntarily use collaborative tools for teaching and learning.
Interviews were conducted with participants (N=67) from the faculties of Science and
Technology, Business, Health and Law. Results from this study indicated that participants
often use Blackboard as an online repository of learning materials and that the
collaboration tools of Blackboard are often not utilised. The study also found that several
factors have inhibited the use and uptake of the collaboration tools within Blackboard.
These have included structure and user experience, pedagogical practice, response time and
a preference for other tools.
Keywords: Learning management systems, Communities of Practice, e-learning

Introduction
In recent years, the necessity for Higher Education Institutions (HEI) to invest in learning
management systems (LMS) that provide a platform for e-learning (electronic learning) has
increased. This has often been seen as an attempt for these institutions to be more
competitive and to capture a larger market share of students [33]. Initially, the idea of using
e-learning systems was focused around the ability to connect with external and distance
education students and provide greater access and flexibility to these students [2; 24].
However, e-learning has now become a core component of the education experience for
many students in higher education and an ever-increasing combination of face-to-face (F2F)
learning and e-learning is now occurring [5; 20]. This learning, referred to as blended
learning, uses technology to expand the physical boundaries of the classroom, providing
access to learning content and resources and enhancing the instructors ability to receive
feedback on learners progress [21].
In creating this blended learning environment in higher education, an LMS (such as
Blackboard or Moodle) is often used to access inbuilt collaboration tools such as blogs,
wikis and discussion forums. These tools, often referred to as web 2.0 or e-learning 2.0
tools, are most common to these environments and touted as having the ability to empower
educators to facilitate a sense of community through the possible interactions that could
occur in these environments. Consequently, it is this buoyant relationship between the use
of Internet collaboration tools and people that has the potential to create powerful online
learning communities [8; 16].

In an attempt to further explore the relationship between current research into the
benefits of using online collaborative tools to create a sense of community, this paper will
describe and report on a small scale-study (N=67) into the use of collaborative tools within
higher education. This study is specifically based on the collaborative tools available as
default within the LMS, Blackboard.

1.

Blended Learning and Collaborative Tools

Blended learning (or hybrid learning) combines e-learning with other, usually more
traditional forms of teaching and learning [21]. Bielawski and Metcalf [3] described it as
blending classroom, asynchronous and synchronous e-learning, and on-the-job training
(p. 71). It is generally held that blended learning combines the advantages of two learning
modalities [34], p.157 with Bowles [6] suggesting that when classroom instruction is
combined with self-paced instruction via the Internet, for example, the face-to-face contact
makes for easy social interaction and allows for instant feedback (p. 47). The advantages
of blended learning may be summarised as follows:
The blended learning approach helps to create a shared understanding of
concepts important to the learning culture and provide opportunities to
reinforce them in a live classroom setting. Leveraging the convenience and
accessibility of online components with traditional classroom instruction also
expands the curriculum without increasing programme completion time.
It is this strong relationship between the F2F interactions and online collaborative tools
in a blended learning environment that has the potential to move educators from a didactic
approach of teaching and learning to an approach that is based on building a sense of
community through computer mediated communications (CMC). CMC is a term referring
to the interpersonal discourse between users with computer-based media. CMC extends
from discussion boards/forums through to contemporary Web 2.0 applications [36] and is
said to enable collaborative reflection, which, in turn, prompts the conceptualisation and
re-conceptualisation of ideas [10; 25]. It is these conversations and interactions between
students that strengthen their deeper understanding of the topic [29].
Learning management systems within the higher education sector provide educators
with an environment containing inbuilt collaborative tools (e.g. discussion forums, blogs
and wikis) to use for their teaching purposes. These collaborative tools can be used for
computer-mediated communication where communities of practice can be supported and
envisaged. When these tools are coupled together with F2F teaching the notion of blended
learning can be realized. In realising this notion of blended learning, a widely used LMS
such as Blackboard, is often used in the higher education sector [27].
The Blackboard website publishes a number of case studies which highlights the
possible advantages of using the Blackboard LMS within higher education. According to
Blackboard [4], the University of Cincinnati has been a Blackboard customer since 1999
and uses their LMS for distributing learning resources, podcasts of recorded lectures and
announcements while the University of North Carolina provides customized library content
for students using their LMS [27]. However, despite the large number of clients using
Blackboard, Heaton-Shrestha [17] found that learning resources and announcements are
the most valued tools of Blackboard used by students and lecturers of the Kingston
University, not collaboration tools. Current literature highlights the importance of these
tools, however, further research [12; 22] identifies a lack of active participation by students
and teaching staff with these tools in the Blackboard learning environment.

While there is little empirical research surrounding the lack of use of these tools in this
environment, Alexander and Boud [1] claim that the potential for online learning is not
being realised due to traditional didactic approaches being transferred to the online
environment. This approach merely mimics the traditional classroom with lecture notes and
resources being placed online and the LMS is seen as a web-based delivery of course
resources or as a communication tool. The need to understand the issues surrounding the
limited use of these collaborative tools within an LMS such as Blackboard is essential for
a blended learning environment to exist. One of the major issues facing researchers is the
rapid advancement of technology used within these environments and the ability for
research to keep abreast of it [15; 28].
According to Greenagel [13] the development of collaborative learning systems that
ignore users learning styles could be one contributing factor to their failure to engage
students and staff in their use, while Everson [11] and Wallace [35] argue that the user
friendliness and interface design need to be considered. Everson [11] advises not to waste
valuable time preparing tools that will only frustrate and disenchant your students.
Romiszowski [29] further claims that these systems should focus attention on efficient
learning materials and not just deal with indexing, coding and tagging teaching objects to
facilitate using digitized learning materials.
These factors of design and usability may contribute to some reluctance by students
and teachers to use the systems, however other factors such as increased workload may also
contribute to the lack of their use [26; 30; 32]. The administration of students and the
monitoring of their interactions can contribute to an increased workload for an educator.
The issue of workload was not only evidenced by educators, but also students, who often
complained when asked to use the collaborative tools as part of their learning experiences
[30]. In contrast to this, Jones, Blackey, Fitzgibbon and Chew [19] claim that students with
individual interests attempted to use the available collaborative tools when afforded to
them.
Given that collaborative tools within an LMS such as Blackboard offers a means by
which blended learning can occur, current research highlights the challenges that educators
within higher education institutions face in actively using these collaborative tools
effectively. Consequently, the following study reported on here investigates the use of
collaborative tools within Blackboard at a university that encourages blended learning
across all disciplines.

2.

Research Design

2.1 Research Objectives


The main purpose of this small-scale study was to investigate how collaborative tools are
being used within Blackboard for teaching and learning at a major Australian University,
and to explore the factors that influence their usage. Blackboard is a conventional learning
management system used for a period of five years at the university where the study was
conducted.
2.2 Research Methodology
The study employed a mixed methods approach that combined both qualitative and
quantitative strategies. Simultaneously using both approaches allows for a holistic view of
the problem to be generated and provides a comprehensive analysis of the research
problem [9]. Open-ended interviews were conducted with participants and usage data

pertaining to the use of collaborative tools in Blackboard were collected and analysed.
The open-ended interviews cater to more substantial information being generated by
allowing respondents to state their own perceptions with their own expressions [31], while
the combined usage statistics build a more holistic view of the study.
2.3 Participants
The participants (N=67) of the study consisted of both teaching staff (n=9) and students
(n=58) from the faculties of Science and Technology, Law, Business and Health, all
studying at the university where the study was conducted. The participants were informed of
the study through emails sent to each of the faculties and they individually volunteered and
gave full consent to participate in the study. Participants who volunteered to be a part of the
study were of varying ages and of mixed sex.
2.4 Interviews
Interviews were focused on the way students and lecturers use the collaboration tools of
Blackboard and were open ended in nature. The number of participants interviewed was
brought to an end once a saturation point had been reached where no new data was collected
from participants. Guest [14] demonstrated that saturation often occurs within the first
twelve interviews and that this is sufficient to obtain a reliable conclusion. The participants
interviewed in this study were 67.

3.

Research Findings and Discussion

The two main sources of data were the responses to the interviews and the Blackboard
usage statistics of the collaborative tools. This section will present the findings in each of
these areas.
3.1 Interviews on the use of collaborative tools
The participant interviews (N=67) revealed that 33% staff interviewed used collaborative
tools within Blackboard while 51% students interviewed indicated that they had used
collaborative tools as part of their learning experience. All the students that indicated that
they had used some of the tools, also stated that they only briefly used them as an add-on to
their existing learning experiences.
An analysis of student (n=58) and staff (n=9) responses from the interview on factors
pertaining to why they did not use collaborative tools in Blackboard could be grouped
under six main categories; structure and user experience, availability of time, preference for
other tools, lack of knowledge about tools, pedagogical practice and response time.
3.1.1

Structure and User Experience

The structure and user experience of collaboration tools within Blackboard accounted for
one of the major reasons why collaborative tools were not used. Over two thirds of student
participants (67.35%) indicated that this was an issue while 67% of lecturers also found
structure and user experience to be a factor. Structure and user experience related to the
ease at which participants could easily navigate and find functions and use the collaborative
tools. General consensus from student participants is that Blackboard is hard to navigate

and it is not user friendly while staff participants also found complicated procedures
associated with using the tools.
3.1.2

Availability of Time

The availability of time was highlighted as a contributing factor by 13% of the student
participants and 55% of the staff participants. Student participants indicated that they
struggled to find time to keep up with the other requirements of the unit and learning how to
use the tools or to participate online was seen as another burden on time. Supporting this
finding one of the students stated that I have no time to do this, because you have to spend
a lot of time to understand how to set up them [the forums] (S-6). Lecturers were also
concerned about the time needed to structure the use of the tools and be actively involved
with either synchronous or asynchronous discussion.
3.1.3

Preference for other tools

A preference for other tools that students (39.5%) were already accustomed to was indicated
as another factor in the student use of collaborative tools in Blackboard. While students
indicated a preference for other tools such as Skype or MSN Messenger to discuss issues or
topics pertaining to their study or a particular unit, no lecturers indicated a preference for
other collaborative tools. One of the students questioned why you would learn something
new when there is already something else available just as good is wasting time (S-5).
3.1.4

Lack of Knowledge about Tools

Lack of knowledge about the functionalities of the various collaborative tools or their
existence within Blackboard was identified as a factor affecting their use. This was
identified as a factor by 48.2% of students and 75% of lecturers. A number of students
referred to Blackboard as merely a platform to access learning materials and receive
announcements. It is within this context that research conducted by Bradford et al. [7]
supports these findings in associating complexity and knowledge of LMS tools as a
limitation of these environments.
3.1.5

Pedagogical Practice

The pedagogical practices of 50% of lecturers were seen as a factor impinging upon the use
of collaborative tools in their teaching. These lecturers indicated that they were comfortable
with their traditional approaches and that shifting to new practices was difficult and time
consuming. This approach is reiterated by Alexander and Boud [1] who claim that these
environments are not being used to their full potential and that didactic teaching practices
have become a part of these online environments.
3.1.6

Response Time

Response time refers to the length of time students had to wait to receive a response using
asynchronous collaboration tools within Blackboard. This was indicated as a contributing
factor to why 48% of students did not use or continue to these tools. This confirms findings
of a number of studies [23] where a lengthy response or no response discourages the student
to use the collaboration tools in Blackboard.

3.2 Statistical Usage Data


The interviews conducted in this study provided qualitative data on factors pertaining to
why participants did not use the collaborative tools in the Blackboard environment. The
statistics presented in this section demonstrate the average time that students in the entire
university spent engaged with Blackboard (Figure1) and further confirmed that the
Blackboard LMS is mostly used as an online repository for teaching resources (Figure 2).

Figure 1: User engagement


Figure 1 shows the rate of university-wide user engagement in Blackboard for one
semester. This user engagement does not distinguish between a user reading and viewing
learning resources or creating resources and the use of collaborative tools. We can ignore
visits shorter than 30 seconds as this is too short a time to indicate a significant collaboration
attempt if at all. A normal distribution of visit durations is observed.

Figure 2: Percentage of courses that use at least one collaborative tool


Figure 2 displays the percentage of units that used Blackboard collaboration tools in
the university. These statistics are captured over a three year period for each teaching
semester and indicate that 10% or less of the units offered at the university use some form of
collaborative tools.
The data presented in Figure 2 not only demonstrates a low percentage of units using
Blackboard collaboration tools but also indicates a decline in the use of these
collaboration tools. These findings further support and confirm the results of previous
studies [12; 17; 18] and beliefs that learning management systems such as Blackboard are
used mainly as a content delivery mechanism and not used to their full potential.
The most prominent factor contributing to the lack of use of the collaborative tools in
Blackboard resulted from a negative user experience with the tools with over two thirds of
both students and lecturers indicating that this was an issue. The next most important factor

for all participants was related to an understanding of the available collaboration tools,
followed by availability of time for lecturers and response time for students. While
literature [8; 16] touts the importance of using these tools for building communities of
practice, it is evidenced in these findings that there would be 10% or less units in the
university with the ability to build these powerful learning communities within
Blackboard, due to the absence of students and teaching staff actively using collaborative
tools.

4.

Conclusion

There is no doubt that e-learning is a significant part of higher education teaching and
learning, however it is vitally important that it is used in ways that promote and encourage
positive learning experiences for all and build communities of learners. The mere existence
of collaborative tools in an LMS such as Blackboard does not automatically equate to
them being used for successful teaching and learning purposes. While the study presented
here is a small-scale study of one typical large university and one LMS, it demonstrates the
need to address the key factors that act as barriers to the use of collaboration tools in higher
education. The most significant factor in the study was that of structure and user
experience. It highlights the need to design computer supported collaboration tools that
encourage student interaction to produce collaborative knowledge building through
communities of practice.

References
[1] Alexander, S. A. & Boud, D. J. (2001). Learners still learn from experience when online. Teaching &
Learning Online - Pedagogies for New Technologies, 3, 15.
[2] Allen, I., & Seaman, J. (2007). Online Nation: Five years of growth in Online Learning. The Sloan
Consortium: Needham, MA.
[3] Bielawski, L. & Metcalf, D. (2003). Blended e-learning: Integrating knowledge, performance, support,
and online learning. Amherst, MA: HRD Press.
[4] Blackboard INC. (2008). Blackboard & the University of Cincinnati [Online]. Available:
http://www.Blackboard.com/CMSPages/GetFile.aspx?guid=90ffd3e6-1073-4815-9ff8-5e34975696b4
[Accessed September 2010].
[5] Borden, J. (2011). The Future of Online Learning.
[6] Bowles, M. S. (2004). Relearning to e-learn: Strategies for electronic learning and knowledge.
Melbourne, Australia: Melbourne University Press.
[7] Bradford, P., Porciello, M., Balkon, N. & Backus, D. (2007). The Blackboard Learning System: The Be
All and End All in Educational Instruction? Journal of Educational Technology Systems, 35, 301-314.
[8] Brown, A., & Campione, J. (1990). Communities of learning and thinking, or a context by any other
name. In D. Kuhn (Ed.), Developmental perspectives on teaching and learning thinking skills
(Contributions to Human Development Series) (pp. 108-126)). Basle, Karger.
[9] Creswell, J. (2009). Reaserch design: Qualitative, quantitative and mixed methods approaches. Los
Angeles: Sage Publications.
[10] Dunn, T. (2003). Preparing preservice teachers for diverse classrooms: Influences of CMS on beliefs and
instructional practices. Proceedings of World Conference of E-Learning in Corporate, Government,
Health Care and Higher Education, 2003 444-447.
[11] Everson, M. (2009). 10 Things Ive learned About Teaching Online. Retrieved 7/04/2012
http://elearnmag.acm.org/featured.cfm?aid=1609990
[12] Green, S. M., Weaver, M., Voegeli, D., Fitzsimmons, D., Knowles, J., Harrison, M. & Shephard, K.
(2006). The development and evaluation of the use of a virtual learning environment (Blackboard 5) to
support the learning of pre-qualifying nursing students undertaking a human anatomy and physiology
module. Nurse education today, 26, 388-395.
[13] Greenagel, F. (2002). The illusion of e-learning: Why we are missing out on the promise of Technology.
Phoenix: League for Innovation in the Community College White Papers.

[14] Guest, G., Bunce, A. & Johnson, L. (2006). How many interviews are enough? Field methods, 18, 59-82.
[15] Guri-Rosenblit, S. (2005). Eight paradoxes in the implementation process of eLearning in higher
education. Higher Education Policy, 18, 5-29.
[16] Hartnett, M., Bhattacharya, M. & Dron, J. (2007). Diversity in online learners: Searching for differences
that may matter. In, 2007. IEEE, 899-900.
[17] Heaton-Shrestha, C., Gipps, C., Edirisingha, P. & Linsey, T. (2007). Learning and e-learning in HE: the
relationship between student learning style and VLE use. Research Papers in Education, 22, 443-464.
[18] Heirdsfield, A. M., Walker, S., Tambyah, M. M. & Beutel, D. A. (2011). Blackboard as an online
learning environment: what do teacher education students and staff think? Australian Journal of Teacher
Education, 36, 1-16.
[19] Jones, N., Blackey, H., Fitzgibbon, K. & Chew, E. (2010). Get out of MySpace! Computers & Education,
54, 776-782.
[20] Keengwe, J. & Kidd, T. T. (2010). Towards Best practices in Online Learning and Teaching in Higher
Education. MERLOT Journal of ONLINE Learning and Teaching in Higher Education 6, 533-541.
[21] Klein, H. J., Noe, R. A., & Wang, C.W. (2006). Motivation to learn and course outcomes: the impact of
delivery mode, learning goal orientation, and perceived barriers and enablers. Personnel Psychology,
59(1), 665702.
[22] Landry, B. J. L., Griffeth, R. & Hartman, S. (2006). Measuring student perceptions of Blackboard using
the technology acceptance model. Decision Sciences Journal of Innovative Education, 4, 87-99.
[23] Liaw, S. S. (2008). Investigating students perceived satisfaction, behavioral intention, and effectiveness
of e-learning: A case study of the Blackboard system. Computers & Education, 51, 864-873.
[24] Mason, R. (2006). The university current challenges and opportunities. S. DAntoni The Virtual
University: Models & Messages.
[25] McLoughlin, C., & Luca, J. (2000). Cognitive engagement and higher order thinking through computer
conferencing: We know why but do we know how? Retrieved 28 July, 2007, from
htpp://lsn.curtin.edu.au/tlf/tlf2000/mcloughlin.html
[26] Pirrone, R., Cannella, V. & Russo, G. (2009). A map-based visualization tool to support tutors in
e-learning 2.0. In: 2nd Conference on Human System Interactions, HSI '09, 2009. 482-487.
[27] Pishva, D., Nishantha, G. & Dang, H. (2010). A survey on how Blackboard is assisting educational
institutions around the world and the future trends. In: The 12th International Conference on Advanced
Communication Technology (ICACT), 2010. IEEE, 1539-1543.
[28] Robertson, H. J. (2003). Toward a theory of negativity teacher education and information and
communications technology. Journal of Teacher Education, 54, 280-296.
[29] Romiszowski, A. J. (2004). How's the e-learning baby? Factors leading to success or failure of an
educational technology innovation. Educational Technology-Saddle Brook Then Englewood Cliffs Nj-,
44, 5-27.
[30] Schroeder , A., Minocha, S. & Schneider, C. (2010). The strengths, weaknesses, opportunities and
threats of using social software in higher and further education teaching and learning. Journal of
computer assisted learning, 26 159-174.
[31] Teddlie, C. & Tashakkori, A. (2009). Foundations of mixed methods research. Thousand Oaks, CA:
Sage.
[32] Trentin, G. (2009). Using a wiki to evaluate individual contribution to a collaborative learning project.
Journal of Computer Assisted Learning, 25, p 43-55.
[33] Turner, W. L. & Stylianou, A. C. (2004). The IT advantage assessment model: Applying an expanded
value chain model to academia. Computers & Education, 43, 249-272.
[34] Voci, E. & Young, K. (2001). Blended learning working in a leadership program. Industrial and
Commercial Training.
[35] Wallace, R. M. (2003). Online Learning in Higher Education: a review of research on interactions among
teachers and students. Education, Communication & Information 3, 241-280.
[36] Wood, A., & Smith, M. (2005). Online communication. Linking technology, identity and culture (2nd
Ed.). London: Laurence Erlbaum Associates.
[37] Yang, Y. & Cornelius, L. F. (2004). Students' Perceptions towards the Quality of Online Education: A
Qualitative Approach. Association for Educational Communications and Technology, 2, 861-877.

Teacher Agency and Student Autonomy in


Inquiry-based Mobile Learning Trail
a

Esther TANa*, Hyo-Jeong SOa & Xujuan ZHANGa


National Institute of Education, Nanyang Technological University, Singapore
*esther.tan@nie.edu.sg

Abstract: This research study investigates students capacity at greater autonomy in an


inquiry-based mobile learning trail in relation to the agent of the teacher (e.g., instructional
design, facilitation and scaffold support). To afford a more coherent study, narrative
interviews and web-based data capturing student-student and teacher-student interaction
were obtained for analysis. Data were analysed based on three key areas: (a) the design of
the mobile learning trail in facilitating autonomous learning, (b) the interaction with the
teachers and (c) the collaboration with peers. Overall findings showed that students
capacity to engage in autonomous learning rests on learning trail design, collaborative
efforts and an awareness of teachers presence. The teachers cited students profile, the
motivational factor and instructional design as important determinants for autonomous
learning. In conclusion, we argue that students capacity for more autonomy in mobile
learning does not necessarily lie in a decrease in teachers control, but rather, it is
contingent on student readiness, learning design, technological mediation, as well as, the
community of learners.
Keywords: Teacher agency, student autonomy, inquired-based mobile learning

Introduction
The advent of mobile technologies has dramatically revolutionized the conventional role
of teachers and students. Harnessing the affordances of technology-mediated cognitive
tools to engage learners, enhance learning effectiveness, empower and enable synchronous
and asynchronous interaction and collaboration is believed to bring about greater student
autonomous learning. Teachers presumably best function as facilitators to scaffold the
learning milestones and to mediate technological support to enhance learner autonomy.
However, what essentially facilitates the occurrence of more student autonomy and/ or
how teachers can orchestrate such learning situations, are needful areas for more intense
research and investigation. On supporting learner autonomy, Black and Deci (2000 as
cited in [1]) liken this phenomenon to a situation where the learners are equipped and
empowered to make autonomous decisions in the learning process given the accessibility
and availability of pertinent information and opportunities for choice (p.28). And In
theorizing mobile learning, Sharples, Taylor, and Vavoula [2] surface control and
context as two of the key areas for reflection: control is distributed across multiple
elements from teacher, peers, technologies to environmental artefacts, and context is
constructed by the learners interacting with the environment, which comprises of
communities of learners and all mediating technologies. And in our context of inquirybased mobile learning trail, the imminent challenge would be to apportion the right
measure of teacher presence without jeopardizing students capacity at autonomous
learning. Hence, supporting learner autonomy is not a simple equation of decentralizing
teacher agency and control; rather, it calls for an informed action taking into account all

contextual elements in the said learn setting.

1. Theoretical Framework
To encapsulate the nature of the inquiry-based mobile learning trail, the contextual
elements and the social actors (i.e., the teacher and the students), we employ situated
cognition to make sense of how learning takes place for two fundamental reasons. First,
the key theoretical premises of situated cognition afford an insight into the dynamic
interplay of critical constructs such as the learning activities, all mediating tools (e.g.,
physical environment, social actors, artefacts, etc.), and importantly, the cultural and
social practices in the learning context. According to Brown, Collins and Duguid [3],
Knowledge is situated, being in part a product of activity, context, and culture in which it
is developed and used (p.32). Second, the theoretical underpinnings of situated cognition
provide a conceptual framework to make sense of student autonomy and teacher agency in
a mobile learning trajectory for it is impossible to discuss sensibly the changing roles of
teachers and students without making reference to the contextual configurations and their
relations. Brown and Duguid [4] contend that, "One of the powerful implications of
situated learning is that the best way to support learning is from the demand side rather
than the supply side (p.8). This has strong implications on the role of the teacher and
the place for students autonomy in the learning process. On the design of the learning
environment, Choi and Hannafin [5] advocate a shift from organizing and sequencing
content to creating and designing environments that induce, then facilitate,
understanding (p.67). The functional role of the teacher here would be to allow an
unstructured space within the structured learning environment, whereby learners have the
liberty to exercise judgment, set new learning intent and pursue new inquiries/ interest
areas. On this note, Snow (1994 as cited in [6]) posits, we must not only learn in context
but also by context (p.84). Learners are empowered to respond to contextual changes
within the framework that guides their inquiry process.
Apart from a theoretical emphasis on learning in an authentic platform as against
decontextualized contexts, situated cognition also exemplifies the importance of
cultivating learning processes versus learning outcomes (p. 53) [5]. Here, it presupposes
two significant groups of players in the learning process. One is the teacher-student and
two, the student-student. First, it inherently implies a marked change in the role of the
teacher from a knowledge dispenser to a facilitator of students learning processes
(Bednar et al., 1991; Duffy & Jonassen, 1991; Winn, 1993 as cited in p. 67) [5]. And
facilitation can take on varying forms such as modeling, scaffolding, coaching and
guiding, collaborating, fading and via different technology-mediated cognitive tools and
resources (p. 63) [3]. It is also the onus of the facilitator to develop in the learners the
capacity and the ability to perform a knowledge and skill transfer across varying contexts.
Second, the individual learners interaction and collaboration with his/ her counterparts
form a critical phase of this collective learning enterprise. Thus, we recognize that student
autonomy is both enabled and shaped by a host of factors at play in a learning situation, of
which, the mediation of cognitive tools and collective cognition play a definitive role. The
teacher, thereby, assumes a more significant and complex function - a designer, a
mediator, and sometimes, a participant and collaborator of the learning enterprise. In a
nutshell, the fundamental role and responsibility of the teacher would be to design the
situation (p.5) [7]: engineering the learning environment and ensuring the availability and
the accessibility of technology-mediated cognitive tools and resources to bring about the
desired learning outcomes.

In the context of an inquiry-based


inquiry
mobile learning trail,, the purpose of this qualitative
research study is to examine how the agent of the teacher in learning design, in
appropriating technology-mediated
mediated cognitive tools to support collaborative mobile
learning, and in apportioning
tioning the measure of teacher presence can impact students
capacity to exercise autonomous learning.
learning We are also interested to identify which of
those above-mentioned elements (e.g., learning design, technology, facilitation, student
readiness) form the key determinants that shape student autonomy and to what measure.

2. Methodology
2.1 Research Background
Building on our previous research efforts to promote collaborative knowledge coco
construction on mobile learning trails,
tra
the present research study seeks to explore the
teachers role in facilitating more student autonomy in an inquiry-driven
driven mobile learning
trajectory leveraging on the rich affordances of the physical environment and technology
mediation. Aligned with the theoretical premises on situated learning, the mobile
m
learning
trail was designed with a focus on inquiry task-type
task type and space for more independent
learning via collective effort, technological mediation and teacher facilitation. Figure 1
illustrates the three-pronged approach - F.A.T (Facilitation, Activity Design, Technology),
a design framework we conceptualized to guide our trail design and implementation. This
holistic approach sees activity design as the primary driver of the other two equally critical
components - facilitation and technological mediation in the design of the learning
situation.

Figure 1: The 3-prong


onged Approach toward Mobile Learning Trail Design
2.2 Design Consideration
The trail tasks were co-designed
designed by the researchers
research
and collaborating teachers in the
integrated humanities department. The F.A.T. design framework guides the design process
(i.e., the overall focus on inquiry-based
inquiry
learning) and the range of activities primarily
determine the type of technological tools
tool and the features of web-based
based platform to
support the collaborative learning space, the facilitation and the communication process.
process
The design of the trail seeks to promote interdisciplinary inquiry-based
based discourse. This
move sees an unprecedented rich integration of History and Geography with the intent to
develop a holistic understanding of the body of cognitive and procedural knowledge and
skills in the integrated humanities. All
ll trail task questions point to an ultimate problem
statement where learners will need to see relationships across the findings to the various
task questions and eventually evaluate and synthesize shared knowledge and
understanding as a collective body.
body As illustrated in Table 1, trail tasks range from

performative (application) to knowledge generative and knowledge synthesis where the


findings and inferences should enable learners to respond to the overarching big question
on Sentosas role in British defence plan.
Table 1: Examples of Tasks at Fort Siloso Mobile Learning Trail
Big Question: What is the role of Sentosa in the Britishs big plan of defence?
Learning Task Type
Task Description
Station
A
Performative T1. Determine the direction of the guns using the iPad
compass.
Knowledge
T2. Describe the dimension of the tunnel and state its
Generative
purpose.
B
Performative T3. Locate the Stealth boat entering the harbor entrance.
Knowledge
T4. Explain why the previous artillery gun (Area A) and this
Generative
one are pointed in the same direction.
& Synthesis T5. Give reasons for the Britishs plan to locate the tower at
area B. Describe the role and purpose of the tower and the
guns.
Facilitation forms a critical determinant on students capacity for autonomous
learning. Three main modes of facilitation were put in place. First, a web-based platform
was designed and developed to host all trail activities with embedded apps (e.g., digital
map) for students to carry out their activities. Each team (max. 4 students) had an iPad,
and students were also able to upload their findings and collated artifacts onto their teams
respective web pages. Further, they were also able to communicate with other teams
through the feedback feature; giving comments and/ or suggestions. Second, trained
facilitators were assigned to all four activity-stations and teachers were also present to
monitor students progress. The physical presence serves mainly to provide students a
sense of assurance of aid should they be confronted with any major difficulties in an
outdoor situation. Facilitators were also briefed to adhere to more unstructured questioning
techniques to avoid constricting students capacity to leverage on situational resources in
the learning process. Third, apart from face-to-face facilitation, virtual teacher facilitation
was deemed necessary to provide immediacy of facilitation, which is of significance in a
mobile learning trail. Virtual facilitation via the public space feature in the web-based
platform affords teacher-student interaction and communication.
2.2.1 Participants
The collaborating institution is also a member of the FutureSchools@Singapore project.
The school leverages on its 1:1 computing initiative to create a technology-rich learning
environment and advocates a small class size of 2025. The mobile learning trail took
place at Fort Siloso, Sentosa Island in March 2011. Participants of the trail were two
classes of secondary one students - Class A (total class size = 20) included mostly highachieving students while Class B (total class size = 22) included mixed-ability students.
2.3 Data Collection and Analysis
The focused group interviews with teachers and students were rendered necessary to find
out about students and teachers perceptions of their inquiry-based mobile learning
experience, as well as, the actual occurrence of discourse amongst students and between
students and teachers. Post-trail focus group interviews with ten students (randomly

selected five from each of the two classes) and a one-to-one interview with six
collaborating teachers were held. The interview questions were semi-structured to solicit
feedback on three critical areas, namely, trail activities and collaboration efforts,
facilitation and technological mediation. In addition, relevant excerpts of data captured on
the web-platform of students interactions with other groups and the teacher facilitators
were also cross-examined to afford a more accurate insight into the research inquiry on
student autonomy and teacher agency in inquiry-based mobile learning.

3. Findings
Aligning with the conceptual framework on situated cognition, we examined and analyzed
the corpus of data in relation to the three key themes: (a) the activities in relation to
contextual elements (b) the mediating tools and (c) cognitive apprenticeship.
3.1 Students Narratives
3.1.1 Impact of Trail Design and Collective Efforts on Students Capacity for Autonomous
Learning
Students felt that task questions integrating Geography and History, opened up the
platform for further inquiries, generation of ideas and hypotheses; triggering a chain of
discussion. One participant voiced that the course of finding answers to the history
questions, had enabled them to see how geographical factors affected human decisions
why they did what they did and this provided the explanation to the historical events
back then. Students felt that there was better engagement with the abstract concepts and a
stronger sense of ownership of their learning during the trail without the constant physical
presence and supervision of the teachers.
Next, for the majority of the students, the inquiry-based approach lends itself better
for collaborative efforts over individual undertaking. On this note, Mark contended that if
you have multiple people working on the same problem so even if you get stuck, maybe
another person know how to do it . Another participant, Cayden concurred that the
very act of coming together to resolve an issue would inevitably give rise to a convergence
of human thinking resources. It promotes distribution of the think processes and gave rise
to the possibility of multiple perspectives on a common task and also assistance within
group when one is confronted with tougher task questions. Other participants felt that it
had increased their overall learning and thinking capacity when group converged again to
share their respective findings; this allowed them to learn from the explanation of the
fellow team mates who were better with a particular task and /or subject area. However,
not all students were optimistic about collaborative efforts, Tiffany recounted that, some
teams are not very receptive to ideas dont disturb usgo awaywe are doing our
work . Some teams conceived of inter-group collaboration as a form of interference;
delaying their work processes. Another reservation about collaborative learning lies in the
issue of reciprocity where students perceived that they could be short-changed by another
group, some of my silly group members go and give them the answers...and some groups
also, they have nothing to say. We attribute this to the gap in belief and actual practices.
That is, although students believe in the benefits of collective cognition, in practice, they
are more attuned to individual display of effort and performance.
3.1.2 Technological Mediation and Student Autonomy

The availability of the feedback and comment functions in the web-based platform
enabled the immediacy of facilitation and asynchronous collaboration. Isaac related their
teams experience with feedback and comments from other teams and how it gave them
some form of directions to re-attempt task questions and re-work through their own work
processes: theres was one point we got stuck then we resort to getting inspiration from
other teamswe try to understand how they got the answer and then incorporate it .
Likewise, the provision of feedback allows students a second chance to rethink through
their findings amid the rich physical affordances. Lucas recalled, lets say you make any
mistake, the teacher will send you a message. The alert function cum instant feedback
from teachers permitted a review of work processes. Immediacy of teacher facilitation
enlarges students independent learning space and thereby increases students capacity to
take control of their own learning journey in a mobile learning environment. Students are
given more autonomy to re-evaluate their initial findings and re-negotiate meaning.
3.2 Teachers Narratives
3.2.1 Activity Design Shapes Inquiry-based Learning Processes
At the cognitive level in relation to greater student autonomy in inquiry-based learning,
the collaborating teachers spoke on the significance of situated learning experiences to
foster autonomous learning and inquiry-based discourse. Mr. Loh explained, Ground
experiences can never be replicated important to view the authentic documents at the
site rather than online to develop empathy and multiple perspectives; giving space to the
internal voice, queries, hypotheses... Ms. Lee further added, the mobile device also
increased the proximity of the learners to the object of inquiry. This allows the students
an up close and personal encounter in the course of their inquiries. Mr. Seah commented
that the use of iPad allows students mobility and accessibility to any work tools and at
the same time, enables students to communicate and collaborate, in turn, teachers were
also able to capture their learning process.
On the notion of mobile learning and inquiry-driven curriculum, Ms. Lee felt that
mobile learning on its own, cannot be a stand-alone instructional tool. The basics should
be done in class ..., pre-trail lessonsand after that trail, post-trail. Further, Mr. Loh
stressed that it is needful to strengthen that link to review how this trail fits into the larger
picture of things, i.e., the entire curriculum. Hence, for students to take on an inquirydriven learning trajectory on the day of trail, teachers rendered preparatory work as a
necessary phase to equip and empower students to fully benefit from the mobile learning
scenario, taking charge of their own learning.
3.2.2 Technological Mediation and Facilitation as a Means to Enhance Student Autonomy
One of the means of increasing student autonomy and appropriating teacher involvement
was the provision of technological cognitive tools. Ms. Ang observed that the web-based
platform was a good communication platform for teachers to be engaged in the whole
collaboration - heighten interactions, give instant feedback and able to gauge students
progress, difficulties. To which, Mr. Yeh concurred, technology makes possible a virtual
facilitator, which made possible for students to receive guidance and on the spot to re-look
at their options. Mr. Yeh continued, the broadcast feature helped them stay on the right
track when they are pre-occupied or intentionally go off tangent. Another teacher
commented that interaction between groups made possible and between teachers and
students. Ms. Lee shared that the web-based platform has made it very viable for
learning, and enables interaction and the immediacy of facilitation. Layout also enabled

teachers to participate in the trail, and I was able to monitor all their responses from where
I was, without moving around.
Teachers felt that the design of the inquiry-driven trail, the provision of technological
tools and online facilitation make it possible for teachers to take a step back, observe how
kids work and give them a chance to maneuver their way through and reach the
destination give them more ownership of their own learning. Students take pride in their
work, remarked Ms. Teh. On the self-same note, Ms. Lee felt that there was less front
loading and transmitting of content - more self-directed learning. This experience changed
our roles as practitioners in the classroom even seeing their responses (if incorrect) and
even if they fail it doesnt make me anxious. The teachers felt that letting go of control
and the expectation for correctness and performance would liberate that space for greater
student autonomy in learning.
3.2.3 Impact of Teacher Presence on Student Motivation and Autonomous Learning
On the significance of teacher presence, be it virtual or physical, Mr. Yeh observed that
broadcast messages create the kind of atmosphere for students motivate them as they
know theres someone out there responding to them unlike cyberspace. Likewise, Mr.
Seah observed that the high level of engagement from the students had to do with the
manageability of tasks and the awareness that teachers are present (virtual and face-toface) to assist them should they stumble in the course of accomplishing their set goals.
This explains their eagerness and motivation moving from one learning station to another.
Mr. Yeh noticed another interesting phenomenon, they do not ask for answers through
the platform, physically yes, if they meet you, they ask for clues. He felt that virtual
facilitation renders a different form of teacher involvement, If answers seem general, try
to find out whether they understood task requirements, try to elicit more from them and
then guide them. Overall, teachers were positive that students obtained a greater sense of
ownership of their work processes in the learning trail experience. However, teachers
expressed the need for more concrete measures to be taken in order to increase student
autonomy and learning effectiveness, namely, soft skills in collective undertaking of tasks
and, questioning and inferential skills in engaging their counterparts during the
collaboration process.

4. Discussion and Conclusion


As exemplified from the findings, the measure of teachers presence and participation
ought to be weighed in the context of the learning situation, the prevailing socio-cultural
practices and the profile of the participants. In sum, student readiness to assume more
autonomy in inquiry-based mobile learning hinges on a variety of contextual factors. First,
the relevance of the learning activities ought to align with the desired learning outcomes,
as well as, the students profile and capacity for autonomous learning in such situated
learning contexts. And of significance would be to see the one-day learning trail as a
continuum of the entire curriculum- teacher support and scaffolds to increase student
capacity at autonomous learning commences at the very stage of the larger curriculum
design. For students to be able to benefit from autonomous learning in a mobile learning
context, autonomy support begins with the day-to-day instructional program. Students
confidence and comfort level to initiate and pursue inquiries, to make informed decisions,
and to conduct constructive interaction discourse is a gradual developmental process,
orchestrated by the teacher. Second, the vicarious presence of the teacher virtual and
face-to-face still has its rightful place. Students still need the endorsement of an

authoritative figure - the teacher. As Ellis (1993 as cited in [8]) puts forth, The teacher
thus has the role of a group member that has the option to qualify the dialogue through
questions (p. 22). And importantly, in an outdoor learning situation, students commence
at different levels of understanding in the collective meaning-making process before
converging at a common shared understanding. In the research study, the appropriation of
the measure of assistance and scaffolding was made possible based on the kind of answers
and feedback students pose on the web-based platform. And further, the teachers presence
as a participant and a collaborator serves as a form of facilitating and regulating the
exercise of autonomous learning on the part of the students. Third, student autonomy also
rests largely on the presence of the collective body of their fellow workmates and the
collaborative learning space. As evident in the narratives, students (esp. the high ability
group) felt that they were not very comfortable with the idea of collaborative efforts as
some still held on to the notion of individual merit and performance. Conversely, mixed
ability group was more open to sharing of ideas and findings. This inadvertently implied
that the socio-techno learning space to a considerable measure dictates student readiness to
become agents of their own learning.
To conclude, the analysis and synthesis of the data findings surface significant
implications pertinent to the design of the learning situation, the agent of the teacher in
shaping student autonomy in such situated learning contexts. The artful balance of teacher
agency and student autonomy requires a sound understanding of the content and context of
learning, and the appropriation of relevant technological mediated tools and facilitation.

Acknowledgements
This research is supported by the FutureSchools@ Singapore project under the National
Research Foundations (NRF) Interactive and Digital Media (IDM) in Education Research
and Development (R&D) Programme.

References
[1] Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning.
Educational Researcher, 18(1), 33-42.
[2] Brown, J. S., & Duguid, P. (1993). Stolen knowledge. Educational Technology, 33(3), 10-15.
[3] Choi, J.I. & Hannafin, M. (1995). Situated cognition and learning environments: Roles, structures, and
implications for design. Educational Technology Research & Development, 43 (2), 53-59.
[4] Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed)
Collaborative learning: Cognitive and computational approaches (pp.1-19). Oxford: Elsevier.
[5] Jonasson, M. (2011). Framing learning conditions in Geography excursions. International Education
Studies, 4 (1), 21 - 29.
[6] Sharples, M., Taylor, J. & Vavoula, G. (2005, October). Towards a theory of mobile learning. Paper
presented at 4th World conference on mLearning, Cape Town, Africa.
[7] Stefanou, C. R., Perencevich, K. C., Dicintio, M. & Turner, J. C. (2004). Supporting autonomy in the
classroom: Ways teachers encourage student decision making and ownership. Educational
Psychologist, 39(2), 97110.
[8] Uden, L. (2007). Activity theory for designing mobile learning. International Journal, Mobile Learning
and Organisation, 1(1), 81-102.

Using Kinect to Track Learning Behavior of


Students in the Classroom as Video Portfolio
to Enhance Reflection Learning
Chuan-Ling LIEN*, Ching-Yi HUANG, Chin-Yeh WANG & Gwo-Dong CHEN
Department of Computer Science and Information Engineering, National Central
University, Taiwan
*ayaling77@gmail.com

Abstract: We propose a novel approach to produce video portfolios in the classroom,


allowing students whole body, which was captured by a Microsoft Kinect appear in a rich
and context-sensitive background. The purpose of the study was to investigate this proposed
system could impact on various learning style students' cognition and metacognition. The
system makes the student have chance to examine and evaluate on one's own and then
promotes metacognition. Experimental results indicated a variety of learning styles do affect
the performance of learners, especially reflective and sense of style, metacognitive have a
significant upgrade capacity by using the system.
Keywords: Learning style, video portfolio, metacognition, authentic learning

Introduction
Video Portfolio has been widely used to improve the quality of supervision, evidence and
transfer in education. Video recorders can capture many interactions. Recorded video can be
assessed formatively with the goal of improved teaching, and they may also be assessed to
yield a summative score or judgement [2]. Students can also reflect more deeply on their
performance by reviewing a particular segment many times at a different location.
Metacognition is an important skill that could be improved by video portfolio. Black &
William [3] stated that the four pillars of the e-portfolio (metacognition, authentic tasks,
contextual feedback and student responsibility) seem to clear up the effectiveness of
formative evaluation. Metacognition is a skill of self-monitoring and reflecting on our own
mental process, and could contribute to critical-thinking skills. With good metacognition
skills, students can find out important information needed to solve a problem by analysis
and inference. They can also know their own demands and resources and consider using the
appropriate strategies at different times to effectively learn. With the support of video
portfolio, students can have opportunities to practice self-monitoring and reflect critically
on their experience.
Currently, video portfolio has been adopted to solve this problem. However, in the
classroom, it is difficult to include authentic context in the video recordings that represented
students performance. To ensure the recorded videos look good with rich and proper
background for review and reflection, the background needs to change accordingly
depending on what themes and students are going on.
We propose a novel approach to let students whole body, which was captured by a
Microsoft Kinect appear in a rich and context-sensitive scene. Students could watch
themselves appeared in the screen with a specific background that is designed by instructors,
and perform what they learnt. Later, students can review their own performance under the

guidance of their teachers or alone. Rich background with their figures of the video portfolio
could allow them perform reflection and enhance metacognition. Moreover, learning style
was taken seriously in recent years. When teaching style coincides with students learning
style, their learning could be easier and effective [19]. We also examine what kind of
learning style that students have could benefit from the approach significantly.
1. Related Work
1.1 Metacognition
Metacognition is extremely important, but really difficult to teach and assess. Part of the
reason why student cannot effectively use strategies and cannot become an active and
independent learner is the lack of metacognition. To become self-directed learners, learners
should develop their metacognition [22].
Flavell [10] found out from the study of human cognition that learners usually
displayed bad cognition due to their inability to monitor their cognition process and adjust it
later to match specific objectives. Metacognition is the ability of an individual to
self-supervise, in terms of their problem-solving ability, and is an important prerequisite
[13]. With good metacognition, students can clearly understand self-monitoring,
self-discipline, self-correcting, and self-assessment while they are solving problems.
Schraw, G. & Dennison, R.S. [20] divided the concept of metacognition into five steps
which could be improved individually:
Planning: Planning, goal setting, and allocating resources prior to learning.
Information Management Strategies: Skills and strategy sequences used to process
information more efficiently (e.g., organizing, elaborating, summarizing, selective
focusing).
Comprehension Monitoring: Assessment of ones learning or strategy use.
Debugging Strategies: Strategies used to correct comprehension and performance
errors.
Evaluation: Analysis of performance and strategy effectiveness after a learning
episode.
Nevertheless, metacognition is situation dependant. Each individual is required to
manage specifications while considering their strategy and adjusting their train of thought
toward the situation that their strategy is going to used [13]. The video portfolio of what they
performed could be useful if students can perform in authentic learning environments.
1.2 Authentic learning
On the other side, authentic tasks are vital to the e-portfolio that could result in the
effectiveness of formative evaluation [3]. Suchmon [21] found that humans' cognitive
activity is limited by the social context of their activities. However, knowledge learned in a
school system is different from the cognitive ability that is obtained from reality [13]. The
true meaning must be realized via practical actions, and students should know how to solve
unknown problems by using environmental resources [14]. Therefore, concepts and rules
must be understood through real experiences, which is not easy to be fulfilled in the
classroom.
Similarly, metacognition should be cultivated in authentic environment. Brown,
Collins & Duguid [4] pointed out that people invent effective strategies to solve problems in
situations due to interacting constantly with a specific situation.
When students perform situational tasks, they can reflect on their learning results after

learning through video portfolios. From video tape to digital memory, video recorders have
been widely used to improve teaching and learning in the classroom. To change the method
of cognition and enhance learning effect, authentic learning environments should be added
to the classroom, allowing students perform in authentic environments and producing video
portfolios for enhancing metacognition.
1.3 Portfolio
Portfolios are widely used to help students learning. Portfolio assessments of learning
outcomes are more realistic and dynamic presentation than the written records examination,
and the students can know their results clearly via portfolio [13]. Albert Bandura's Social
learning theory mentioned that learners change individual behavior through observing and
imitating others. Learners can better understand and analyze their advantages and
disadvantages via comparing themselves to others. Through portfolios, teachers and parents
can also view the child's advances, and give them timely help [5],[15],[16],[23]. In short,
Paulson Leon & Paulson Pearl [17] mentioned that promotion of reflection, self-evaluation,
self-understanding and the ability of metacognition is the most important objective that
learning portfolios could be.
Portfolio is individualized data [1], purposeful data collection [18], visual process
presentation [9], and authentic performance results [18]. Chang [6] revealed that the system
of network learning files can produce many effects such as helping to control the learning
process, reflecting on the advantages and disadvantages, and enhancing the growth,
progress, and benefit of learning. Currently, video portfolio has also been frequently utilized
to assist students learning and its effectiveness has been proved hugely. This study try to
enhance the way that video portfolio was taken in the classroom. Students performances
and the according backgrounds are mixed together and recorded simultaneously as the video
portfolio. Comparing to the portfolio recorded by DVRs, the video portfolio which is
generated by our proposed approach could provide students authentic context.
1.4 Learning Style
Besides examine the impact that the proposed idea could be, this study also wants to realize
if there is any difference while students with different learning styles. Here gives a brief
description of learning style. Learning styles are various approaches or ways of learning.
They involve educating methods that are presumed to allow individual to learn best. Several
dozen learning style models have been developed. One of the most common and
widely-used categorizations of the various types of learning styles is Fleming's VARK
model [12]: (1) visual learners have a preference for seeing, (2) auditory learners best learn
through listening, and (3) kinesthetic or tactile learners prefer to learn via
experiencemoving, touching, and doing. One learning style is neither preferable nor
inferior to another, but is simply different, with different characteristic strengths and
weaknesses [7].
Category of learning style allows teachers to prepare classes to satisfy students
preferences. Students can also use the model to identify their preferred learning style and
maximize their educational experience by focusing on what benefits them the most. The
proposed method to produce video portfolios might be only suitable for students with
specific types of learning style. This study also statistically examines what kind of students
could benefit from the new approach.

2. System Design and Implementation

2.1 System Architecture


In order to record students learning activities, we used an L-shape authentic learning
platform with a vertical screen to display situation and a horizontal interactive touch table to
allow students to immerse into situation and perform interactive tasks.
When we put teaching materials into the L-shape platform to do authentic learning,
Kinect let the image of students into the vertical screen, and students can see their true
performance. While learning activity to proceed, HyCam2 (screen recording program) is
called additionally to videotape the screen. In this way, we have the materials for students to
review and reflect after class.
To make students reviewing those recorded video portfolios easily, learning objectives
and the Learning adventure book videos were cut into small episodes. Besides, those videos
and meta-information were stored in a database. Then, we used the programming languages,
PHP and MySQL, to store those data and provide students personalized review.

Fig.1 L-shape platform using situation

Fig. 2 The system using situation

2.2 Instructional Design


Our teaching material is a designed situational game which makes students feel immersive
in learning and interact frequently with their peers. Under this learning environment,
teachers and students can carry out the group activities on the both sides of the platform.
Through the help of teachers and the cooperative learning with peers, students can engage in
learning activities and gain learning outcomes. The game allows students to view their own
learning process to enhance metacognition. Here we choose English as second language
learning as objectives, and integrate the materials into game-based learning activities.
After students complete the learning activities, we classify and cut the videos
propriately, and let students have their personal video portfolios. Those videos are divided
into three parts: evaluation, films, learning adventure book. Evaluation is categorized by
learning objectives of self-evaluation and self-performance. Films are related to learning
objectives, personal films and situations. The personalized video portfolios present the films
of students plus narrative description. Following are the steps that students review the video
portfolios. Firstly, they login the system via entering their own accounts and passwords.
There is an easily understood instruction which contained all the evaluation, films, and
learning adventure book on the left side of the screen. We reminded students to fill out the
learning objectives of self-evaluation to realize how much they have learned in that
objective. If there is inadequate, the system will recommend students intimately to review
the video portfolio again. The demonstration of teachers and outstanding films of peers will
be marked with special tags for students' references. Secondly, students watch their own

videos, then complete the assessment, evaluate and reflect on their academic performance.
After finished the two stages, students could choose any video which they were interested in
or want to enhance. They can click the Learning adventure book to review the content they
have learned quickly and completely. Finally, if students feel that the review and reflection
completed, they can end the learning.
2.3 The design to boost metacognition
The experience of metacognition usually happened in the moment of cognitive failure. In
the process of trying to solve the problems, we sometimes would know whether our
performance was smoothly or not [11]. If students can make reflection to realize their own
shortcomings, they would find out that there were many shortcomings can be improved. In
this way, students can trigger the learning motivation. However, students usually forget or
neglect their personal performance due to tension. Therefore, we should have a complete
record to allow students review their personal performance.
The system could provide a possible solution. The system used the screen recording
program to record the performance when a student carries out activities. In this way, we
dont be afraid not filming to the front of students and dont need extra-manpower to control
the camera. Students can use the system after class on their own computers. The films of the
record of student carrying out activities will import into the system. We utilize the video
categories to meet the demand of viewing learning performance. Students can click their
peers films to exchange study, or click the sample films of teacher in the learning
objectives to review again. Furthermore, we make books of Learning adventure book by
their learning record in situational stage. Using these books, it makes students clarify the
overall concept. Besides, we added the learning adventure book and evaluation into our
tools which included students' self-reflection and feedback of teachers.
Through the image recording and complete planning, the system brings some new
possibilities for situational learning.
Self-assessment of the effectiveness of learning: By fill out the self-assessment of
learning objectives," students can reflect on their learning situation, and more
understanding of what objectives should be completed, and then click on the "learning
target film to strengthen their own lack of part. In this way, students can promote their
metacognitive skills of self-reflection, self-evaluation, and self-understanding.
Self-monitoring:Personal videos provide students watch their performance and
observe something they did not notice before and the part that they did not perform
well on. Fill out the Assessment of self-expression assists students to view their
performance again, and to stimulate students' understanding and reflection.
Imitation and learning: Students see videos of other peers can learn the advantages of
others, strengthen the impression of learning content, reflect on their own whether the
same needs to be improved.
Integration concept: An learning adventure book that presents the whole activity in a
storybook style not only increases the lessons appeal but also enhances the students'
level of understanding. The student is the protagonist, providing them with a
spectacular sense of actually being there. It can also be provided for the parents, so they
could know what their children have done.

3. Experiment and Results


Here we would like to examine whether the proposed system could impact on student's
cognition and metacognition. Besides, we also examine what kind of learning styles that

students processed could significantly benefit from the proposed system.


3.1 Procedures
We designed an experiment to find out the outcome of our system. The experiment was held
at a university in Taiwan. The subjects were 15 university students. Before conducting the
teaching activities, they first do a Metacognitive Awareness Inventory (MAI) scale of the
Regulation of Cognition to detect their habits and the degree of metacognitive awareness
[20]. This is a 35-question questionnaire. In addition, there is another questionnaire
(Learning Styles) with 44 questions to understand the students' behavior and habits of
learning [8]. Then the subjects conducted a 40-minute learning activity. Five days later, we
invited them to use the system again. Before using the system we provided 5-minute
introduction to the system and experimental procedures. Next, we let them use the system
for 20 minutes. After using the system, the subjects conducted the same MAI questionnaire
that was to understand the effectiveness of learning after using the system and compare the
correlation between effectiveness of metacogniton and learning style. Besides, we let them
write satisfaction questionnaire to realize which functions are efficacious. Besides, we also
videotaped the whole learning process to gain the observational data.
3.2 Result Analysis
Table.1 Learning styles and its corresponding point of pretest and posttest questionnaire
Planning

Information
Management
Strategies

Comprehension
Monitoring

Debugging
Strategies

Evaluation

Overall

M
22

P
0.143

M
18.7

P
0.532

M
20.5

P
0.346

M
23.9

25.7
21.2

0.005*

17.5
19.4

0.313

21.8
19.8

0.086

25.1
23.1

26
21.4

0.001*

20.2
19.2

0.8

22.2
19.9

Active

pre

M
24

P
0.788

M
34.7

Reflective

post
pre

24.5
21.8

0.03*

36.2
33.2

Sensing

post
pre

26.3
22.8

0.023*

38.4
33.5

post
pre
post
pre
post
pre
post
pre

26.4
22
20.5
22.9
24.7
22
26
22.5

post
pre
post
pre

16
22.5
24.6
22.7

post

25.6

Intuitive
Visual
Verbal
Sequential
Global
Overall

0.742
0.141
0.228
0.05*

0.537
0.044*

38
35.5
34.5
34.6
37.4
31.5
38
33.3
37.2
33.8
37.5
33.8
37.5

P
0.458

0.021
*
0.019
*
0.874
0.151
0.158
0.109

0.187
0.033
*

26.5
22.5
22
22.1
25.5
20
26.8
21.2
25.9
22.5
25
21.5
25.9

0.874
0.023*
0.029*
0.005*

0.178
0.001*

19.4
19
17.5
19.2
18.8
19
20
18.9
18.9
19.6
19.5
19.1
19.1

0.656
0.676
0.68
1

0.907
0.878

22.5
21
19
20.9
22
17.8
22.3
19.4
21.8
21.3
22.3
20.1
21.6

0.009
*
0.626
0.216
0.131
0.037
*
0.741
0.041
*

26.6
23.4
26.6
24.0
22.7
23.9
25.7
22.1
26.6
23.1
26.0
24.0
25.8
23.4

P
0.554

0.011
*
0.01*

0.144
0.103
0.144
0.046
*
0.479
0.028
*

25.9

*stand for significant different

Table 1 show that five abilities in the questionnaire: planning, information


management strategies, comprehension monitoring, debugging strategies, and evaluation.
A paired t-test was used to examine the effect. There is a significant difference between
pretest and posttest in Comprehension Monitoring (p=0.001, p<0.05) indicating that
assessments of learning strategies were effective after using the system. There is a
significant difference between pretest and posttest in evaluation (p=0.041, p<0.05)
indicating that their performance and ability of strategy analysis were improved after using

the system. There is a significant difference between the pretest and the posttest in planning
(p=0.044, p<0.05) indicating that ability of goal setting and planning was improved after
using the system. There is a significant difference between pretest and posttest in
information management strategies (p=0.033, p<0.05) indicating that students can improve
their abilities of organization and information reconstruction effectively. However, there is
no significant difference between pretest and posttest in debugging strategies indicating this
system cannot assist students in correcting their strategies.
In five parts and the overall effectiveness of metacognition compared with learning
styles, general discussion excluding no significant differences in the debugging strategies.
The data show that sensing style has significant differences in all metacognitive skills.
Reflective styles for planning, information management, strategies and comprehension
monitoring have significant differences. Sequential style for the planning, comprehension
monitoring and evaluation has significant differences. Visual and verbal style only in
comprehension monitoring has significant differences.

4. Discussion
We find that the system has significant differences in information management strategies.
We explore from the data of MAI and system usefulness questionnaire, and we find that
there is effective assistance for students via teaching by situational games and films of peers
who perform well or teachers demonstration. Nevertheless, the significance of evaluation is
barely satisfactory, exploring several students whose achievement is relatively low in this
part. We find that they are not accustomed to reflect on their performances.
About learning style, experimental results show that there is mostly significant in
reflection. This means that students who usually had a habit of independent thinking and
judgment are able to exert the skills of self-reflection and learning from peers. Students with
Sequential style have significant effectiveness as a whole. This may be because the system
designed as entire teaching content but planned completely as different learning objectives.
This makes students who accustom to understand and learn individually can easily get
started. Students with Intuitive style have almost no significance in all respects. This may be
that these films are too much alike to these students who dont like repetitive things.
Therefore, they wont click these films respectively to do comparative learning.
Furthermore, there is a particular finding. The Sensing style has quite significance in all
respects except in the part of Debugging Strategies. The part of Debugging Strategies has
quite demand of thinking and changing the learning strategy. Wherefore students who are
thoughtful (EX: students with Reflective style), their performance will be relatively better.
According to the satisfaction questionnaire of system, students who have been learning
styles with Sensing and Visual have relatively high preference for the system. This is
obviously due to this system that is suitable for their usual customary way of learning, so
there is no obstacle in the learning.

5. Conclusion
We propose a novel approach to produce video portfolios in the classroom, allowing
students whole body, which was captured by a Microsoft Kinect appear in a rich and
context-sensitive background. Students can review their own performance under the
guidance of their teachers or alone to enhance their cognition or metacognition.
Experimental results show that the proposed approach could significantly enhance students'
metacognition on Reflective, Sensing and Sequential styles. In the future, we would suggest

that providing references to students whose self-reflection ability is relatively inadequate.


We also suggest that adding some interactive elements in the system to make students not
lose their attention while reviewing these video portfolios.

Acknowledgements
This work is partially supported by the National Science Council, Taiwan, under the
grant number NSC 101-2631-S-008-001

References
[1] Collins, A. (1992). Portfolios for science education: Issues in purpose, structure, and authenticity. Science
Education, 76, 451-463.
[2] Berg M. H., & Smith J. P. (1996). Using Videotapes to Improve Teaching. Music Educators Journal,
82(4), 31-37.
[3] Black, P., & William, D. (1998). Inside the white box: raising standards through classroom assessment.
Phi Delta Kappa, 80(2), 139148.
[4] Brown, J. S., Collins, A. & Duguid P. (1989). Situated Cognition and the Culture of Learning.
Educational Researcher, 18(l), 32-42.
[5] Carpenter, C. D. et al. (1995). Portfolio Assessment: Opportunities and Challenges, Intervention in School
and Clinic, 31, 34-41.
[6] Chang, C. C. (2001). A Study on the Evaluation and Effectiveness anAlysis of Web-based Learning
Portfolio, British Journal of Educational Technology, 32, 435-458.
[7] Felder R.M., & Brent R. (2005). Understanding student differences, Journal of Engineering Education,
94(1), 57-72.
[8] Felder, F.M., & Soloman, B.A. (1991). Index of Learning Styles. North Carolina State University.
[9] Fischer, C. F., & King, R. M. (1995). Authentic Assessment: A Guide to Implementation. The Practicing
Administrator's Leadership Series, Corwin Press.
[10] Flavell, J. H. (1976). Metacognition aspects of problem-solving,. In L.B. Resnick(Eds), The nature of
intelligence (pp. 231-235). Hillsdale, NJLawrence Erlbaum.
[11] Flavell, J. (1979). Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental
Inquiry. American Psychologist, 34, 906-911.
[12] Fleming, N. D. (2001) Teaching and learning styles: VARK strategies. Christchurch, New Zealand: N.D.
Fleming.
[13] Huang, C.Y. et al. (2011). Using "Learning Video Portfolio" to Enhance Students' Metacognition in
Authentic Learning. International Conference on Advanced Learning Technologies. Rome, Italy.
[14] Lave, J & Wenger, E (1991). Situated learning: Legitimate peripheral participation, Cambridge NJ:
Cambridge University Press.
[15] Lambdin, D. V., & Walker, V. L. (1994). Planning for Classroom Portfolio Assessment, Arithmetic
Teacher, 41, 318-24
[16] Moss, P. A. (1992). Shifting Conceptions of Validity in Educational Measurement: Implications for
Performance Assessment Review Of Educational Research, 62, 229-258.
[17] Paulson, F. L., & Paulson, P. R. (1994). Assessing Portfolios Using the Constructivist Paradigm
[18] Paulson F. L., Paulson P. R., & Mayer C. A (1991). What makes a portfolio a portfolio. Educational
leadership, 60-63.
[19] Renzulli, J. S. (1994). Schools for talent development: A practical plan for total school improvement.
Mansfield Center, CT: Creative Learning Press.
[20] Schraw, G., & Dennison, R.S. (1994). Assessing metacognitive awareness. Contemporary Educational
Psychology, 19(4), 460475.
[21] Suchmon, L. A. (1987). Plans and Situated Actions: The Problem of Human-Machine Communication.
New York: Cambrige University Press.
[22] Turner, G. Y. (1989). Promoting learning autonomy: helping students become independent learners,
Reading Horizons, 29(2), 110-116
[23] Nolet, V. (1992). Classroom-Based Measurement and Portfolio Assessment. Assessment for Effective
Intervention, 18, 5-26.

A Science Learning Environment using a


Computational Thinking Approach
Satabdi BASUa*, John S. KINNEBREWa, Amanda DICKESb, Amy Voss FARRISb,
Pratim SENGUPTAb, Jaymes WINGERa & Gautam BISWASa
a
Department of EECS/ISIS, Vanderbilt University, U.S.A.
b
Mind, Matter & Media Lab, Vanderbilt University, U.S.A.

Abstract: Computational Thinking (CT) defines a domain-general, analytic approach to


problem solving that combines concepts fundamental to computing, with systematic
representations for concepts and problem-solving approaches in scientific and mathematical
domains. We exploit this trade-off between domain-specificity and domain-generality to
develop CTSiM (Computational Thinking in Simulation and Modeling), a cross-domain,
visual programming and agent-based learning environment for middle school science.
CTSiM promotes inquiry learning by providing students with an environment for
constructing computational models of scientific phenomena, executing their models using
simulation tools, and conducting experiments to compare the simulation behavior generated
by their models against that of an expert model. In a preliminary study, sixth-grade students
used CTSiM to learn about distance-speed-time relations in a kinematics unit and then about
the ecological process relations between fish, duckweed, and bacteria occurring in a fish
tank system. Results show learning gains in both science units, but this required a set of
scaffolds to help students learn in this environment.
Keywords: Computational thinking, agent-based modeling, visual programming,
simulation, scaffolding, inquiry learning

1. Introduction
Computational thinking (CT) describes a general analytic approach to problem solving,
designing systems, and understanding human behavior [10,15]. It draws on fundamental
concepts in computing and computer science to support practices (e.g., problem
representation, abstraction, decomposition, verification) that are central to modeling,
reasoning, and problem solving in scientific and mathematical disciplines [10,11].
Developing scientific practices and problem-solving skills requires sustained,
immersive educational experiences, which can be implemented as learning progressions for
science in K-12 classrooms [7,9]. CT can support such progressions in the K-12 science
curricula by integrating its inherent domain generality with support for domain specific
representations, reasoning, and analysis of real world problems across multiple domains
[10,11,15]. Balancing and exploiting this trade-off between domain-generality and
domain-specificity, however, presents an important educational design challenge.
Previous studies on integrating programming with K-12 science have pointed out a
variety of similar challenges [3,4]. Curricula that have effectively addressed these
challenges take advantage of reflexivity, which hypothesizes that learning programming in
concert with concepts in another domain can be easier than learning each separately [6].
Several researchers have shown that programming and computational modeling can serve as
effective vehicles for learning challenging science and math concepts [2,5,8]. Further, many
programming and CT concepts parallel important aspects of STEM (Science, Technology,

Engineering, & Mathematics) learning. For example, the creation of coherent, formal
representations of scientific phenomena and mathematical representations of scientific laws
are similar to object-oriented programming concepts of encapsulation, abstraction, and
generalization. Conversely, the biological concepts of taxonomy and inheritance are the
inspiration for class inheritance concepts in programming.
Leveraging the synergy between CT, science, and math learning, we have designed the
Computational Thinking in Simulation and Modeling (CTSiM) learning environment and
are implementing it using a learning-by-design progression. The learning environment
combines visual programming and simulations to allow for flexible iterations between
initial instruction in the science topic; modeling the appropriate entities and processes using
a visual, agent-based computational framework; simulating and studying the behavior of the
model; using explanation and argumentation skills to understand and verify the model; and,
applying the developed model and science understanding to problem-solving tasks. The
learning progression can be implemented over a progression of topics.
This paper presents the rationale for the learning environment design and a two-unit
science sequence (kinematics and ecology) to demonstrate our computational thinking
approach across domains. We describe an initial CTSiM study with 6th-grade students in a
middle Tennessee public school. The results demonstrate the effectiveness of our approach,
supporting the premise that students conceptual understanding of science topics improves
after engaging in CT-based curricular units. Finally, we present the categories of scaffolds
used in the study and discuss the role these scaffolds likely played in the students learning.

2. The CTSiM learning environment and curricular units


To support learning-by-design activities, CTSiM comprises three primary components [11]:
(1) the Construction (C) world, (2) the Enactment (E) world, and (3) the Envisionment (V)
world. The C world provides a visual programming interface where students build
computational models for the science topics they are learning. It includes a library of visual
primitives corresponding to agent actions, sensing conditions qualifying agent actions, and
controls for regulating the programs flow of execution (e.g., conditionals and loops). Each
visual primitive is defined in terms of an underlying domain-independent computational
primitive. Students drag and drop these primitives and arrange them spatially to generate
their computational models, as illustrated in Figure 1 for an ecology unit.

Figure 1: The Ecology unit Construction world with a breathe procedure for fish agents

In the E world, illustrated in Figure 2, students set initial parameter values and observe
the Netlogo-based simulations corresponding to their models. NetLogo visualizations and
plotting functions [15] provide students with a dynamic, real-time display of how their
agents operate in the microworld simulation, thus making explicit the emergence of
aggregate system behavior (e.g., from graphs of a species population over time).
The V world provides students the opportunity to perform systematic experiments to
compare their models behavior against behavior generated by an expert model. This
side-by-side comparison of plots and microworld visualizations for the two models makes it
easier for students to investigate and revise their models. With proper scaffolding, we
believe that the overall process of model construction, analysis, comparison, and refinement
will help students gain a better understanding of science phenomena, mathematical concepts
(e.g., rates), and computational constructs and methods.

Figure 2: A screenshot of the Enactment world for the Ecology macro-unit


2.1 Kinematics Unit
Activities in the Kinematics unit were divided into three phases [12, 13].
Phase 1: Turtle Graphics for Constant Speed and Constant Acceleration We
introduced students to programming commands by showing them how to manipulate
different elements in the user interface. Then, we asked them to generate algorithms to draw
simple shapes (squares, triangles and circles) to familiarize them with programming
primitives like forward, right turn, left turn, pen down, pen up and repeat.
Next, we asked students to modify their algorithms and generate spiraling shapes in which
each line segment is longer (or shorter) than the previous one. This exercise introduced
students to the speed-up and slow-down commands, and it gave them a chance to
explore the relationship between speed, acceleration, and distance.
Phase II: Conceptualizing and re-representing a speed-time graph - In this activity,
students generated shapes such that the length of segments in the shapes were proportional
to the speed in a given speed-time graph. Figure 3 depicts the speed-time graph provided to
all students, along with a sample student output where the initial spurt of acceleration is
represented by a small growing triangular spiral, the gradual deceleration by a large
shrinking square spiral, and constant speed by a triangle. The focus was on developing
mathematical measures from meaningful estimation and mechanistic interpretations of the
graph, and thereby gaining a deeper understanding of concepts like speed and acceleration.
Phase III: Modeling motion of an agent to match behavior of an expert model - For this
activity, students modeled the behavior of a roller coaster as it moved on different segments

of a track: up (pulled by a motor), down, flat , and then up again. Students were first shown
the simulation results produced by an expert roller coaster model in the V world. Then,
they were asked to conceptualize and build their own agent model to match the observed
expert roller coaster behavior for all of the segments.

Figure 3: Acceleration represented in a speed-time graph and sample student output


2.2 Ecology Unit
For the Ecology unit students modeled a closed fish tank system in two steps: (1) a
macro-level semi-stable model of the behavior of fish and duckweed; and (2) a micro-level
model of the waste cycle with bacteria. The macro model included several key relations: (1)
the food chain and respiration cycles of the fish and duckweed, (2) the macro-level elements
of the waste cycle (fish produce waste, duckweed consume nitrates), and (3) the
reproduction of duckweed.
The non-sustainability of the macro-level model (the fish and the duckweed gradually
died off), which students built first, helped them reflect, which, in turn provided the
transition to the micro model. When prompted to think about why the system was not
self-sustaining, students could identify the continuously increasing fish waste as the culprit,
and this provided the trigger to introduce the role of bacteria in the system.
At the micro level, students modeled the waste cycle with bacteria and the related
chemical processes that converted the toxic ammonia in the fish waste to nitrites, and then
nitrates, which sustained the duckweed. The graphs generated from the expert simulation
helped students understand the producer-consumer relations: (1) Nitrosomonas bacteria
consume ammonia and produce nitrites; (2) nitrites are consumed by Nitrobacter bacteria to
produce nitrates which provide food for the duckweed.
2.3 Sequencing of units and activities
Within each unit, the learning activities were designed to introduce students to (1) the
agent-based program structure, i.e., agents and their actions, and (2) the general
computational constructs, i.e., conditionals to model situation-based interactions, loops to
capture repeated agent behavior, and mathematical operations. The units provided a natural
sequencing in which students first learned to model and reason with a single agent in
kinematics and then went on to model multiple agents and their interactions in ecology. The
focus in the kinematics unit was on modeling real-world phenomena by generating
computational abstractions. In the ecology unit, students had the more complex task of
modeling multiple agent types (e.g., fish and duckweed at the macro level) and composing
several procedures to define the behavior associated with each agent type (e.g., swim, eat,
and breathe procedures for fish agents).

3. Method

The study was conducted with 6th-grade students from an ethnically diverse middle school
in middle Tennessee. 15 students worked on the system outside the classroom with
one-on-one guidance from members of our research team (Scaffolded or S-Group), while
the remaining 9 students in the class worked on the system in the classroom (Classroom or
C-Group) with some instruction from the researchers and the classroom teacher. The C
group also received individual help from the researchers if they raised their hand and asked
for help. The students were assigned to the groups by their classroom teacher. During the
intervention, five interviewers worked one-on-one with the S-Group students and provided
verbal scaffolds. In the C-Group, students received minimal one-on-one scaffolding. We
formulated two research hypotheses:
1. The intervention will help both groups improve their understanding of science concepts
as demonstrated by their pre-to-post-test learning gains.
2. The one-on-one scaffolding will help the S-Group learn more than the C-group.
As part of the design-based research, we collected and characterized the scaffolds provided
by the researchers to inform future system development.
All students worked on the three phases of the kinematics units before the ecology
macro and micro units. After completing the ecology micro model, the S group received an
additional scaffold: they discussed the combined micro-macro model with their assigned
researcher and were shown how the two models were causally linked to support
sustainability. Students were given the paper-and-pencil task of building a causal model of
the cycles, and then prompted to use this representation to explain the effects of removing
one agent on the stability of the cycle.
Students worked on the two science units in hour long sessions for three days each. On
day 1 of the study, we administered pre-tests for both units. Students worked on the
kinematics unit from day 2 to 4, and then took the kinematics post-test on day 5. This was
followed by work on the ecology unit from day 6 to 8, and the ecology post-test on day 9.

4. Results and Discussion


4.1 Learning Gains in Kinematics and Ecology
The Kinematics pre/post-test assessed students abilities to reason causally about
mathematical representations of motion and determined whether agent-based modeling
improved their abilities to generate and explain these representations. Specifically, the
questions on the test required interpretation of speed versus time graphs and generating
diagrammatic representations to explain motion in a constant acceleration field. For the
Ecology unit, the pre- and post-tests focused on students understanding of roles of species
in the ecosystem, interdependence among the species, the waste and respiration cycles, and
how a specific change in one species affected the others. Some of the questions checked
students declarative knowledge about the fish tank system (Declarative Knowledge Check
or DKC), others required performing Causal Reasoning about entities using the Declarative
Knowledge (CRDK), and a Transfer Question (TQ) required students to reason about the
carbon cycle. An example DKC question asked was For each of the following species in
the fish tank,: a) Goldfish, b) Duckweed, c) Nitrosomonas, d) Nitrobacter, mention the
chemical(s) it directly needs to stay alive, while an example CRDK question asked was
Your fish tank is currently healthy and in a stable state. Now, you decide to remove all
traces of nitrobacter bacteria from your fish tank. Would this affect a) Duckweed, b)
Goldfish, c) Nitrosomonas bacteria? Explain your answers.
Students in both groups had higher percentage scores on the kinematics pre-test than
the ecology pre-test, indicating that they had a better initial understanding of the kinematics

domain. However, pre-test scores and the mean TCAP (Tennessee Comprehensive
Assessment Program) science scores suggested differences in prior knowledge and abilities
of the S and C groups [significant differences (t=3.15, p<0.005) in mean TCAP science
scores between the two groups]. Hence we computed a repeated measures ANCOVA with
TCAP science scores as a covariate to study the interaction between time and condition.
There was a significant effect of condition on pre-post learning gains in ecology
(F(1,21)=37.012, p<0.001), and a similar trend was seen in kinematics (F(1,21)=4.101,
p<0.06). The plots in Figure 4 show that the S groups adjusted gains were higher than the C
group in both units.

19
S

18.5
18
17.5

17

16.5
Pre

Estimated marginal means of ecology


Estimated marginal
means

Estimated marginal
means

Estimated marginal means of kinematics

30
25
20
15
10
5

Post

S
S

C
Pre

time

Post

time

Figure 4: Comparison of gains between groups using TCAP scores as a covariate


Given the significant interaction between time and condition, we performed paired
t-tests on pre-to-post gains for each condition. Table 1 shows that the intervention produced
statistically significant gains for ecology unit, but not for the kinematics unit. However, for
both units, the S group, which received direct one-on-one scaffolding, showed higher
learning gains than the C group.
Table 1: Paired t-test results for Kinematics and Ecology pre and post test scores
Kinematics

Ecology

PRE (S.D.)
(max=24)

POST (S.D.)
(max=24)

t-value

P-value PRE (S.D.) POST (S.D.) t-value P-value


(2-tailed) (max=35.5) (max=35.5)
(2-tailed)

S-Group
(n=15)

18.07
(2.05)

19.6 (2.29)

2.699

0.017

13.03(5.35) 29.4(4.99)

C-Group
(n=9)

15.56 (4.1)

15.78 (4.41)

0.512

0.622

9.61(3.14)

8.664 <0.001

13.78(4.37) 3.402 <0.01

The lack of statistical significance in the kinematics unit may be attributed to a ceiling
effect in the students scores. One exception was a question that asked students to
diagrammatically represent the time trajectory of a ball dropped from the same height on the
earth and the moon. The students were asked to explain their drawings and generate graphs
of speed versus time for the two scenarios. The S group showed significant gains (p<0.0001)
on this question, while the C group showed an increasing trend, although it was not
significant (p=0.16).
For the ecology unit, the S-Group students gained on all categories of questions,
though all of the gains were not statistically significant. Table 2 reports normalized learning
gains (gain/maximum possible gain) by question category for both the groups. Significant
gains were observed on the DKC and CRDK questions, which can be attributed to an

increased awareness of the entities in the fish tank and their relations with other species. For
example, pre-test results indicated that the students did not initially know about the bacteria
and their roles. Though students in both groups were told about the role of bacteria during
the intervention, the supplementary causal-reasoning activity helped the S-group students
gain a better understanding of the interdependence among the species. The S groups gains
on the TQ were not significant due to a ceiling effect (most students had strong prior
knowledge about the carbon cycle). On the contrary, the C-Group gained only on the CRDK
questions, though less than the S-Group (F(1,21)=21.06, p<0.001). This can be explained by
the C groups minimal scaffolding and, especially, the absence of scaffolds targeted towards
causal reasoning.
Table 2: Normalized learning gains on categories of Ecology questions
S-Group normalized gains (P-value)
DKC
CRDK
TQ
.865 (<0.0001) .725 (<0.0001)
0.495 (.11)

C-Group normalized gains (P-value)


DKC
CRDK
TQ
0 (NA)
.192 (<0.01)
0 (NA)

4.2 Types of scaffolds provided and their effectiveness


A preliminary post hoc analysis of the recorded interviews was used to categorize the
different scaffolds provided to the S group, as summarized in Table 3. The SS helped
students become familiar with the different modeling primitives, the interface elements, and
the modeling task. The MS helped with specifying correct parameter types for the modeling
blocks, prompting reflection about how the commands in the model corresponded to the
observed simulation behavior, and describing how different functionalities of an agent
needed to be separated into different procedures. The TS were specific to the unit the
students were working on, and helped clarify the task for the student. The DS suggested that
students slow down the simulations to study differences between their models behavior and
that of the expert model using the simulations and plots as guides. The students had to
explain possible reasons for the observed differences, and then identify procedures and
primitives to modify in order to match expert behavior. The CRS prompted students to
reason in causal chains about different entities of the system to help understand global
system behavior. For example, as mentioned in Section 3, the S group investigated a
combined model of the fish tank and then re-represented the fish-duckweed cycle on paper
to reason about the roles of different species in the fish tank.
Table 3: Categories of scaffolds provided to the S-Group of students
Scaffold Category
System-use Scaffolds (SS)
Model-building Scaffolds (MS)
Task-based Scaffolds (TS)
Debugging Scaffolds (DS)
Causal Reasoning Scaffolds
(CRS)

Description
Familiarizes students with the UI and use of modeling primitives
Helps students correctly parameterize modeling primitives, modularize
code, and correlate models with the resultant simulations
Helps clarify the particular activity being worked on in a unit
Helps students identify differences between their model-generated
simulations and expert simulations, elicits explanations for the reason(s)
behind the differences and the methods for rectifying them
Encourages reasoning about system entities in causal chains to
understand global system behavior

Although the results in Section 4.1 illustrate the overall effectiveness of the
scaffolding, they do not identify the utility of particular scaffolds. However, some scaffolds
likely had a larger impact than others. For example, after adjusting for variations in TCAP
scores, there was a significant difference in performance between the S and C groups in the

Ecology Unit on the CRDK questions (F(1,21)=21.06, p<0.001), which may be a result of
the CRS provided only to the S-Group. Reasoning causally through multiple agent
interactions in the fish tank provided a global view of the ecosystem dynamics [1], which
was essential for understanding concepts of balance and interdependence covered by the
CRDK questions.

5. Conclusion
We have presented a learning environment which integrates computational thinking, visual
programming, and agent-based modeling and simulations to help middle school students
learn science across multiple domains. Our results indicate that the learning environment
helped produce significant learning gains, as measured by pre- and post-test scores, for both
the Kinematics and Ecology units. We also demonstrated the necessity of scaffolding and
some types of scaffolds required in such an environment. As next steps, we will integrate
such scaffolds into the CTSiM environment by building scaffolding tools and providing
feedback via a virtual mentor agent. Also, now that we have developed common
computational constructs for modeling in different domains, one of our design goals is to
help students realize and exploit these commonalities.

References
[1] Basu, S., & Biswas, G. (2011). Multiple Representations to Support Learning of Complex Ecological
Processes in Simulation Environments. In Proceedings of the 19th International Conference on Computers
in Education. Chiang Mai, Thailand.
[2] Blikstein P. & Wilensky, U. (2009). An Atom is Known by the Company it Keeps: A Constructionist
Learning Environment for Materials Science Using Agent-Based Modeling. Int Jounal of Computers for
Math Learning, 14:81119.
[3] diSessa, A., Hammer, D., Sherin, B., and Kolpakowski, T. (1991). Inventing Graphing: Meta
Representational Expertise in Children. Journal of Mathematical Behavior, 10, 117-160.
[4] Guzdial, M. (1994). Software-realized scaffolding to facilitate programming for science learning.
Interactive Learning Environments, 4(1), 1-44.
[5] Hambrusch, S., Hoffmann, C., Korb, J.T., Haugan, M., and Hosking, A.L. (2009). A multidisciplinary
approach towards computational thinking for science majors. In Proceedings of the 40th ACM technical
symposium on Computer science education . ACM, New York, NY, USA, 183-187
[6] Harel, I. & Papert, S. (1991). Software design as a learning environment. Constructionism, 5152.
[7] Krajcik, J., McNeill, K. L. & Reiser, B. (2008). Learning-goals-driven design model: Curriculum materials
that align with national standards and incorporate project-based pedagogy. Science Education, 92(1), 1-32.
[8] Kynigos, C. (2007). Using half-baked microworlds to challenge teacher educators knowing, Journal of
Computers for Math Learning, 12(2), 87-111.
[9] Lehrer, R., Schauble, L., & Lucas, D. (2008). Supporting development of the epistemology of inquiry.
Cognitive Development, 23 (4), 512-529.
[10] NRC (2010). Report of a Workshop on The Scope and Nature of Computational Thinking.
[11] Sengupta, P., Kinnebrew, J., Biswas, G., and Clark, D. (2012). Integrating Computational Thinking with
K-12 Science Education: A Theoretical Framework. Proceedings of the 4th International Conference on
Computer Supported Education
[12] Sengupta, P., & Farris, A.V. (2012). Learning Kinematics in Elementary Grades Using Agent-based
Computational Modeling: A Visual Programming Based Approach. In Proceedings of the 11th
International Conference on Interaction Design & Children, pp 78 87.
[13] Sengupta, P., Farris, A.V., & Wright, M. (2012). From Agents to Continuous Change via Aesthetics:
Learning Mechanics with Visual Agent-based Computational Modeling. Technology, Knowledge &
Learning. Vol. 17 (1 - 2), pp 23 - 42.
[14] Wilensky, U. (1999). NetLogo. http://ccl.northwestern.edu/netlogo/. Center for Connected Learning and
Computer-Based Modeling, Northwestern University, Evanston, IL.
[15] Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions
of the Royal Society, vol. 366, pp.

Student Learning Behavior in an


Unsupervised Learning Environment
Paul Salvador INVENTADOab*, Roberto LEGASPI, Rafael CABREDOab &
Masayuki NUMAO
a
The Institute of Scientific and Industrial Research, Osaka University, Japan
b
Center for Empathic Human-Computer Interactions, De La Salle University, Philippines
*inventado@ai.sanken.osaka-u.ac.jp

Abstract: Learning is commonly associated with knowledge transfer involving guidance


from a teacher. However, as people grow older they are expected to know how to learn by
themselves. In this research, we analyzed student learning in an unsupervised learning
environment, i.e., performing academic research, wherein students have complete control
over their learning thus requiring them to manage it. Transition likelihood metrics were used
to analyze the interplay between emotion, learning and non-learning related activities while
students did research. Several observations were seen from students learning in this
environment such as students experiencing cognitive disequilibrium but experiencing
disengagement faster. Non-learning related activities were also shown to have the potential
of motivating students to resume learning. Lastly, user-specific traits and context seem to
affect the interplay between learning and non-learning activities in an unsupervised learning
environment. This highlights the need to not only create general models to predict student
behavior but also user-specific models to allow future systems to provide appropriate
feedback in this environment.
Keywords: unsupervised learning environment, cognitive and affective state transition,
likelihood metric

Introduction
Learning is an essential part of human life. It starts from the very beginning of our lives
when we start learning simple actions and then eventually moving on to very complex
processes. We usually learn through knowledge transfer from a teacher whose role is
fulfilled by a parent, a sibling, a peer or a teacher in school. Over time however, we are
expected to learn on our own so that we no longer need constant guidance from teachers in
acquiring knowledge and applying it to solve the problems we wish to address. In most
cases, we become the teachers too and it becomes our turn to share our knowledge to others.
Many computer systems have been created to aid the process of learning so that in
cases when a human teacher is unavailable, a computer system can take on that role and
provide support for students while learning a particular topic [8]. Although these systems
model the students knowledge of the topic being learned, these do not consider how the
student interacts with elements outside of the learning environment and its effects on the
learning session.
In this work, we focused on unsupervised learning environments which we define as an
environment characterized by the absence of supervision from either a human or an
automated teacher. As a result, this environment requires students to manage their learning
apart from accomplishing their learning goals. Some elements that need to be managed
include the goals for the session, the amount of time spent in completing goals, the affective

states experienced, the transitions to non-learning related tasks and the avoidance of
distractions. Support is important in this kind of environment because students unable to
manage learning will most likely perform poorly or fail to complete their requirements.
Our goal for this work was to analyze and model students learning behavior in an
unsupervised learning environment so we can enable future systems to provide appropriate
feedback in such an environment.

1.

Related Work

Most existing systems designed to support learning provide cognitive and/or affective
feedback to students as they solve problems or explore environments generated by the
system [1][6]. Tracking both cognitive and affective states have allowed these systems to
model students more completely and also provided a better understanding of student
learning. This in turn was used as basis for designing and identifying the appropriate
feedback for students using these systems for learning.
DMello and Graesser [4] developed a model that used both cognitive and affective
elements to explain how students emotions transitioned while learning. This model can be
used to predict student behavior and provide appropriate feedback in cases when students
engage in activities or experience emotions that are not helping them learn. Students using
computer-based learning environments also engage in non-learning related activities and
certain cognitive and affective states have been reported to lead to off-task activities like
gaming the system and talking to seatmates [2][13]. Although learning environments
prevent or discourage students from engaging in non-learning related activities, students
learning by themselves can freely do so and are left to deal with them on their own. If they
are unable to manage their learning, they may learn less and may need to spend more time to
learn [9].
Not much work has been done to investigate the role of non-learning related activities
while learning especially when these are not purposely prevented or discouraged. We
believe that a better understanding will allow us to design systems that can leverage from its
benefits and try to reduce its negative effects.

2.

Data Gathering

We gathered data from four students who performed academic research. We considered this
an unsupervised learning environment because the students did not receive supervision and
they needed to manage their own learning. When students worked on their research, they
spent time developing their ideas, performing analysis and making conclusions. They
usually consulted or got suggestions and ideas from their supervisor but when they did
actual research work, it was their responsibility to identify their goals for the day, which
activity to prioritize and how much time to spend on an activity. Research required the
students to spend much time working and it was common for them to engage in
non-research related activities such as reading an email from their supervisor, exchanging
instant messages with a friend and viewing a video aside from others.
The participants consisted of one male undergraduate student, one male masters
student and two female doctoral students. All of these students were required to do research
as a requirement for their degrees and they commonly did their research with the help of a
computer using different applications such as web browsers, word processing software and
programming environments. Students were given the freedom to engage in both learning
and non-learning related activities either on or our outside of the computer. The annotation

software used and the methodology for collecting data are described in the following
subsections.
2.1 Sidekick Retrospect Software
The Sidekick Retrospect Software was developed by the authors to help students manage
learning and annotate their behavior. It encouraged the use of self-regulation strategies
specifically goal setting, self-monitoring and self-reflection [14]. When students began their
learning session, they first identified their goals for the session and inputted each one on the
interface. While learning, students were allowed to add more goals in case they found the
need to. The software also logged the applications used by the students and took screenshots
of their desktop and webcam feed throughout the session.

Figure 1. Interface for annotating learning behavior.


When the students ended the session, they were presented with a timeline representing
their entire learning session. The corresponding desktop and webcam screenshots were
shown whenever the mouse hovered over the timeline to help students recall what activity
they did at that moment. Students annotated their behavior by clicking and dragging the
mouse to select a time span and then identifying their corresponding intention, activity and
emotion label using the interface shown in Figure 1. Intention is the students purpose for
performing an activity which was either goal related or non-goal related. In this work, goal
related and non-goal related activities are synonymous to learning and non-learning related
activities because the goals defined by the students were for learning. An activity is what
students did at a certain point in time such as using the Google Chrome browser to search for
information, chatting using Yahoo Messenger, or reading a technical paper. Lastly, emotion
is the students experienced emotion while performing the activity. Two emotion sets were
used during annotation. When students performed goal related activities, they selected one
emotion from: delight, engagement/flow, confused, frustrated, surprised, afraid and neutral.
The list comprised of academic emotions which are commonly observed when students
learn [3]. When students performed non-goal related activities, they selected one emotion
from Ekmans six basic emotions namely angry, disgusted, sad, delighted, afraid, sad and
neutral [7]. A different emotion set was used for learning related activities because
academic emotions do not only refer to an affective state but are also indicative of certain
cognitive states which are useful for providing feedback. Academic emotions do not have

special meaning in non-learning related activities thus Ekmans basic emotions were
considered sufficient.
After annotation, students were asked to identify how much of their goals were
completed, which activities helped them complete their goals and how productive they felt
the learning session was. This helped students self-reflect to identify which activities were
helpful to learning or which were not and also for evaluating their performance.
At the end of the entire session, the software generated a log file containing the
intention, activity and emotion labels with their corresponding time stamp. Instances in the
log file were one second apart.
2.2 Data Gathering Methodology
Before data gathering, each participant was first given a short tutorial on how to use the
Sidekick Retrospect software and an explanation of the different annotation elements. They
were told that all activities they did which were related to their research goals for that day
would be considered goal-related and any other activity would be non-goal related. They
were also asked to indicate the different activities they did regardless if it was done on or
outside the computer. Lastly, each emotion was explained to them so they would know how
to differentiate them. In situations when they experienced more than one emotion, they were
asked to identify the most prominent one.
After the tutorial, the software was installed on the participants computers and they
were taught how to start and stop the data gathering module and create annotations in the
annotation module. They were then asked to use the software in five separate sessions,
wherein each session lasted around two hours. Students had the freedom to choose when,
where and which materials to use for learning.
At the end of every session, the students were asked to annotate their learning behavior
and then answer a survey. The survey contained questions regarding the students
realizations from their learning behavior. Data was gathered from each student over a period
of one week, with five two-hour sessions per day resulting in a total of 40 hours of data from
all participants.

3.

Analysis

Majority of the students used the software and annotated their learning behavior when they
worked at their own table in their respective research laboratories. This was where students
commonly did their everyday work wherein other research students were also present in the
same room interacting with each other and sometimes with the participant. The students
learned in the same setting as they always did and the software did not require them to
change their behavior while learning. Although students were asked to annotate their data,
this was done after the learning session. We believe that the methodology we used resulted
in naturalistic data.
The data showed that students spent about one hour and 30 minutes engaging in
learning related activities out of the entire two hour session. The participants set three goals
on average throughout the duration of the session. They rarely completed all of the goals
they defined but in most cases they completed at least 75% of one of the defined goals. This
further indicates that the data gathered from the participants was naturalistic because the
students did not avoid non-learning related activities nor favored it. It also shows that they
were capable of managing their learning behavior because they were able to identify their
goals, perform activities to accomplish their goals, and spend majority of their time in
learning related activities. Students experienced engagement, delight, boredom and

confusion over prolonged periods of time when learning. On the other hand, students
experienced delight and were in the neutral affective state over prolonged periods of time
when non-learning.
We further investigated the interplay between learning and non-learning related
activities using a transition likelihood metric to identify what caused students to shift
between these activity types. DMello et al. [5] introduced a metric to measure the
likelihood of transitioning from one affective state to another while students used an
intelligent tutoring system. However, in our research we did not only consider activities
related to learning but also non-learning related activities. We modified the likelihood
metric to include the type of activity done by the student which is shown in Eq. 1. The
equation measures the likelihood of transitioning from a state pi to state pi+1 where a state
consists of the activity intention I, which may either be learning or non-learning and an
emotion E, which may either be delight, engagement/flow, confused, frustrated, surprised,
afraid and neutral for learning activities and angry, disgusted, sad, delighted, afraid, sad and
neutral for non-learning activities. In our analysis, we were only concerned with transitions
into a different state since these described instances when certain factors affected the
student to move out of the current state. Assuming that a student performs the following
transition: State A State B State B State C, the prolonged state is treated as a single
State B
State C. The metrics
state resulting in the following transition: State A
resulting value ranges from 1 to -. When the value is above zero, it indicates a likely
transition with increasing likelihood as it approaches 1. A transition likelihood of zero
indicates that the transition is equal to chance and values below zero indicate that the
transition is less likely to occur compared to the base frequency of the succeeding state.
L( pi , pi +1 ) =

Pr( pi +1 | pi ) Pr( pi +1 )
(1 Pr( pi +1 ))

(1)

where : p =< I , E >


All possible transitions can be generated by combining all possible pairs resulting in a
14x14 matrix. To get a view of the students learning behavior, we averaged each of the
students transition likelihood values and used a t-test to identify statistical significance.
Table 1 shows the resulting matrix of likelihood values containing only the transitions that
were more likely than chance (i.e., L>0) with their corresponding p values in parentheses.
Likelihood values without p values indicate that there was only one instance of that
transition over all sessions from all students. Significant transition likelihood values (i.e.,
p<0.05) are highlighted. Transitions to the same state were omitted since the data used for
the likelihood matrix only contained state changes and not prolonged states.
As many researches have already shown, emotions play an important role in learning
which were also observed from the data we gathered. Specifically, there were highly
significant and likely transitions from engagement to confusion (L=0.15) which is
indicative of cognitive disequilibrium. Cognitive disequilibrium occurs when students
encounter errors or anomalies in what they are learning requiring them to either accept new
information or to alter their current understanding to return to an equilibrium state [12].
Cognitive disequilibrium is commonly attributed to learning and usually co-occurs with
feelings of confusion [10][11]. We also observed that students likely transitioned from a
bored learning state to a neutral non-learning state (L=0.40). Boredom is a state where
students have already disengaged from the learning activity and it is quite understandable
for them to simply shift to a non-learning activity most especially because they are not
prevented from doing so. A similar behavior was also seen in the shift from a confused
learning state to a neutral non-learning state (L=0.20). Although the transitions p value was

p=0.054 it can still be considered statistically significant. This may partially explain why
students did not experience hopeless confusion. DMello and Graesser [4] described
hopeless confusion as a state wherein students in a confused state are unable to resolve the
problem and eventually become frustrated. In the case of an unsupervised learning
environment, it seemed that when students were not able to resolve the cause of confusion,
instead of being frustrated they simply disengaged. On one hand, this is disadvantageous as
students would more likely disengage instead of solving the problem. On the other hand,
this could have led to lesser frustration, resulting in less stress and allowing them to possibly
have more motivation to continue learning at a later time.
Table 1. Transition likelihood matrix with corresponding statistical significance
EN

FR

EN

L
CO
0.15
(0.001)

BO

DE

NE
0.04
(0.390)

DI

NL
DE
SU
0.05
0.05
(0.502) (0.172)

FR

0.74
0.03
(0.558)

CO

0.03
(0.607)
0.03
(0.691)
0.14
(0.514)

BO
DE
NE

NL

DI
DE

0.41
(0.031)
1.00

0.11
(0.351)

0.25
(0.024)

0.02
(0.402)

0.02
(0.334)

0.09
(0.303)
0.08
(0.241)
0.50
(0.336)

0.20
(0.054)
0.40
(0.016)
0.26
(0.610)

0.01
(0.422)

0.35
(0.013)

SU
NE

NE
0.17
(0.037)

0.43

0.23

0.06
(0.454)

0.01
(0.895)

0.13
(0.106)
0.14
0.09
(0.212)

0.02
(0.334)

Intentions: L Learning; NL Learning; Learning emotions: ENgaged, FRustrated, COnfused,


BOred, DElighted, NEutral; Non-learning emotions: DIsgusted, DElighted, SUrprised, NEutral

As mentioned earlier, students were likely to resume learning as shown by the


transitions from a delighted and neutral state while non-learning to an engaged learning
state (L=0.34 and L=0.26 respectively). This can be interpreted in two ways. It can be that
students were capable of fending off distractions in an unsupervised learning environment
or, the presence of non-learning related activities reduced stress or helped students maintain
a level of motivation which helped them relax more and resume learning at a later time.
Lastly, the likely shift from a neutral learning state to an engaged learning state
(L=0.26) also indicated that students were not always engaged. Presumably, they first read
and understood content before getting immersed in the activity they were trying to
accomplish. Over time, they did become engaged which is the ideal state for learning.
Some students transitioned to and from other states however these were not observed in
other students or were only seen in a small number of sessions. This brought up two
concerns that needed to be considered when dealing with unsupervised learning
environments. First, personality or some other user-specific trait may have an effect on a
students learning behavior that was observed in one student but not others. A more in depth
data analysis showed that only one student transitioned to a neutral learning state before
experiencing other emotions while learning. Consider the following transitions observed
from the student while engaging in a learning activity: EN NE CO; EN NE BO;
EN NE EN; CO NE EN; BO NE EN. The transition to a neutral state indicates
that the student might have lost focus first then eventually experienced confusion or
boredom. While in the state of confusion or boredom, the student seemed to get back his
learning momentum first before transitioning to an engaged state again. Other students

transitioned directly between these emotions without passing through a neutral state.
Secondly, context may also play a big part in these transitions. For example, when a student
had an upcoming deadline, his transitions to a delighted learning state or even to
non-learning states were not as frequent as the time when he did not have a deadline.
Both of these concerns indicate that although we were able to get a good idea of how
non-learning states interplay with learning states in general, the differences in both context
and user-specific traits require a more user-specific and context-dependent analysis. It is
also important to observe learning behavior over a longer period of time to gather data on
more scenarios that students may encounter.
When the students made annotations using the software, they were also able to reflect
about their own learning behavior. The students answers to the survey reflected the
learning behaviors that were uncovered from the data. For example, one student said I got
distracted a lot but I needed it as it helps me get back into focus showing the value of
engaging in non-learning related activities. Another student said I felt bored before
transitioning from a goal related to a non-goal related activity which explained the high
likelihood of transitioning from a bored learning state to performing a non-learning related
activity.

4.

Conclusion and Future Work

In this research we defined an unsupervised learning environment as a learning environment


wherein students do not receive guidance from a human teacher or an automated system and
wherein they freely control their activities. Students learning in this environment need to be
capable of managing their goals and controlling the attention they give to non-learning
related activities.
An analysis of students learning behavior was conducted which showed the
occurrence of cognitive disequilibrium. This indicated that the environment was
challenging enough for the students to engage in learning. The accessibility of non-learning
related activities also seemed to cause students to disengage faster but also minimized their
frustration or stress. Non-learning activities also seemed to help students maintain or regain
motivation to continue learning.
The data showed that although students were capable of learning in this environment as
well as fend off distractions from non-learning related activities, there were still instances
when they needed help in managing them. Students commented that they spent too much
time in non-learning related activities and easily gave in to distractions, highlighting the
importance of support in such an environment.
The data gathered from the students showed transitions shared among all students and
also transitions which were unique only to some students. This indicated that user-specific
traits and context may have affected how students transitioned between learning and
non-learning activities. This requires not only a general model but also a user-specific
model to predict their learning behavior. A general model can be used as a basis for
designing feedback which is applicable to all students, while a user-specific model can be
used to adapt feedback over time and to provide more appropriate feedback based on their
traits and the contexts which they commonly learn in. The creation of such models can lead
to the development of systems that can provide support for students learning in
unsupervised learning environments. Such a system will also help students become more
equipped for learning on their own in the future.

Acknowledgements
This work was supported in part by the Management Expenses Grants for National
Universities Corporations from the Ministry of Education, Culture, Sports, Science and
Technology of Japan (MEXT) and JSPS KAKENHI Grant Number 23300059. We would
also like to thank all the students who participated in our data collection.

References
[1]

[2]

[3]
[4]
[5]

[6]

[7]
[8]

[9]
[10]
[11]

[12]
[13]

[14]

Arroyo, I., Cooper, D. G., Burleson, W., Woolf, B. P., Muldner, K., and Christopherson, R. (2009).
Emotion sensors go to school. In Proceedings of the 2009 Conference on Artificial Intelligence in
Education, pages 17-24, Amsterdam, The Netherlands. IOS Press.
Baker, R. S., Rodrigo, M. M., and Xolocotzin, U. E. (2007). The dynamics of affective transitions in
simulation Problem-Solving environments. In Proceedings of the 2nd international conference on
Affective Computing and Intelligent Interaction, ACII '07, pages 666-677, Berlin, Heidelberg.
Springer-Verlag.
Craig, S. D., Graesser, A. C., Sullins, J., and Gholson, B. (2004). Affect and learning: An exploratory
look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3):241-250.
D'Mello, S. and Graesser, A. (2012). Dynamics of affective states during complex learning. Learning
and Instruction, 22(2):145-157.
D'Mello, S., Taylor, R. S., and Graesser, A. (2007). Monitoring affective trajectories during complex
learning. In Proceedings of the 29th annual meeting of the cognitive science society, pages 203-208,
Austin, TX, USA. Cognitive Science Society.
DMello, S., Jackson, T., Craig, S., Morgan, B., Chipman, P., White, H., Person, N., Kort, B., Kaliouby,
E. R., Picard, R. W., & Graesser, A. (2008). Autotutor detects and responds to learners affective and
cognitive States. In Proceedings of the Workshop on Emotional and Cognitive Issues at the International
Conference on Intelligent Tutoring Systems. Montreal, Canada.
Ekman, P. (1992). Are there basic emotions? Psychological Review, 99(3):550-553.
Graesser, A. C., Conley, M. W., and Olney, A. M. (2012). Intelligent Tutoring Systems, volume 3 of
Applications to Learning and Teaching, pages 451-473. American Psychological Association,
Washington, DC.
Kirschner, P. A. and Karpinski, A. C. (2010). Facebook and academic performance. Computers in
Human Behavior, 26(6):1237-1245.
Lehman, B., D'Mello, S., and Graesser, A. (2012). Confusion and complex learning during interactions
with computer learning environments. The Internet and Higher Education, 15(3):184-194.
Lehman, B., D'Mello, S., and Person, N. (2010). The intricate dance between cognition and emotion
during expert tutoring. In Aleven, V., Kay, J., and Mostow, J., editors, Intelligent Tutoring Systems,
volume 6095 of Lecture Notes in Computer Science, pages 1-10. Springer-Verlag, Berlin, Heidelberg.
Piaget, J. (1952). The Origins of Intelligence. International University Press, New York.
Rodrigo, M. M., Baker, R. S. J. D., Lagud, M. C. V., Lim, S. A. L., Macapanpan, A. F., Pascua, S. A. M.
S., Santillano, J. Q., Sevilla, L. R. S., Sugay, J. O., Tep, S., and Viehland, N. J. B. (2007). Affect and
usage choices in simulation Problem-Solving environments. In Proceedings of the 2007 conference on
Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work, pages
145-152, Amsterdam, The Netherlands. IOS Press.
Zimmerman, B. J. (2002). Becoming a Self-Regulated learner: An overview. Theory Into Practice,
41(2):64-70.

A New Course Design for Advanced


Communication: Debate and Beyond
A Technology Enhanced Course for Communication Incorporating Empathy
Tosh YAMAMOTOa*, Masanori TAGAMIb , Chiaki IWASAKIc &
Minoru NAKAZAWAd
a~c
The Center for Teaching and Learning, Kansai University, Japan
d
Dept. of Information Engineering, Kanazawa Inst. of Tech., Japan
*ctltosh@kansai-u.ac.jp
Abstract: Based on the fundamental concept of the communication being mainly composed
of logic in information, the university in Japan has just begun offering such courses as
logical thinking, critical thinking, and debate as subjects for the general education to
freshmen and sophomores before they begin their special areas of study. However, after the
natural disasters since the last year, the society has realized that the basis of communication
may be supported not mainly by logic or critical thinking but more or less by some empathy
driven factor. Because the university has the mission to raise promising potential members
for the future society, a new concept for a course has come out in which the successful
communication consists of both logical and critical thinking on the one hand, and emotion or
empathy on the other. This paper attempts to propose a new course to such issue enhanced
by the information technology available to us.
Keywords: communication, empathy, negotiation, ICT, SNS

Introduction
Since the Benesse Corporation reported in its journal VIEW in 2011, it has been revealed
that most universities do not satisfy the needs from the society in their curricula or diploma
policies, [1]. For example, after three years after graduation, most university graduates feel
the need for such fundamental skills as problem solving skills (data collection, analysis,
problem solving), continuous learning (intellectual curiosity and active learning),
independence (exercising independence and displaying leadership in projects), teamwork
(collaborative attitudes & perform ones duties and responsibility in projects),
self-management (setting goals, planning well, and working accordingly), problem setting
and solving (analyzing the situation to identify the problem to be solved), and logical
thinking. It follows that the universities in Japan do not provide the education necessary for
the graduated students. In this way, such fundamental skills have been beginning to be
incorporated in the freshman and sophomore curricula. The fundamental academic skills are
generally considered as the skills necessary to do well in an educational setting as well as in
a social situation after graduation. Thus, most universities thought that such incorporation of
the fundamental skills into the curriculum was enough to meet the required needs by the
graduates.
However, after the natural disasters happened on March 11 the last year, the society has
realized, in the process of recovering while demonstrating resilience, that the basis of
human communication to build the strong connections between people may be supported
not mainly by logic or critical thinking but more or less by some empathy driven factor. This
means that although most universities have been offering courses such as logical thinking,
critical thinking, and debate, students graduated without having good communication skills

to do well in the society. As a matter of fact, in his book titled Become an Effective
Leader, Dale Carnegie emphasizes that most part of communication to reach the heart of
others is composed of affection to them instead of the logic or rhetoric of the language that
people use, [2].
Is there any way to combine the traditional communication course of teaching logical
and critical thinking with empathy to build a long-term good relationship with others? It is
proposed that a new course may be designed with the help of a new approach to
communication as well as the cutting-edge technology.

1.

A New Communication Course Incorporating Empathy

1.1 Background and the Problem Defined


Kansai University has in the curriculum the basic study skills courses including
problem-identifying, problem-solving, note-taking, report writing, presentation, computer
literary, and debate. Debate is placed as one of the most advanced courses in the basic study
skills courses, which incorporates the logical thinking and meta-cognition skills. However,
the debate course does not offer anything to build trust and good relation between students.
Further, it has been found that emotion or empathy was not in any part of the
communication in the debate. Rather, it is a virtue to hide emotion while debating.
1.2 Kansai Universitys Challenge to Design a New Course
In order to remedy the lack of empathy or emotion in the course, it is proposed that a new
course for communication incorporating empathy be introduced in the curriculum to
enhance the series of courses for the basic academic skills. In the course of such
incorporation, trust building through communication is fostered through negotiation
seminar and exercises in groups. In such a course, the simulation experience is crucially
employed. The rationale behind this is that the simulation of the real experience will have
extremely high learning outcome close to the real life experience, as Edger Dale claims, [3].
See Figure I.
Figure I. Edger Dales Cone of Learning

Figure II. The Framework for Negotiation

From: www.cals.ncsu.edu/agexed/sae/ppt1/sld012.htm From: J. Tamura, et. al. (2010). Visual Explanation:


An Introduction to Negotiation, Nihon Keizai
Shinbun.

1.3 Incorporating Trust Building into the Basic Communication Courses with Empathy
The main goal of the methodology for communication proposed by Roger Fisher at Harvard
University is generally called negotiation to lead to say yes. The negotiation requires prior
preparation to set a mission, which is the optimal goal to be aimed at, n this case, for
maintaining a long-term good relationship, [5]. In order to accomplish such mission, some
preparation is needed. Since it is not usually possible to achieve the 100% of the mission
through negotiation, a zone of possible agreement is set to limit the range between the
maximum and the minimum goals prior to the actual negotiation. This zone of goals is
called zopa, [6]. Furthermore, if all the options for possible agreements prepared for the
zopa fail, the best alternative to the zopa, called batna, is further planned, as shown in Figure
II, [8], [9].
The negotiation skill described above demonstrates the skill for gathering information
for a certain topic and then making a decision based on the information. By understanding
the common framework of negotiation consisting of the mission, zopa, and batna, and by
conforming to them, it is possible to demonstrate the logical thinking skill, the critical
thinking skill, as well as the skill to conform to the rules and conditions, [8]. Further, by
exercising the framework of negotiation, students can develop their communication skills to
elicit information necessary for the negotiation as well as the arrangement skill for
conducting the negotiation, [9]. This can be only possible with the mission aiming at a
long-term good relationship.
1.4 New Course Design
Having the basic concepts of negotiation, the negotiation methodology has been developed
incorporating a case study and a role-play simulation, [10]. In order to generate the
maximum learning outcome for the designed negotiation course, it requires at least three
hours in a session. However, the traditional time frame or contact hour for the course is
90-minute long, meeting only once a week. Thus, it requires two weeks of class contact
hours in order to have a session of three hours. The problem is how to maintain students
motivation and attention to be stretched to the class on the second week. In other words, the
retention of the learning experience from the within group discussion will be lost on the
second week if nothing is done by the students. Thus, by the time when the class resumes on
the second week, the students would need to review what they learned on the previous week.
In other words, much time would end up being wasted to refresh what the students learned
on the first week.
Table I. One Session Ranging Over Two Weeks

Figure III. Discussion Activities on SNS

One may think that the students could meet face-to-face in groups after the class or
later in the week to continue the discussion for the second week. However, taking into an
account that the students from the thirteen colleges are taking the course as an elective for
the general study in addition to their required courses of 15 to 24 credit hours per semester,
it is impossible for the students to get together face-to-face except for the class contact
hours.

2. Proposed Solution to the Problem


After the within group discussion on the first week, the students are asked to post the
minimum of five times to post their own strategic plans to the SNS designed for the class
and to give comments to other members posts. The two within groups have their own
communities and thus they do not have cross-membership to both within groups. In this way,
the SNS can be accessible for 24/7 for the students, [11], [12]. Refer to Figure III.

3. Conclusion
A new course incorporating empathy or emotion to the traditional communication course
contents was developed and elaborated in this paper. In the course of development, Dr.
Roger Fishers methodology for negotiation was employed to satisfy the needs. However, in
order to achieve a high learning outcome, at least three hours are required to one session for
negotiation, which conflict with the traditional class schedule (a 90 minute-session per week
times 15 weeks). Thus, one negotiation session (i.e., three hours) must be extending for two
weeks. This means that the students retention to the course contents must be maintained
till the following week so that the first half of the three-hour session be smoothly fed into the
second half. It was proposed that the SNS be employed as out-of-class activities between the
segments of the negotiation session. With the help of information technology, the
psychological interruption has been overcome.

References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]

The Benesse Corporation. (2011). The View. Vol 21. Tokyo, 2011.
Carnegie, Dale. (2012). Become an Effective Leader, New York: Dale Carnegie & Associates, Inc. In
Become an Effective Leader, Tokyo: Sogensha.
Dale, Edgar. (1969). Audio-Visual Methods in Teaching, 3rd ed., (p. 108). New York: Holt, Rinehart &
Winston,
Bruner, Jerome S. (1966). Toward a Theory of Instruction. (p.49). Cambridge, MA: Harvard University
Press.
R. Fisher, L. U. William, & I. B. Patton. (1991). Getting to Yes: Negotiating Agreement Without Giving
In. New York: Penguin Books.
R. Fisher & S. Brown. (1989). Getting Together: Building Relationships As We Negotiate. New York:
Penguin Books. 1989.
J. Tamura. (2004). Negotiation Strategies, An Introduction to Negotiation Based on Critical Thinking.
Tokyo: Diamond Press.
M. Isshiki, J. Tamura, & K. Sumida. (2010). An Introduction to Negotiation. Tokyo: Nihon Keizan
Shinbun Shuppansha.
Diamond, Robert M. (1989). Designing and Improving Courses and Curricula in Higher Education.
San Francisc: Jossey-Bass.
B. E. Drushel & K. German. (2011). The Ethics of Emerging Media: Information, Social Norms, and
New Media Technology. New York: The Continuum International Publishing Group.
C. Fuchs. (2011). Foundations of Critical Media and Information Studies. New York: Routledge.

Virtual Learning Environments in Primary


SchoolsUsing Learning Theory to Develop an
Interactive Virtual Medium
Eiman Tamah Al-Shammari
Kuwait University, Kuwait
Eiman.tamah@gmail.com

Abstract: Virtual Learning Environments (VLEs) bring traditional educational strategies


into the 21st century by using technology as a deployment method. Online environments can
be utilized alone or in conjunction with face-to-face instruction to develop a robust
educational paradigm that is tailored to the students' needs. This paper intends to describe
how learning theories recently applied in traditional classrooms can be transferred to an
interactive virtual medium to achieve this paradigm. In addition, this paper also
demonstrates how the practical model of Piaget's Constructivist Theory of Learning and
Vygotsky's Zone of Proximal Development, and their related strategies, could be applied to
achieve a set of specific and effective e-learning objectives. This paper will also discuss
identify how these strategies can be used to develop a VLE appropriate for primary schools.
Keywords: E-learning, Piaget's constructivist theory, Virtual Learning Environments, Zone
of Proximal Development.

Introduction
With the available e-learning tools in hand and their perceived deficiencies on record, we
will attempt to design a new primary school VLE e-learning model that takes into account
the principles of constructivist theory in conjunction with Vygotsky's theory [7] of the Zone
of Proximal Development. We will also try to assess and evaluate how we can create
effective VLEs to integrate seamless pedagogical values into the e-learning system.
Effectively speaking, the launching of VLEs in both primary and secondary schools could
be extremely effective and have tremendous positive results [2], in its monumental report in
the effectiveness of VLEs, suggests that:
"To set up an efficient VLE to implement school curriculum, the school management
may need to think of its effectiveness that in turn is influenced by three important factors:
knowledge management, pupils' approach to learning, and academic performance (p32)."
In other words, any VLEs used in a primary school should satisfy the above-mentioned
pedagogical parameters in a way that eventually leads to success of the e-learning initiative.
An effective VLE design should provide seamless knowledge and skill management. Before
designing a prototype model, we need to look at various factors that influence the e-learning
outcome. The VLE prototype should meet the following criteria:
1. It should consider various issues that relate to learners' mental development stages,
cognitive development and user preferences.
2. It should be flexible, easy to use and have an effective interface.
3. It should address the concerns of instructors cited previously in this paper. Using
effective VLEs to integrate seamless pedagogical values

1.

Pedagogical Framework to the Proposed Model

Swiss developmental psychologist Jean Piaget dedicated his life to studying child
development and is credited with the development of the constructivist theory of knowing.
His theory acknowledges that people bring previous knowledge to each knew encounter,
and as they take in new information it is assimilated with that previous knowledge in a
continuous process to gain a higher level of understanding.
Constructivism (also commonly known as social constructivism) recognizes that each
student brings a unique and diverse perspective to the learning environment. This is
especially beneficial in a VLE since there are no physical boundaries as to where the
learners may be; instead of a traditional classroom where the students may all share a culture
or socioeconomic background, virtual learners might be scattered across the globe.
Piaget's constructivist theory links up well with another prominent learning theory, the
Zone of Proximal Development (ZPD), which was developed by Lev Vygotsky in the early
20th century. ZPD promotes the idea that primary-age children learn first and best through
social interactions, with the teacher serving as observer. The ZPD theory is based on the
concept that learners are motivated to reach slightly outside their comfort zone to further
develop their knowledge and skills. The concepts of constructivism and ZPD form the
pedagogical framework to our proposed VLE model for primary school students.

2.

Description and salient features of the new e-learning model

Incorporating both constructivism and ZPD into our e-learning prototype for primary school
students, we have established the following methodology for learning interactions:
1. Socialization
2. Individual Pre-assessment
3. Active Learning
4. Individual Post-assessment
This methodology is intended to be circular, with the experience of knowledge transfer a
cyclical process. In other words, the learning experience does not end. It is built upon by
repeating the process to continually increase knowledge. Let's take a look at each step of the
process and how it can be applied to the primary school VLE.
Both Piaget and Vygotsky understood the critical role that socialization plays in the
learning process, and that is why this prototype includes socialization as its initial step.
Using a simple chat-like interface that will likely be familiar to most primary school
students, each learner should be encouraged to share information: a personal detail, such as
their favorite color; an interesting experience, such as a recent vacation; or any other
information that the student wants to share with his or her peers. Not only does this provide
the basis for peer-to-peer relationships, it also provides the foundation for learning in a
constructivist methodology. Students take in information from their peers and assimilate it
with their own knowledge and experiences. For example, when Sarah shares that her family
took a vacation to Disney World in Florida and also visited the ocean, the other learners may
assimilate that Florida is near an ocean.
At this point, the VLE shifts into a pre-assessment, designed to evaluate the existing
level of development on the topic at hand. For example, if the lesson is covering colors, the
VLE may provide a technology-based quiz that assesses the student's knowledge of various
colors. Keeping in mind that some primary students may not yet be able to read, the VLE
should incorporate audio as well as visual effects to meet each learner's needs. The
pre-assessment provides a baseline for the instructor before moving into the active learning

phase of the methodology. The baseline identifies each student's existing zone of
knowledge, which is necessary to understand the ZPD--that area that is just slightly outside
the existing zone. If a student is pushed to make too large a leap between the existing zone of
knowledge and the next zone, they are likely to back away from the learning process
altogether. The pre-assessment encourages the effective building of skills.
With the information gained in the pre-assessment, teachers can adapt an active learning
environment to each student's needs. Continuing with the color example, if a teacher
determines that most of the students can identify primary colors but have difficulty
identifying secondary and tertiary colors, the teacher can tailor the learning experience to
meet those needs. Likewise, if a pre-assessment identifies a student who needs remedial or
more challenging work, the instructor can accommodate those needs. The teacher can also
group learners based on level of ability to encourage peer-to-peer socialization and learning,
and to make sure the student continues to build on existing knowledge.
This building process has come to be known in recent years as "scaffolding." This term
was developed in the 1950s by cognitive psychologist Jerome Bruner, and it originally
related to the development of language skills in young children. However, it has grown to
include all cognitive development based on a model of helpful interactions between an adult
and a child that build the child's skills. The adult provides boundaries and perhaps even a
template for the learning experience. As it relates to our methodology, scaffolding takes
place in the third stage as the instructor provides an active learning experience.
Instructors may have to use some creativity when designing active learning
experiences delivered in a VLE. In his constructivist learning theory, Piaget noted three
types of learning in children: functional play, symbolic play, and games with rules.
Traditional classrooms allow for physical functional play, such as running or jumping up
and down, and symbolic play, such as making paper dolls. In the VLE, however, instructors
may have to be creative when developing play-based learning experiences that lack the
physical component. For example, instructors can encourage role-playing as a type of
symbolic play that can be accomplished in a VLE.
The key to the active learning step is to continue to provide the socialization that is so
critical in both Piaget's and Vygotsky's methodologies. It is important to note that "active"
learning refers to active cognition, rather than active physical behavior. A student can be
physically calm sitting in a chair in front of a computer and still be engaged in active
learning. The critical piece of active learning is that the student is engaged in the learning
process rather than passively receiving facts and information.
Following the active learning step in our methodology, the student returns to a brief
individual post-assessment to evaluate the effectiveness of the learning experience. This
stage should parallel, although not exactly mirror, the pre-assessment, to fairly evaluate
what was learned. Integrating the individual stages of this methodology, the pre- and
post-assessment, with the more socially interactive stages provides a balance to the learning
experience and gives the student time to reflect on what has happened and what will happen
next. With this VLE model, the primary advantages include the following:
1. It is a user-centric model. According to Anghern [1], the learner should be at the
center of the e-learning model and this approach becomes very critical for managing
individual knowledge capital and competence.
2. It is extremely interactive. The model proposed here is very dynamic and interactive.
The instructor is continuously available to the student and the peers are in ready
contact as well. Primary school learners need constant monitoring and this model
helps the instructor mentor the young learners on a consistent basis.
3. The scaffolding effect allows the learners to use a rich learning environment to try
out exploration, manipulation and construction of new ideas and learning
fundamentals

Table 1: Constructivism Learning Model in a VLE


Stage
Description
Example
Socialization
Teacher gives a topic;
"Today we're going to talk about USA,
students share by typing
Who wants to share something
their responses in the
about a state they've been to, other
chat area of the VLE.
than the one we live in?"
Individual
Online assessment of ability Each student individually sees a map
Pre-assessment
of the U.S. & a state name, the
student should click on the state
outline that matches the state name.
Active Learning During the lesson, the As the teacher talks about a state, it
teacher, using the VLE,
lights up on the screen and icons
gives the lesson. Each
appear to illustrate the discussion.
student must participate
For example, as the teacher
by
responding
to
discusses Iowa's farming & biofuel
questions & Interacting
industries, an icon of corn appears.
in the VLE.
Individual
Online
assessment
of Each student sees the same map as
Post-assessment
learning
seen in the pre-assessment, but now
must match icons to the states they
represent.

3.

Conclusion

E-learning is a far-reaching pedagogical transformation for a traditionally inclined


curriculum. Digital inequality and inequitable access to online curriculum could pose a
great challenge when designing a new VLE model. An easy-to-use and flexible e-learning
tool could act as a strong foundation for effective distribution of school curriculum.
Many authors and academicians believe that VLEs in a primary school setting are out
of context and not useful. Many schools, both in developed and developing countries, are
finding deployment of VLEs very challenging. One of the biggest concerns that most of
these schools have is whether the e-learning tools used are transforming the pedagogical
values of teaching in an adverse manner.
Of late, there is an intense debate on the efficacy of commercial e-learning tools in
primary classroom teaching. One of the concerns expressed by [4] is that the
learning-management system vendors are trying to maneuver their tools as the focal point
for e-learning in organizations, eventually to remove control from the tools' end-users; here,
the school management, instructors and learners. Merrienboer et al [6] believes that:
"It is not the media, but the instructional methods and the delivery systems that can enhance
the overall quality of education and learning experience."
Most of the VLE systems are excellent at delivering course materials. However, this
cannot be the sole criteria for the success of e-learning. In his article on Learning
Management Systems - The wrong place to start learning, Siemens [4] opines that:
"A good learning tool will help enhance informal learning, support superior
performance and manage skills and knowledge development. In fact, as an e-learning tool
becomes more sophisticated and feature-rich, it tends to lose its practical utility and basic
functionality."
Significant progress has been made in designing and engineering aspects of many
e-learning platforms, tools and models. Although feature-rich and good in delivery, they
may lack from inherent deficiencies and user-side weaknesses. In addition, such tools have

been working with a common principle as noted by Siemens [4] tool selection first and
instructor requirements second. Simplified, it means that the tool itself tends to override the
user and user's preferences or options.
Gagne's theory of learning espouses the ideas of signal learning, stimulus-response
learning and verbal association [9]. Although, Novak and Taylor's Stimulus-Response
Theory (1977) forms the basis of Gagne's theory, the latter includes a number of cognitive
factors in the learning model that eventually leads to an all-around development of young
learners' mental and brain development.

References
[1] A. Angehrn, T. Nabeth and C. Roda. (2001, Nov 2001) Towards personalised, socially aware and active
e-learning systems: Illustrated with the agent-based system k-inca [Online]. Retrieved April, 7 2011.
http://www.calt.insead.edu/eis/documents/K-InCA-whitepaper.pdf.
[2] British Educational Communications and Technology Agency (Becta). A review of the research literature
on the use of managed learning environments and virtual learning environments in education, and a
consideration of the implications for schools in the united kingdom, Coventry, Becta, 2004.
[3] C.H. Orrill. (2000) Learning objects to support inquiry-based online e-learning. In D. A. Wiley (Ed.), The
Instructional Use of Learning Objects: Online Version. Retrieved April 6,2011, from the World Wide
Web: http://reusability.org/read/chapters/orrill.doc.
[4] G. Siemens. (2004, November 22) Learning Management Systems: The wrong place to start learning,
[Online] Available. http://www.elearnspace.org/Articles/lms.htm
[5] J. Novak and R. Tyler. ( 1977) A Theory of Education. Cambridge University Press, New York.
[6] J. V. Merrienboer, T. Bastiaens and A. Hoogveld. (2004) Instructional design for integrated e-learning.
In, Intergrated e-learning; implications for pedagogy,technology and organisation (Ed): Jochems,W;
Merrienboer, JV & Koper, R.London: Routledge Falmer
[7] L.S. Vygotsky. (1978) Mind and society: The development of higher mental processes. Cambridge, MA:
Harvard University Press.
[8] M.A.C. Gonzalez, D. Suthers and J.G. Escamilla De Los Santos. (2003) Coaching web-based
collaborative e-learning based on problem solution differences and participation. International Journal of
Artificial Intelligence in Education 13(1).
[9] R.M. Gagne. (1985) The Conditions of Learning and Theory of Instruction. New York: CBS College
Publishing.
[10] S.K.H. Chu, S.K. Tse and K. Chow. (2011) Using collaborative teaching and inquiry project-based
learning to help primary school students develop information literacy and information skills, Library &
Information Science Research, Volume 33, Issue 2, April 2011, Pages 132-143

Development of STEAM program Math


centered for Middle School Students
EunJung Kim*, SunHoi Kim & DongSoo Nam &TaeWuk Lee
Dept. Computer Education, Korea National University of Education, Korea
*mathrose@hanmail.net

Abstract: Now as a society of knowledge and information , it is required the develoment of


autonomous and creative human resources that can create new intellectual value using the
knowledge and information given by rather than simply memorizing the knowledge. In
terms of development of creativity, mathematics is a very important subject. However,
many students do not feel the need and the value of mathematics and they just study without
any interests. So looking at the buzzword of these days, STEAM . STEAM is the fusion of
five fields of studies which are science, technology, engineering, art, mathematics. In the
early advanced study it can be seen that using STEAM , education has a positive impact on
attitudes and interests . And computer programming enhances students' algorithmic thinking
in mathematics. Therefore, this research wants to develop STEAM program centered with
mathematics using scratch to improve the mathematical interests of students.
Keywords: STEAM, Scratch , Mathematical interest, Learning program.

Introduction
We are living in an information society away from the industrial economy such that
knowledge and information is the driving force of economic development. In such a society,
development of autonomous and creative human resources is required because personal and
national competitiveness is determined by the value of new intellectual value-added
production capacity based on the high level of information processing capabilities. As a
seed of invention and discovery, creativity is the most high-level thinking skills and is the
most needed ability in a information and knowledge, cutting-edge science and technology
era. Thus, creativity is the main goal in all stages of school education[1].
Mathematics has begun with the history of mankind and can be found as a common
language of humanity in the worldwide culture and is an essential tool in everyday life .
In particular, mathematics plays a very important role to understand and use a lot of
information that is invaluable in a rapidly changing and today's information society.
It is one of the aspects of creativity in the way it makes us to think outside the fixed frame
to find the solution[2].
Most students recognize mathematics as an important subject but very few students
properly recognize the value of mathematics. Studying for tests, but it's rare for the fun of
math, STEAM is an integrated learning approached in the aspects of science, technology,
engineering, mathematics, art to one subject. This increases the efficiency of learning and
motivation as well as has a positive impact on the development of learners' interest.
In addition, the linkages between various disciplines through STEAM education is
consistent with the buzzword "convergence" emerged in the modern world. STEAM is a
great way of education is the opinion of experts that are common. In recent years, United
"This work was supported by the National Research Foundation of Korea(NRF) grant
funded by the Korea government(MEST) (NRF-2012-S1A5A2A01016950)

States, Britain, Australia, Canada, and in developed countries have an attracting attention
that the core of educational reform is STEAM . In earlier studies conducted, the results that
STEAM has a positive effect on improvement of the level of learning attitude in math and
science are coming out[3].
According to the principle of learning in NCTM, it emphasizes that students should
consist new information actively based on previous experiences and knowledges. It says
learnig through understanding is effective. It means when students are in the process where
they feel and contemplate by themselves, there they will experience the complete
understanding and will have the possibility of creativeness developing [4].
Learning math through real life is very similar in meaning with the concept of
STEAM. By utilizing a computer program to solve the problem, students effort to think of
the ideas and actually look the mathematics through the implement the program so it would
increase the interests in mathematics.
Scratch is a programming language usable in all subjects. The command block works
in a way of drag and drop. Compared to the existing engineering tools, it is more visual and
dynamic, intuitive and easy to use for school students. Thus, I developed STEAM program
centered with mathematics using the Scratch programming language in order to improve
their math interests .

1.

STEAM

STEM education was introduced in order to improve competitiveness in Science and


Technology in the United States in 2003. STEM teaches science, technology, engineering,
mathematics in an integrated way. In 2007 George yakman has announced STEAM in
addition art to the STEM. [Figure 1] is the STEAM Pyramid. Yakman said by the STEAM
education we can increase their relevance to real life and interests[4]. Many education
scholars into a unified art in STEM education, said be STEAM.
Out from the dichotomous thinking such that science of conventional is logical and art
is not logical, STEAM is to foster creative human resources by integrating STEM and art.
Science provides a methodological tool in the art and art provides creative model in the
development of science. Science uses imagination and emotion, thinking that the power of
visualization principles of art and art uses scientific discoveries and principles of science[9]

2.

Math based on STEAM

Principles and Standards for School Mathematics argues that mathematics should deal with
topics related to the world we live in and it should be related with other fields of subjects.
This is proposing mathematics should be integrated with other fields and be explored with
real life[10]. Looking for examples of math related with real life, in ji-yeun Lees paper,
she finds the reason why students dont like math and says current math class teaches
contents far away from real life. So she proposed contents realted with real life.
When instruct Mathematics, to generate mathematical interest and mathematical
thinking and problem-solving kidneys, it is necessary to be away from a common
textbook-centered learning methods but to introduce real-world problems associated with
real life experience to ensure mathematical affinity[5]. When we see those products or
technologies that we often use in everyday life we can see it is integrated with several
studies. Thus, when teaching mathematics , it would be more interesting lessons if
mathematics is integrated with other subjects and related to real life and based on STEAM .

3.

Math and Computer Programming

In the bottom of Computer programming, it is situated mathematics there. In addition, in a


course of mathematics, there is numerical analysis which uses computer to solve math
problems and prove the algorithm. Learners improve logical thinking and can look back
mathematical concepts such as functions through computer programming . Above all, using
a computer. we can solve the problems which is difficult to solve by hand. The
programming improves algorithmic thinking, problem-solving, particularly reflective
thinking through the process of error correction[6].
Looking at the impact of computer programming on mathematical thinking, computer
programming can help to develop individual's intellectual capacity by organizing the
information given in a accurate and systematic way and giving the environment to arrange
the information to operate logically. In the course of these programming learners are on the
a position to build the system of knowledge rather than being audience[8]

4.

SCRATCH

Scratch is an educational programming language made by the MIT Media Lab in 2007 for
programming beginners and the teens. Programmers create programming so easily as
building several blocks in eight block area. Scratch is so intuitive language that provides
almost everything to make computer games, multimedia Presentations, interactive stories,
illustrations, and animations [6]. If we list the features of Scratch,
1. It can be used in a variety of ages.
2. It was created based on Squeak.
3. It is programmed using the visual object.
4. It is the programming building blocks like Lego or puzzle pieces.
5. It is possible to parallel execution and stepping.
6. It is possible to utilize a variety of media.
7. Sharing and collaboration is possible.
8. The program is free and open source.
To study math based on STEAM, we need to realize it in some ways. While reviewing
scratch, In the process making project using scratch, students learn how to choose and
create and manage many objects while choosing many images, animations etc. And scratch

makes students to respond to an unexpected situation with a creative solution. So it


increases creative thinking skills [12]. So I think Scratch is a good way to realize it.

5.

Methodology of the STEAM program centered with mathematics

5.1 Directivity and select content of Study program

Choose the theme which learners can see mathematical principles and meet easily in
real life
Choose the theme which includes STEAM elements.
Consider middle school students interest, study level, learning ability.

5.2 Design the model for teaching


Table 1. The model for STEAM and content [7]
STEAM
content
Introduction
- Introducton Project based Learning program Motivation
Deciding Subject and - Decide topic
planning
- Decide subtopic and make teams
-Secure study sources.
Becoming one with -Investigate and research, sharing datas, Meeting and collaboration
project
sharing and expanding thinkings
- Express subject with art elements.
Presentation of the - Represent the results
results
-Compare own results with others through representation
finish and evaluation - Evaluate various methods
- Finish and reconstruct
5.3 Instructional design of STEAM program centered with math.
Table 2. Example of Math Program <Making a Clock>
Contents
Of the shape, Proportional expression
We can make the clock using proportional expression and the
Learning goals
angle.
S: The difference between analog and digital
T: The development of technologies of watch
STEAM
E: Advanced clock
A: Clock Design
M: The concept of angle in geometry, proportional
Time can be displayed using the angle and proportional
expression
Detailed activities
Understand the difference between analog and digital, and should
design the clock with the advantage of this.

Table of content
Unit

Table 3. Example of Math Program < Create geometric patterns >


Table of content
Contents
Unit
Function
Learning goals
Understands the graph of function and utilize

STEAM

Detailed activities

6.

Putting it into the props to create their own patterns.


S: The principle of fabric dyeing
T: The Development of dyeing technique
E: The process of fabric dyeing
A: design
M: function and graph
Understand the function and can draw the graph
Think about the principle of fabric dyeing and program it with
scratch to create their own patterns and we can relate with real
life.

Conclusion and Future Work

In this study, I developed STEAM program centered with mathematics using scratch to
improve mathematical interests for middle school students. This program could increase
learning interest. In the learning process, programming will help learners mathematical
thinking. To inspect the effects of this program, we have to apply this method to real class.
Developed STEAM program will be taught to students in class and well observe the
students learning activities and analyze outcomes of learning.
STEAM-related learning programs have not been developed a lot. Learning programs
based on STEAM should be developed a lot for school teachers field to utilize them. We
need to discipline teachers to teach the program well and train many teachers. From now on,
there should be many attempts of STEAM education. As the concept of STEAM, many
teachers from various fields get together and share their knowledge to make better program.

References
[1] Dong-Hee Lee. (2009). The Effect of Climbing Learning Method on Mathematical Creativity and
Inclination for Mathematical Creativity. Master Thesis. Busan national University.
[2] Byeng-Jo Kim. (2002). The Study on the effectiveness of the creativity training program by DESK
model. Master thesis. Kyung Hee University.
[3] Hyo- nyung Lee. (2011). Investigation of STEM education of USA for the start of STEAM. Journal
science creation .
[4] Woo- Jin Kim. (2012). STEAM Program Development and Application for Improving Creativity of the
Gifted Elementary Student about Math Focused on 4D-Frame Teaching Aid Activity. . Master thesis.
Korea National University of Education
[5] Ho-Soon Kim. (2002). The Effects of Problem Centered Learning based on Real Life Problems in
Mathematical Beleifs and Attitudes. Master thesis. Korea National University of Education.
[6] Yun -A Kim. (2012). A study on the Function of first grade Middle School Using an Educatioal
Programming Language Scratch. Master thesis. Korea National University of Education.
[7] Soon- Beom Kwon. (2012). The Effects of Convergence Education based STEAM on Elementary
School Students' Creative Personality. Master thesis. Korea National University of Education
[8] Byung-duk Ahn. (2004). Elementary computer science curriculum which based on Visual Basic
programming in order to improve mathematical problem solving skills. Master thesis. Yonsei
University.
[9] hyung-ju Park.(2012). A Study on analysis of Mathematical textbook based on STEAM Education.
Master thesis. Ewha Womans University.
[10] Shin-deuk Lee. (2006). Development and application of ICT teaching and learning Materials driven real
life and the changes of students' recognition after mathematics instruction. . Master thesis. Korea
university.
[11] Ji-Yeon,Lee. (2011). Mathematics in (Real) Life. Master thesis. Chung nam National University.
[12] Natalie Rusk, Mitchel Resnick,& John Maloney. (2010)
Learning with Scratch.
http://info.scratch.mit.edu/sites/infoscratch.media.mit.edu/docs/scratch-21centuryskills.pdf

Digital Classrooms for Developing Subject


Knowledge, Information Literacy and
Twenty-first Century Skills
Siu Cheung KONG
The Hong Kong Institute of Education, Hong Kong
sckong@ied.edu.hk

Abstract: A study was conducted for creating digital classrooms to support students to learn
subject knowledge and develop information literacy competency as well as critical thinking
skills. A total of 144 students in four Secondary 1 classes in a secondary school in Hong
Kong were invited to participate in the study. In the 12 trial lessons, every three students
shared a Tablet PC for learning the topic Air Pollution Problems in Hong Kong in the
Integrated Humanities subject. After the trial teaching, the students and teachers were
interviewed about their perception of the support from the designed digital classrooms. The
study found that the students and teachers positively perceived the support from the
designed digital classrooms on developing subject knowledge, information literacy
competency and critical thinking skills. Three recommendations are made for enhancing the
pedagogical designs in digital classrooms that emphasize the use of mobile technology.
Keywords: Critical thinking skills, information literacy, Integrated Humanities, digital
classroom, secondary school

1.

Introduction

School education in the twenty-first century is expected to equip students with both subject
knowledge and the twenty-first century skills in order to meet the requirements of a
vigorously changing society [1]. Information literacy and critical thinking skills are two
important parts of the twenty-first century skills [1, 2]. Information literacy (IL) refers to the
mastery of necessary knowledge of gathering, synthesizing, analyzing, interpreting and
evaluating information; and the proper attitudes for information processing with an
understanding of the rationale behind using information [3]. Critical thinking skills are the
capabilities to think reflectively and judge skillfully, so as to decide what information is
reliable and what actions should be taken during reasoning and problem-solving [1, 4].
Future classrooms in the twenty-first century are digital classrooms wherein students
have many chances to use digital technology to access digital resources that contain
information in digital form for learning subject content [5]. Inside the digital classrooms
supported by mobile technology, students use mobile devices that are wirelessly
interconnected for completing learning tasks, especially accessing additional learning
information from sources other than textbooks [6, 7]. For the successful learning in digital
classrooms, students need to apply IL competency and critical thinking skills in daily
subject learning [1, 2], in order to properly process information from different sources and
then critically assimilate information for subject learning. In this regard, special
pedagogical designs with appropriate learning tasks should be made for students to develop
subject knowledge, IL competency and critical thinking skills in digital classrooms [1, 7].

For the effective development of subject knowledge in class, the class activities should
be designed to progressively promote students active learning, constructive learning and
interactive learning [8]. In this regard, teachers should arrange class activities that engage
students in the processes of resource access, knowledge construction and peer interaction.
Examples of those class activities include information search, mind map drawing, and group
discussion.
For the effective development of IL competency in class, there are three ways
potential to prepare students to skillfully process learning information [2]. In way (i),
teachers integrate IL elements into class activities, such as asking students to search
information for completing learning tasks. In way (ii), teachers lead students to discuss IL
elements involved in learning tasks after class activities. In way (iii), teachers guide students
to discover IL elements tacit in learning tasks.
For the effective development of critical thinking skills in class, there are four ways
potential to prepare students to critically solve learning problems [1]. In way (i), teachers
give students sufficient time to think about the problem-solving questions. In way (ii),
teachers give students sufficient time to discuss the problem-solving questions with group
members. In way (iii), teachers guide different groups of students to make sharing with the
whole class. In way (iv), teachers explain the answers for the problem-solving questions and
then guide students to make reflection. The study reported in this paper created digital
classrooms to support students to learn subject knowledge and develop IL competency as
well as critical thinking skills. The class activities in the study were designed in line with the
suggestions from [1], [2] and [8] as above.

2.

The Study

This paper reports an initial experience from a study in Hong Kong that created digital
classrooms to support students to learn Integrated Humanities (IH) knowledge and develop
IL competency as well as critical thinking skills. By purposeful sampling, a total of 144
students from four Secondary 1 classes in a secondary school in Hong Kong were invited to
participate in the study. Table 1 shows the profile of students participated in the study.
Table 1: Profile of students participated in the study
Number of students
Ratio of boys to girls
Mean age in years

Class A
36
8:28
12.6

Class B
36
10:26
12.3

Class C
36
20:16
13.1

Class D
36
18:18
13.2

A trial teaching on the topic Air Pollution Problems in Hong Kong in the IH subject
was conducted for each participating class. The trial teaching amounts 960 minutes, with 12
lessons of each lasted for 80 minutes. The two IH teachers of the four participating classes
were responsible for the trial teaching. The years of teaching experience for the teacher of
Class A and Class B was seven; while for the teacher of Class C and Class D was 13.
The students in each participating class were divided into groups of three for class
activities. In a typical trial lesson, each group of students mainly used the Tablet PC
provided for completing worksheets specially designed for the trial teaching. These
worksheets aimed to lead students to properly process information from different sources
and critically assimilate information on target topic in class. The students were first asked to
answer worksheet questions by referring to the paper-based textbook and the selected
e-learning resources on the school-based e-learning platform. The students then discussed
worksheet questions with their group members. In the discussion activities, the students

needed to apply critical thinking skills such as identifying keywords in the worksheet
questions for problem-solving, and to apply IL competency such as searching information
from reliable websites. After the discussion activities, the students were guided by their
teachers to present and discuss their works with the whole class. In the trial teaching, the
teachers integrated the three ways suggested by [2] and the four ways suggested by [1] for
developing students IL competency and critical thinking skills, respectively. The study
focused on two research questions:
(1) How do the students perceive the support from the designed digital classrooms on
developing IH knowledge, IL competency and critical thinking skills?
(2) How do the teachers perceive the support from the designed digital classrooms on
developing IH knowledge, IL competency and critical thinking skills?

3.

Methods

Semi-structured interviews were conducted with the students and teachers participating in
the study for investigating their perception of the support from the designed digital
classrooms on developing IH knowledge, IL competency and critical thinking skills. Two
groups of students were randomly selected from the four participating classes for two group
interviews. The two teachers participating in the study were invited for two individual
interviews. The semi-structured interviews focused on three aspects, namely the
school-based approach of developing IH knowledge in class; the three suggested ways of
developing IL competency in class; and the four suggested ways of developing critical
thinking skills in class. The invited students and teachers were asked about their
experiences, comments and expectations of learning and teaching in the designed digital
classrooms.

4.

Results and Discussion

In general, the students and teachers perceived that the designed digital classrooms could
support the development of IH knowledge, IL competency and critical thinking skills.
Table 2: Major interview feedback on the development of IH knowledge in the trial lessons
Experiences

Comments

Major interview feedback


The students and teachers indicated that the school-based e-learning platform was used in
class to support the convenient access to additional resources and easy drawing of mind maps.
The students pointed out that the trial teaching emphasized much on the use of mobile
devices and e-learning resources, but less on the use of school-based textbook.
The teachers reflected that they were unable to give their students sufficient time and
guidance to interpret the key contents of the additional resources within class time.
The students and teachers agreed that the use of additional resources and the drawing of
mind maps were effective for supporting concept building and knowledge consolidation.
The students indicated that after trial teaching they were able to link the knowledge
learned to other topics of the same subject, such as economic development and ecology
system.
The teachers suggested that the additional resources selected should match students
learning stage, avoid lengthy and complicated documents that students cannot fully
comprehend.

Table 2 shows the major interview feedback from students and teachers on the
development of IH knowledge in the trial lessons. The results showed that the teachers were
able to implement the school-based approach of developing IH knowledge in trial lessons,

with a focus on class activities of accessing e-learning resources and drawing mind maps for
subject learning. The students and teachers valued the many chances to access update
learning information and exchange subject knowledge in the class activities, which
promoted students to better understand the target topic and link knowledge among different
topics in the target subject. The students and teachers expected for the use of more
additional e-learning resources that match the level and need of students in learning within
class and after school.
Table 3: Major interview feedback on the development of IL competency in the trial lessons
Experiences

Comments

Major interview feedback


The students and teachers indicated that all the three suggested ways were adopted in
class, with way (ii) was adopted most frequently; way (i) followed; and then way (iii).
The students were impressed that the teachers often adopted way (ii) to lead the whole
class to discuss effective ways of information search after completing every group task.
The teachers reflected that they did not have sufficient time to adopt way (iii) to guide
students to discover IL elements tacit in the worksheets questions.
The students and teachers thought that way (ii) was most effective for supporting junior
secondary students to develop IL competency, with way (i) the second and way (iii) the least.
The students indicated that after trial teaching they often tended to search and compare
information from various reliable websites for judging the suitability of target information.
The teachers suggested that junior secondary students should be provided with more
guidance on the rationale behind IL.

Table 3 shows the major interview feedback from students and teachers on the
development of IL competency in the trial lessons. The results showed that all the three
suggested ways for developing IL competency were implemented in trial lessons, with
different levels of frequency and effectiveness. The way of teacher-led discussions about IL
elements after learning tasks was most frequently implemented and regarded as the most
effective way to support junior secondary students to develop IL competency in class. The
students and teachers generally perceived that these three suggested ways made students
become more aware of the necessary knowledge and proper attitudes for processing
information. The teachers hoped to provide more guidance for junior secondary students to
explore and reflect on the IL elements tacit in the process of learning tasks in class.
Table 4: Major interview feedback on the development of critical thinking skills in trial lessons
Experiences

Comments

Major interview feedback


The students and teachers indicated that way (i) and way (iii) were adopted in almost
every lesson; while way (ii) and way (iv) were often omitted because of time limit.
The students were impressed that the teachers often adopted way (iii) and provided
special hints for question interpretation, as well as award incentives to peer discussions in
class.
The teachers reflected that their students lacked sufficient time for discussions in
problem-solving tasks, and lacked sufficient guidance in thinking exercises.
The students and teachers thought that way (iii), which emphasized sharing and
discussion among students, was very helpful for the development of critical thinking skills.
The students and teachers indicated that the trial teaching gradually promoted skills in
critical thinking, such as distinguishing keywords relieve and resolve for
problem-solving.
The students and teachers expected for the provision of clear guidelines and sufficient
time for group discussion in class in future.

Table 4 shows the major interview feedback from students and teachers on the
development of critical thinking skills in the trial lessons. The results showed that all the
four suggested ways for developing critical thinking skills were implemented in trial lessons,
with different levels of frequency and effectiveness. The way of teacher-guided sharing and

discussion among students was most frequently implemented and regarded as the most
effective way to support junior secondary students to develop critical thinking skills in class.
The students and teachers generally perceived that these four ways made students become
more aware of the process and skills in critical thinking, such as distinguishing different
keywords relieve and resolve for problem-solving. The students and teachers expected
for more time and guidance for the whole class to discuss and reflect on the process and
products of critical thinking in subject lessons.

5.

Conclusion

The study found that students and teachers in general positively perceived the support of the
designed digital classrooms on facilitating the learning of IH knowledge and the
development of IL competency as well as critical thinking skills. Based on the results of the
study, three recommendations are made for the future development of digital classrooms
using mobile technology in school education. First, the teachers in the interviews made a
reflection that there was insufficient time for students discussions in trial lessons. Teachers
are therefore recommended to increase time for interactive learning in digital classrooms.
Second, the teachers in the interviews made a reflection that they did not make sufficient
guidance in the use of e-learning platform for promoting students to develop IL competency
and critical thinking skills. Teachers are therefore recommended to increase the use of
e-learning platform and enhance their work on guiding students to develop IL competency
and critical thinking skills before and after class time. Third, the teachers were observed to
have difference in the mastery of pedagogies for promoting students to develop IL
competency and critical thinking skills. Schools are therefore recommended to provide
support of teacher professional development related to digital classrooms, so as to prepare
teachers for the teaching work in the twenty-first century.
The second round of the study is conducting to collect quantitative data on the
achievement and perception of students in learning the target topic in the designed digital
classrooms. More results related to the impact of mobile technology supported classrooms
on formal subject learning will be further reported.

References
[1]
[2]
[3]
[4]
[5]
[6]
[7]

[8]

Gut, D. M. (2011). Integrating 21st century skills into the curriculum. In G. Wan & D. M. Gut (Eds.),
Bringing schools into the 21st Century (pp.137-157). Dordrecht; New York: Springer.
Kang, M., Heo, H., Jo, I. H., Shin, J., & Seo, J. (2010). Developing an educational performance indicator
for new millennium learners. Journal of Research on Technology in Education, 43(2), 157-170.
Kong, S. C. (2008). A curriculum framework for implementing information technology in school
education for fostering information literacy. Computers and Education, 51(1), 129-141.
Ennis, R. H. (2002). Goals for a critical thinking curriculum and its assessment. In A. L. Costa (Ed.),
Developing minds (3rd Edition) (pp. 44-46). Alexandria, VA: ASCD.
John, P. D., & Wheeler, S. (2008). The digital classroom: harnessing technology for the future of
learning and teaching. London: Routledge.
Wong, L. H., & Looi, C. K. (2011).What seams do we remove in mobile assisted seamless learning? A
critical review of the literature. Computers and Education, 57(4), 2364-2381.
Chang, S. B., Lin, C. J., Ching, E., Cheng, H. N. H., Chang, B., Chen, F. C., Wu, D., & Chan, T. W.
(2009). EduBingo: Developing a content sample for the one-to-one classroom by the content-first design
approach. Educational Technology and Society, 12(3), 343-353.
Chi, M. T. H. (2009). Active-Constructive-Interactive: A conceptual framework for differentiating
learning activities. Topics in Cognitive Science, 1(1), 73-105.

An Analysis of using Social Bookmarking for


Acquiring Web Resources for e-Learning
Tak-Lam WONG*, Siu Cheung KONG
Department of Mathematics and Information Technology,
The Hong Kong Institute of Education, Hong Kong
*tlwong@ied.edu.hk

Abstract: With respect to the increasing trend that students acquire information from the
Internet for their formal or informal learning, we aim at analyzing the use of social
bookmarking to help students collaboratively access useful and creditable Web resources,
facilitating collaborative learning. We also investigate the use of social bookmarking for
after-school learning. Social bookmarking allows users to search for, tag, and share Web
resources, and hence recommend students Web resources that are potentially useful and
relevant. Experiments were conducted to study how social bookmarking can improve the
quality of the acquired Web resource. In particular, students using social bookmarking
could acquire a more focused and high quality set of Web resources compared with students
not using social bookmarking.
Keywords: social bookmarking, Web resources, collaborative learning

Introduction
With the rapid growth of Internet technology, students are readily to access different kinds
of information in the forms of online articles, Web sites, Wikis, etc., from the World Wide
Web (the Web) for learning. Students increasingly use the Web to obtain informative Web
resources to replace visiting libraries [1] for learning or completing their assignments. In
light of this, this raises the need of cultivating the information literacy of students and
developing their ability to evaluate the credibility and relevance of the Web resources such
as Web pages, documents, journal articles, etc. [2]. On the other hand, finding accurate,
trustful and reliable Web resources becomes essential. Tools are needed to assist students to
acquire the correct information, so as to avoid them from obtaining incorrect information
and learning the wrong knowledge [3,4].
Though a number of researches have been conducted and showed that social media can
help collaborative learning [5], the use of social media to help students collaboratively
obtain useful Web resources has not been well investigated. Social bookmarking, which is a
particular form of social media, can help students collaboratively access useful and share
Web resources. Students can supply key terms to the social bookmarking system to search
for relevant Web resources. Instructors and students can provide additional information to
their acquired Web resources by making use of descriptive tags if the resources are useful.
Moreover, each user of the social bookmarking system can see the tags given by other users
to any Web resources and have a rough understanding of the Web resources. Figure 1
illustrates the relationship between the Web resources, users, and tags in a social
bookmarking system. The three entities are inter-dependent and correlated. For example,
suppose user u1 tags a Web page d3 in the system tags t2 and t4. Users u2 and u3 may read
and give tags to the same Web page d3 because of tags given may raise their interest.

Similarly, the acquired Web resources and the tags given by users u2 or u3 will influence
u1. This leads to the situation that multiple users may access the same set of resources via
the system and users have influence on each other via the acquired resources and the tags
given, leading to collaborative learning. Therefore, social bookmarking is particularly
useful for sharing Web resources. However, from our best knowledge, no effective
pedagogical approaches have been developed to exploit social bookmarking to help learners
acquire useful and quality Web resources. In this paper, we aim at analyzing the use of
social bookmarking to help learners search for quality Web resources and investigate the
possibility to use social bookmarking for e-learning and after-school learning. Durlak &
Weissberg showed that after-school programs could improve students self-perceptions and
bonding to school, their positive social behaviors, and their school grades and level of
academic achievement [6]. In particular, our research questions are as follows:
1. Can social bookmarking improve the quality of acquired web resources for e-learning?
2. Is there any potential for adopting social bookmarking for after-school learning?

u2

u1

u3

t1,t4
t1,t2

t2,t4
t1,t2
t1,t2,t3

d1

d2

t1,t4

d3

t2,t4

d4

t4,t5

d5

t5,t6

d6

Figure 1. An illustration to show the relationship between users, tags, and Web resources
1.

Literature Review

The vast amount of information on the Web triggers the need for students to cultivate their
information literacy [7,8]. One important component of information literacy is to evaluate
the usefulness, trustfulness, and creditability of the information acquired. Brandt has
discussed several aspects to evaluate information on the Internet [9]. A study has been
conducted to investigate the practices of students to verify the credibility of the Web
information [1]. Another approach has been proposed to evaluate the credibility of the
information in Wikipedia, which can be authored by any Internet user and may contain
inaccurate information [2]. However, this method mainly focuses on Wikipedia articles
instead of generic Web resources. On the other hand, several methods have been proposed
to help access useful information. For example, Lee proposes a method to help students
retrieve Web sites based on the searching patterns of the peers in a course [3]. The main
idea of the method is to recommend a student potentially useful Web sites by employing
association rule data mining algorithm to analyze the searching patterns from students in the
same course. One limitation of this method is that it lacks interaction between students and
hence not having sufficient training of collaboration to students. Mangina and Kilbride

propose another approach to making personalized recommendation of documents based on


user modeling and information retrieval techniques [4]. However, little prior research on
such recommendation systems has focused on the quality of the accessed Web resources
and assisting students to obtain useful and creditable Web information.
Different forms for computer-mediated communication applications and social media
have been utilized in E-learning [10]. For example, using asynchronous online discussion
forums has been shown to be effective in constructing deep knowledge among students [11].
Forms of social media such as Wikis, blogs, microblogs, etc, have been applied to facilitate
formal or informal learning of students [5,12]. However, some researches show that it is
doubtful in the effectiveness for using social media in learning [13]. Besides, the success of
using social media greatly relies on the participation of students [14].

2.

Method

20 students, who were in-service teachers of primary or secondary schools, from a course
were invited to conduct the experiments. The students had general knowledge of computer
and Internet usage. We setup a social bookmarking system for our classes1. A 30-minute
training session about the usage of the social bookmarking system was given to all students
to familiarize how to search for, tag, and share Web resources. In addition, the criteria of a
good Web resource and examples of good Web resources were illustrated.
An experiment with experimental group and a control group was used as the overall
research design. Students were divided into two groups. The first group consisted of 11
individual students who searched for Web resources using their own method. This was the
control group of the experiment. The second group consisted of 9 individual students who
used the social bookmarking system to search for and share Web resources. Students from
both groups were asked to spend 30 minutes to search for relevant and useful Web resources
in a topic. During this period, they were not allowed to communicate, so as to demonstrate
the collaboration effect of social bookmarking. In the experiment, they were asked to search
for resources about information literacy. The obtained Web resources were then
evaluated quantitatively and qualitatively. Quantitatively, we measured the number of Web
resources found by each student, and the number of Web resource tagged by each subject.
Qualitatively, we invited human experts to assess different aspects of each of the Web
resource found by the students. Note that the human experts did not know the students from
whom the Web resources originate. Each human expert is required to complete the
following survey for each Web resource. Some questions of the survey are referenced to
[2,9]. For each of the questions, a 5-point rating is used to score the resource, where point 1
refers to the most disagree and 5 refers to the most agree.
Q1: The resource is relevant to the topic.
Q2: The resource is trustworthy (e.g., the resource is published in an authority
Web site, journals)
Q3: The resource is authoritative (e.g., the resource is published with names of
authors, contain list of references)
Q4: The resource is accurate (e.g., the resource does not contain incorrect
information)
Q5: The resource is unbiased (e.g., the resource does not have related
advertisement)
Q6: The resource is informative (e.g., contain advanced knowledge, analysis)
1

We employ the open-source social bookmarking system scuttle, which can be obtained in
http://sourceforge.net/projects/scuttle/.

Q7: The resource is well-organized (e.g., the resource is easy to follow.)


The questions of the survey were provided to student as assessment rubrics to the Web
resources acquired.

3.

Results and Discussion

Table I shows the statistics obtained from the experiments. We have conducted statistically
hypothesis testing, specifically, paired t-test, to evaluate the following hypothesis.
Hypothesis:

There is no improvement in the quality of the acquired Web


resources via using social bookmarking.

For Q1 of our survey, the average points of the experimental group and the control group
were 4.46 and 4.25 respectively. It could be observed both experimental group and control
group could acquire relevant Web resources for a particular topic. However, the points of
the experimental group were significantly higher than those of control group from Q2 to Q7.
It showed that using social bookmarking systems could improve the quality of the Web
resources. The major reason was that students could collaboratively search for good Web
resources, and share them via tagging. Students could give meaningful descriptive tags the
Web resources, which were considered to be in high quality by the students. On the
contrary, the students in control group could only search for Web resources individually
without any collaboration and communication, and hence good resources could not be
shared.
Table I. Statistics of the experimental group and control group
Experiment Group
Control Group
Question
Mean
S.D.
Mean
S.D.
Q1
4.46
0.65
4.25
0.64
Q2
3.86*
0.99
3.47
0.94
Q3
3.80*
0.95
3.40
0.88
Q4
4.09**
0.77
3.66
0.73
Q5
4.17*
0.65
3.77
0.90
Q6
4.14*
0.83
3.66
0.82
Q7
4.09**
0.81
3.66
0.87
*p < .05, **p < .01
The experimental group and the control group obtained a total of 26 and 39 different
Web resources respectively. The average numbers of different Web resources acquired by
each student in the experimental group and the control group were 2.88 and 3.54
respectively. Experimental group obtained less but better quality set of different Web
resources because students could read the shared resources via the social bookmarking
system. For the control group, students obtained a more diversified Web resources, but the
average quality was less satisfactory. This illustrated that students using social
bookmarking obtained a more focused set of good resources. This is particularly useful if
students are asked to have group discussion during classes after studying these resources
because they have a better common ground for discussion. In light of this, social
bookmarking can be effective for after-school learning. Students can access the social
bookmarking system via the Internet and collaboratively search for Web resources.
After-school learning activities can be carefully designed so that students can leverage

social bookmarking to receive useful information and knowledge. For examples, tasks of
searching for resources of a particular topic can be designed as after-school learning
activities for students. Such tasks can help students obtain additional information and deep
knowledge outside classes.

4.

Conclusions

This study conducted an experiment to investigate the use of social bookmarking to acquire
quality Web resources. The results indicated that the students can obtained a more focused
and better set of Web resources in terms of different dimensions such as relevance,
trustworthiness, etc. The findings suggest that social bookmarking can be effective for
after-school learning. Particularly, social bookmarking can be integrated into project-based
learning or inquiry-based learning. Project-based learning aims at engaging students
investigating and tackling challenging problems. The objective of inquiry-based learning is
to construct knowledge via generating, investigating and answering questions. Social
bookmarking can help students obtain good resources in a collaboratively to analyze the
problems and make decision, facilitating project-based learning and inquiry-based learning.

Acknowledgements
The work described in this paper is substantially by the Internal Research Grant from the
Hong Kong Institute of Education, Hong Kong (Reference Number: RG 76/2010-2011).

References
[1] Metzger, M. J., Flanagin, A. J., & Zwarun, L. (2003). College student Web use, perceptions of
information credibility, and verification behavior. Computers & Education, 41 (3), 271-290.
[2] Kubiszewski, I., Noordewier, T., & Costanza, R. (2011). Perceived credibility of Internet encyclopedias.
Computers & Education, 56 (3), 659-667.
[3] Lee, Y.-J. (2005). VisSearch: A collaborative Web searching environment. Computers & Education, 44
(4), 423439.
[4] Mangina, E., & Kilbride, J. (2008). Evaluation of keyphrase extraction algorithm and tiling process for a
document/resource recommender within E-learning environments. Computers & Education, 50 (3),
807-820.
[5] Huang, S.-L., & Yang, C.-W. (2009). Designing a semantic bliki system to support different types of
knowledge and adaptive learning. Computers & Education, 53 (3), 701-712.
[6] Durlak, J.A., & Weissberg, R.P. (2007). The impact of after-school programs that promote personal and
social skills. Chicao IL: CASEL.
[7] Kong, S. C. (2008). A curriculum framework for implementing information technology in school
education to foster information literacy. Computers & Education, 51 (1), 129-141.
[8] Kong, S. C., & Li, K. M. (2009). Collaboration between school and parents to foster information literacy:
Learning in the information society. Computers & Education, 52 (2), 275-282.
[9] Brandt, S. D. (1996). Evaluating information on the Internet. Computers in Libraries, 16 (5), 44-46.
[10] Barnes, S. B. (2003). Computer-mediated communication: Human-to-human communication across the
Internet. Boston, MA: Allyn and Bacon.
[11] Berge, Z., & Collins, M. (1995). Computer-mediated scholarly discussion groups. Computers &
Education, 24 (3), 183-189.
[12] Cavus, N., Bicen, H., & Uzunboylu, H. (2011). The efficient virtual learning environment: A case study
of web 2.0 tools and Windows live spaces. Computers & Education, 56 (3), 720-726.
[13] Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches.
Computers & Education, 52 (1), 141-146.
[14] Hrastinski, S. (2009). A theory of online learning as online participation. Computers & Education, 52 (1),
78-82.

The Role of Teacher-Teaching Experience in


VAE Teachers Integration of ICT
Mohd Khairezan RAHMATa* & Wing K. AUb
School of Education, University of South Australia, Australia
b
School of Education, University of South Australia, Australia
rahmk001@mymail.unisa.edu.au
a

Abstract: The introduction of Information and Communication Technology (ICT) and its
integration has made a significant impact on the Malaysian education system. Its rapid
development has become a major priority in achieving the nations Vision 2020 of
becoming a fully developed country by the year 2020. Despite its huge potential advantages,
the Visual Art Education (VAE) teachers were reportedly not utilizing the full potential of
ICT in the Visual Art classroom. Based on the literature, it appeared that only a limited
published research had tried to explain the influence of teachers experiences on their ICT
integration (ICTInt). The purpose of this study is to explore the VAE teacher- teaching
experience as moderator variables in their decision to integrate ICT. The Technology
Acceptance Model (TAM) was used as a theoretical grounding of the study. Based on TAM,
perceived usefulness (PU) and perceived ease of use (PEoU) were two specific beliefs that
determine individuals ICTInt. Self-reported data was gathered through an on-line survey
from 296 VAE teachers in the state of Selangor, Malaysia. The findings revealed that both
PU and PEoU were able to explain 65 percent of the variance toward their ICTInt. The
findings also show that teacher-teaching experience was found to influence the relationship
between their PU and PEoU toward ICTInt. It is expected that findings derived from this
study will contribute toward assisting the school administrator and the Ministry of
Education in ensuring all teachers would pose and instill a degree of ICT integration.
Keywords: Perceived usefulness, perceived ease of use, ICT integration

Introduction
Over the past two decades, the Malaysian government has been proactive in fulfilling the
demands of a technologically literate, creative and innovative workforce for the Information
Age. As an important agent of change, teachers are required to make a complex adjustment
and substantial rethinking of new possibilities in teaching. However, local researches have
indicated ICT integration is far from reaching its target. Many senior teachers are also
reported reluctant to change their conventional way of teaching [1]. The notion of users
experience having an impact on their decision to integrate a proposed technology has been
highlighted by previous researchers [2]. The embracement of ICT in VAE classrooms will
provide opportunities for students to construct their own knowledge, meaning and solution.
The potential usages of ICT in conducting successful art instruction have also been noted
through its simulation, manipulation and creative expression activities [3]. Despite those
advantages, many VAE teachers were reported to be reluctant to embrace ICT into the VAE
classroom [4]. Given the importance to accelerate the integration of ICT, the study attempts
to explain the relationship between VAE teachers perceived usefulness (PU) and perceived
ease of use (PEoU) toward ICT integration (ICTInt). It is envisaged that the findings of this
study will act as a guide and reference for school administrator and the Malaysian
government toward establishing a standard of successful integration of ICT.

1.

Theoretical Framework and Hypotheses Development

Adopted from the Technology Acceptance Model (TAM), users PU and PEoU were two
specific beliefs that determine their decision to integrate a proposed technology. PU is
defined as individual degree of beliefs that using particular technology would enhance job
performance; while PEoU is defined as individuals degree of beliefs that using particular
technology would free them of any effort [5]. Although users might perceive a proposed
technology as being advanced, they will not adopt it if they think the technology cannot
facilitates them [6]. The first hypothesis was formulated as:
H1: The VAE teachers PU and PEoU will positively influence their ICTInt.
In determining a potential moderator that might influence teachers intention to
integrate ICT, their teaching experience was also considered. Previous research on students
acceptance of Web-based courseware by Stoel and Kyu [7] found that as the experience
with technology increases, users perceive it to be easier and more useful, which in turn
leads to more usage. In similar vein, Ramayah [1] research found that prior experience has
moderated the relationship between PU, PEoU and computer usage. Based on the following
justifications, the second hypothesis is stated as below:
H2: The relationship between the VAE teachers PU and PEoU toward ICTInt is moderated
by their teaching experience.

2.

Research Methodology

This study employed a quantitative research method through 20 items on-line survey. Both
PU and PEoU questions were adopted from the TAM questionnaire [5]. The items used to
measure users ICTInt were adopted from the UTAUT questionnaire [8]. A five point
Likert-type scale was employed, starting with SD as strongly disagree, D = disagree, U =
undecided, A = agree and SA = strongly agree. The VAE teachers in the state of Selangor
(n=887), Malaysia were identified as a target sample of the study. Out of that number only
296 (33.4%) teachers responded and thus become the respondent of this study. This
response rate was acceptable for an on-line survey [14].

3.

Research Findings and Results

3.1 Reliability and Validity of the Instrument


The Cronbachs alpha coefficient was used to test the reliability of items of the
questionnaire. As can be seen from Table 1, all the measurement items were considered to
be good (exceed 0.80) [15]. The result also indicates a large correlation between items
(exceed 0.50), thus confirming the reliability of the instrument.
Table 1: Cronbachs Alpha and Item-to-total Correlation Values
Measurement Items
No. of
Cronbachs
Item-to-total
Items
Alpha
Correlation
Perceived Usefulness (PU)
6
0.882
0.659 0.735
Perceived Ease of Use (PEoU)
6
0.881
0.620 0.742
ICT Integration (ICTInt)
8
0.806
0.500 0.858

A factor analysis was used to test the construct validity of the questionnaire. The result
suggested that all items were represented by three factors. The KMO value was 0.860 and
the significant level was .000. These results thus indicate that all items are tapping and
collapsing into the same constructs [15].
3.2 Descriptive Analyses
This sub-section presents the descriptive analyses of teachers teaching experience.
Teachers maximum years of teaching experience were divided into three groups, which
represent the low, moderate and high experience teachers. As can be seen from Table 2, a
majority (67.9%) of the respondents were categorized as low experience, where their
teaching experience was less than eight years on average.
Table 2: Summary of Characteristics of Respondents of the Study
Characteristics
Group
Cases (n)
Percentage (%)
Teaching Experience
Low Experience
201
67.9
Moderate Experience
60
20.3
High Experience
35
11.8
3.3 Hypothesis Testing
H1: The VAE teachers PU and PEoU will positively influence their ICTInt.
The result reported from Table 3 shows that the significant F-Change value was highly
significant (p<0.01). The coefficient determination (R-square) value was found to be 0.651;
which indicates that 65 percent of the variance of the VAE teachers ICTInt can be
explained by their PU (B = 0.77, p<0.05) and PEoU (B = 0.79, p<0.01). This result indicates
that research hypothesis 1 was accepted.
Table 3: Regression Analysis between the Teachers PU and PEoU toward ICTInt
Model
Unstandardized
Standardized
t
Coefficients
Coefficient
B
Std. Error
Beta
PU
0.112
0.050
.077
2.227*
PEoU
0.733
0.32
.797
23.025**
R Square
0.651
R Square Change
0.651
F Change
273.583
Sig.-F
0.000
*p<0.05; **p<0.01
H2: The relationship between the VAE teachers PU and PEoU toward ICTInt is moderated
by their teaching experience.
It is apparent from Table 4, the significant F-Change value in the third block was
significant (p<0.01) when the moderator was entered in the second block. The R-square
value of 0.507 in the third block also indicates that 50.7 percent of the variance between
teachers PU and PEoU toward ICTInt was moderated by their teaching experience. This
result indicates that hypothesis 2 was accepted.

Table 4: Moderating Effect of Teachers Teaching Experience on the Relationship


between Teachers PU and PEoU toward their ICTInt
Variables
Standardized
Standardized
Standardized
Beta Step 1
Beta Step 2
Beta Step 3
Predictors
PU
.106*
.097*
.075
PEoU
.633**
.597**
.853**
Moderator
Teaching Experience
-.251**
.213
Interaction Terms
PU*Teaching Experience
-.680**
PEoU*Teaching Experience
-.196
R Square
.429
.491
.507
Sig. F-Change
.000
.000
.010
*p<0.05; **p<0.01
Two graphs were drawn in showing the moderating effect of teachers teaching
experience. Firstly, teachers PU and PEoU were categorized into two levels (low and high)
based on its median score. From Figure 1, it is apparent that ICTInt among the VAE
teachers who gained low teaching experience increase dramatically when ever their level of
PU and PEoU was high. This finding also confirms that VAE teachers teaching experience
was moderate the relationship between their PU and PEoU toward ICTInt.

Figure 1: The Moderating Effect of Teachers Teaching Experience on the Relationship


between Teachers PU and PEoU toward ICTInt

4.

Discussion

The aim of the study is to test the relationship between the VAE teachers PU and PEoU
towards their ICTInt in the VAE classroom. Result of this study found that both PU and
PEoU have positive significance effect on the VAE teachers ICTInt. This finding is
consistent with beliefs postulated in the TAM and UTAUT. It is confirmed that individuals
integration of technology is increased whenever the technology is perceived to be useful and
easy to be used [1]; [6]. In ensuring better acceptance to the proposed technology, it is also
possible to conclude that the technology should be perceived as better than the existing ones

[3]. The result from this study also shows that teaching experience does moderate the
relationship of both the teachers PU and PEoU toward ICTInt. This result support previous
research that individuals experience should be considered in determining their intention to
integrate proposed technology [2]. The result indicates that when a teacher becomes
comfortable and confident with their teaching, they are less likely to be attracted to integrate
ICT into their instruction.

5.

Implication and Conclusion

The result of this study demonstrates that the VAE teachers PEoU was the more influential
driver than PU in determining their ICT integration. Therefore, considering teachers
expectation toward ICT functions are required in proposing its usage in classrooms. The
authorities (viz. the Ministry of Education) need to reflect on suggestions to better meet
teachers performance expectations. Basically, when ICT is perceived easy to use, teachers
feel it is more useful; therefore, they will integrate it. Further, the result of the present study
also suggests that teachers who have experience and confident with their subject area, are
more likely to perceive the usefulness and ease of use of ICT, thus affecting their decision to
integrate it [7]. Finally, the study suggests the needs of ongoing professional development
courses, sufficient facilities and continues supports from authorities in ensuring teachers are
ready to adopt ICT effectively into their instruction. Teachers training colleges and
universities must also take full responsibility in producing knowledgeable, skilful and
confident teachers.

References
[1] Ramayah, T. (2010). The role of voluntariness in distance education students' usage of a course website.
The Turkish Online Journal of Educational Technology, 9(3), 96-105.
[2] Baennett, R., Hartel, C. E. J., & McColl-Kennedy, J. R. (2005). Experience as a moderator of
involvement and satisfaction on brand loyalty in a business-to-business setting. Industrial Marketing
Management, 34, 97-107.
[3] Gregory, D. C. (2009). Boxes with Fire: Wisely Integrating Learning Technologies in the Art Classroom.
Art Education, 62(3), 47-54.
[4] Roland, C. (2007). The Art Teachers Guide to the Internet. MA: Davis Publications.
[5] Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use and User Acceptance of Information
Technology. MIS Quarterly, 13(3), 319-340.
[6] Lai, C., & Gu, M. Y. (2011). Self-regulated out-of-class language learning with technology. Computer
Assisted Language Leaning, 24(4), 317-335.
[7] Stoel, L., & Kyu, H. L. (2003). Modelling the effect of experience on student acceptance of Web-based
courseware. International Research: Electronic Networking Applications and Policy, 13(5), 364-374.
[8] Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information
Technology: Toward A Unified View. MIS Quarterly, 27(3), 425-478.
[14] Sekaran, U., & Bougie, R. (2010). Research Methods for Business: A skill building approach. New
Jersey: John Wiley & Sons.
[15] Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis: A global
perspective (7th ed.). Upper Saddle River, New Jersey: Englewood Cliffs, Prentice Hall.

Designing and implementing e-learning


classrooms to improve students writing
Kevin Kai-Wing CHAN*, Siu Cheung KONG
Department of Mathematics and Information Technology,
The Hong Kong Institute of Education, Hong Kong
* kevin@mysmartedu.com

Abstract: This paper aims to critically review how we can incorporate technology in school
education to provide students with the critical skills needed for the 21st century. This paper
will discuss how the knowledge economy demands a new set of survival skills from our
students, and how appropriate use of pedagogical use of technology can enhance the quality
of school education in the twenty-first century. Technological Pedagogical Content
Knowledge (TPCK) framework and Active-Constructive-Interactive (ICAP) framework
have been reviewed to examine the important balance of content, technology and pedagogy
in designing and implementing effective e-learning classrooms for our students. The design
of an online automated writing evaluation system has been examined to evaluate how it can
improve students English writing and their 21st century skills. This system will be put into
practice in the next stage of this study.
Keywords: Process writing, 21st century skills, English writing skills, evaluation of CAL
systems, automated writing evaluation

Introduction The world is changing FAST!


IBM conducted a survey with 1,500 Chief Executive Officers from 60 countries in 33
industries in 2010 reveals alarming results [11]: 1) Fewer than HALF of the CEOs believe
that their companies are equipped to deal with complexities and velocity of a world that is
changing on a massively interconnected system. 2) The impact of technology on
organizations has jumped from the 6th to the 2nd place in importance, and the executives
believe that more technology-based solutions will be needed in the 21st century. 3)
Creativity has been selected as the most crucial factor for future success in an increasingly
complex world. Are our students equipped with these skills?
Thomas Friedman [8] also makes his case on his book, The World is Flat, that the
world is shifting from an industrial economy to a highly complex knowledge economy. This
global and Web-enabled platform allows any individual, any group, any school and any
organization in the world to use new tools to communicate and collaborate. Wagner argues
that the current school systems only prepare students for the tests, and even the best schools
in the US do not teach the must-have skills students need to have to survive in the 21st
century [17]. He describes such issue as the Global Achievement Gap and proposes that
students must acquire subject content knowledge as well as the seven survival skills for the
twenty-first century: critical thinking and problem solving, collaboration across networks
and leading by influence, agility and adaptability, initiative and entrepreneurialism,
effective oral and written communication, accessing and analyzing information, and
curiosity and imagination.

1.

The 21st Century Skills (21C)

1.1 What are the 21st Century Skills that our students need to develop?
Knight describes that education systems in the 20th century prepared people for work related
to manufacturing, and supported a vocational training mentality [13]. The knowledge
economy in the 21st century focuses on the trade in knowledge through the medium of
communication technology. Knight indicates that there is a need to transform the traditional
models of education with one that reflects the knowledge economy and the need for lifelong
learning. Kong [12] further summarizes the 21st century skills (21C skills) as the skills
needed to achieve the desired learning outcomes/educational goals in the 21st century:
inquiry, critical thinking, communication and collaboration.

2.

E-learning classrooms for the 21st century

2.1 Educational goals for the e-learning classrooms


When we design e-learning classrooms for the 21st century, it is important for us to make
sure that we are providing students and teachers with an environment to support to achieve
the 21st century educational goals [12]: use digital technology to facilitate learning and
teaching, increase students autonomy through the usage of ICT (on determining the
educational goals and learning strategies, shift towards a more learner-centric model), as
well as to provide students with more authentic learning opportunities through simulations,
inquiry and collaborative learning (to develop students inquiry, reflection, communication
and collaboration skills)
2.2 Designing the e-learning classrooms
Many researches indicate that computer-assisted instruction in class had a positive effect on
teaching and learning [6]. Mishra & Koehler [15] make the case that merely introducing
technology to the educational process is not enough. They proposed a conceptual
framework which describes three main components of the learning environments: content,
pedagogy and technology. They emphasize the connections, interactions, affordances
between and among these three components. The Technological Pedagogical Content
Knowledge (TPCK) model suggests that knowledge about content (C), pedagogy (P) and
technology (T) is crucial for effective teaching and learning. However, these three elements
should not be viewed independently, and we should emphasize how these three elements
relate to and complement each other.
Another important area to consider when we design e-learning classrooms relates to
the design of different learning activities for our students. Chi [4] presents a conceptual
Active-Constructive-Interactive framework for differentiating passive, active, constructive
and interactive learning activities. Chi also proposes a hypothesis that interactive activities
(dialoguing) are most likely better than constructive ones (generating) ; constructive
activities will likely be better than active activities (manipulating) ; while active activities
will likely be better than being passive (receiving).

3.

Process writing / ETS Criterion an online automated writing evaluation service


for secondary school students

3.1 Process writing to improve students writing skills


Process writing has proven to be a long-lasting and innovative teaching approach since the
1980s [2005]. Traditional approaches to the teaching of writing focus on a teacher-centric
model and the written product [13]. As a result, students writing pieces are rather
mechanical, and they also lack the skills needed to do free writing [13]. In addition,
struggling students tend to produce writing pieces that are shorter, more poorly organized
and weaker in overall quality [9]. They lack the motivation and confidence in writing [3].
Flower and Hayes [7] are among the first group of researchers who studied writing as a
cognitive and problem-solving process. They identify that writing can be viewed as a set of
unique thinking processes, and these processes have a hierarchical and non-linear structure.
In addition, Flower and Hayes indicate that writing is goal-oriented, and that writers explore
and refine their goals through the process of writing (planning, translating, reviewing and
the monitor). Teachers in Hong Kong were introduced to the process approach to the
teaching of writing in the 1990s [3] & [16]. Research studies on teaching process writing to
students confirm that this approach can be a workable and effective approach in enhancing
students writing skills [2], [3] & [10]. It also increases students confidence in writing and a
greater awareness of the different stages in writing. Despite the long history of its benefits,
the process approach to writing is still not being widely adopted by schools in Hong Kong
[5] & [13]. That could have been caused by the lack of teacher training and time constraints
on both the students and the teachers. Teachers need to allocate extra lessons to go through
the different writing stages and students need the time to do the actual writing.
3.2 Will an online automated writing evaluation service promote process writing?
We are implementing a process writing project with three secondary schools utilizing ETS
Criterion. Criterion is a web-based automated writing evaluation (AWE) service
developed by Educational Testing Service (ETS) in the US to evaluate students writing
skills and provide instant score reporting and diagnostic feedback [1]:
Educational goals:
Improve students subject content, essay writing skills (e.g.
cause-and-effect and persuasive essays), collaboration and critical
thinking skills
Pedagogy:
Process writing & Peer Reviews
Content:
Language across the curriculum (existing subject content being
taught)
Technology:
Criterion online writing evaluation service (with Artificial
Intelligence & Natural Language Processing techniques)
3.3 Technological-Pedagogical-Content Model
Teachers create their own writing assignments or select from a list of predefined topics from
the online library. Once a student submits his/her essay online, the system provides instant
scoring and feedback on errors in grammar, usage, mechanics, styles, as well as
organization & development. Criterion uses an artificial intelligence system with natural
language processing to extract distinct features (over 50 language features) from essays, and
to predict (with statistical techniques) what human raters will score a particular writing
assignment [1]. Students will then be able to quickly revise, edit and re-submit/re-publish
their essays. There are no preset limits on the number of resubmissions, and the ability for

the system to provide instant feedback to the students upon their submission will motivate
them to write more and learn from the trait feedback analysis and comments from teachers.

Figure 1: Students receive instant feedback upon submission of their writing


Teachers will also be able to give students feedback online, and the system also supports
peer-editing. Teachers will also be able to access different analytic reports online to monitor
students writing progress.
Because the Criterion Service is available online, students and teachers can access it
seamlessly from anywhere with an internet connection at anytime school, home,
libary or office. The instant feedback and easy-to-use features encourage frequent writing
practice, one of the keys to improving writing. Students can also choose from writing
templates/scaffolds to plan and organize their essays, as well as communicate with their
teachers and fellow classmates on peer editig The writing service also enables teachers to
spend more time on teaching and focus their instruction in critical areas by reducing time
spent assigning, reading and correct student writings.
Attali [1] and Chen & Cheng [3] reported that students benefited from automated
writing evaluation systems, for example students had higher writing scores [1]] fewer errors
in their resubmissions and increased writing practice. Students could also use the system
effectively during the drafting and revising stages of process writing [3].
That is consistent with Wagners view regarding how this generation is motivated to
learn: instant gratification and use of the web for self-directed learning and peer
interactions. That would allow teachers to better understand their students writing ability
and thinking process. While it supports all stages of English-language writing, the Criterion
service does not grade essay content and cannot take the place of instruction and feedback in
a blended learning environment. We plan to study this area in detail (e.g. how we can most
effectively incorporate this automated writing evaluation system into the Process Writing
curriculum) in the next stage of our study.

4.

Conclusion

This paper critically reviews how we incorporate technology in school education to provide
students with the critical skills needed for the 21st century. Technological Pedagogical
Content Knowledge (TPCK) framework and Active-Constructive-Interactive (ICAP)
framework have been reviewed to illustrate the important balance of content, technology
and pedagogy in designing and implementing effective e-learning classrooms for our
students. The design of an online automated writing evaluation system has been examined
to evaluate how it can improve students English writing and their 21st century skills. This
system will be put into practice in the next stage of this study.

References
[1] Attali, Y. (2004, April). Exploring the feedback and revision features of Criterion. Paper presented at the
National Council on Measurement in Education (NCME), San Diego, CA.
[2] Chan, K.K.W., & Kong, S.C. (2011, December). Process writing: An online collaborative writing
environment for primary school students. Proceedings of the 19th International Conference on
Computers in Education: ICCE 2011, Thailand.
[3] Cheung, M. and Chan, A. 1994. Teaching Writing as a Process. Hong Kong: Education Department.
[4] Chi, M. T. H. (2009). Active-constructive-interactive: A conceptual framework for differentiating learning
activities. Topics in Cognitive Science, 1(1), 73-105. doi:10.1111/j.1756-8765.2008.01005.x
[5] Curtis, A., Heron, A. (1998). On being less innovative: Peer groups and process writing in Hong
Kong.1(1), Asia Pacific Journal of Language in Education, 99-118.
[6] Debevec, K., Shih, M.Y., & Kashyap, V. (2006). Learning strategies and performance in a technology
integrated classroom. Journal of Research on Technology in Education, 38(3), 293-307.
[7] Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and
Communication, 32(4), pp. 365-387.
[8] Friedman, Thomas (2006): The World is Flat: The Globalized World in the Twenty-First Century. London:
Penguin.
[9] Harris, K. R., Graham, S., & Mason, L. H. (2006). Improving the writing, knowledge, and motivation of
struggling young writers: Effects of self-regulated strategy development with and without peer support.
American Educational Research Journal, 43(2), 295-340.
[10] Ho, B. (2006). Using the process approach to teach writing in 6 Hong Kong primary classrooms. New
Horizons in Education, 53, 22.
[11] IBM (2010): IBM 2010 Global CEO Study: Creativity Selected as Most Crucial Factor for Future
Success. Available: http://www-03.ibm.com/press/us/en/pressrelease/31670.wss
[12] Kong, S. C. (2007). The development and validation of an information literacy model for hong kong
students: Key issues in the professional development of teachers for capacity building. Technology,
Pedagogy and Education, 16(1), 57-75. doi:10.1080/14759390601168031
[13] Knight, C., Knight, B., & Teghe, D. (2006, May 31). Releasing the pedagogical power of information and
communication technology for learners: A case study. International Journal of Education and
Development using ICT [Online], 2(2). Available: http://ijedict.dec.uwi.edu/viewarticle.php?id=167.
[14] Lim, C. P., & Chai, C. S. (2008). Teachers pedagogical beliefs and their planning and conduct of
computer-mediated classroom lessons. British Journal of Educational Technology, 39(5), 807-828.
doi:10.1111/j.1467-8535.2007.00774.x
[15] Mishra, P., & Koehler, M. J. (2006). Technological Pedagogical Content Knowledge: A new framework
for teacher knowledge. Teachers College Record. 108(6), 1017-1054.
[16] Pennington, M. C., Brock, M. N., & Yue, F. (1996). Explaining Hong Kong students' response to process
writing: An exploration of causes and outcomes. Journal of Second Language Writing, 5(3), 227-252.
[17] Wagner, T. (2008) The global achievement gap: Why even our best schools don't teach the new survival
skills our children need--and what we can do about it. New York: Basic Books.

The Use of ICT in the Chinese Classroom: A


Singapore Perspective
Meng HUANG* , Dongbo ZHANG & Shouhui ZHAO
National Institute of Education, Nanyang Technological University, Singapore
*meng.huang@nie.edu.sg
Abstract: The Chinese language (CL) curriculum for Singapore primary schools proposed in
2004 highlights two pedagogical changes: the modular approach and the integration of ICT.
The modular approach puts students into three different groups (or modules, including
bridging/reinforcement, core and enrichment) with an aim at catering to students diverse
learning needs. To respond to differentiated learning as initiated in the new curriculum,
differences in the type of ICT and the frequency of ICT use in actual teaching practice are
expected. This paper endeavors to provide a detailed picture of ICT integration in the Chinese
lessons in Singapore primary schools. 207 Primary two Chinese lessons in 20 schools were
observed, and teachers teaching practices were coded by applying the Singapore Chinese
Pedagogy Coding Scheme. The preliminary results showed that no significant differences of
ICT usage were found across the three modules, which implies that Chinese teachers use of
ICT has not met the needs of differentiated learning as designated in the Modular Curriculum.
Keywords: Primary education, Chinese language teaching, use of ICT

1.

Introduction

1.1 Research Purpose


Singapore is a multiethnic and multilingual country with its population comprised of three
major ethnic groups, namely, Chinese, Malay and Indian. English, Mandarin, Malay and
Tamil are designated as the countrys four official languages. Ever since 1969, Singapore has
been adopting an English knowing bilingualism policy that requires every Singaporean to
master English as well as the language of their own ethnicities, viz., three designated Mother
Tongue languages (MTLs): Mandarin for Chinese, Malay for Malay, and Tamil for Indian
(Pakir, 1993). To be more specific, English is taught as the students first language and the
main medium of instruction in all schools for all subjects except for the MTL courses and a
couple of humanity subjects such as civil and moral education.
As an effect of such bilingual policy, in the Chinese community, an inter-generational
language shift from Chinese languages (including Mandarin and other dialects) to English has
occurred in familial settings. According to the latest Singapore Census (2010), nearly half of
the children entering primary school in that year were reported to use English dominantly at
home, as compared to 9.3% in the 1980s. This home language shift has led to greater
difficulty in CL learning, and a reform in the CL curriculum has been initialized to help
students to adapt and thrive in this globalized world.
In 2004, the Singapore Ministry of Education (MOE) proposed a nation-wide curriculum
reform in CL education in primary schools. The new curriculum reform proposed a

pedagogical change featuring in differentiated learning. It also recommended the use of


information and communication technology (ICT) to enhance students interest in CL learning
and promote the use of CL. In view of such a curriculum reform, this study aims to investigate
how ICT is applied in CL teaching in Singapores primary schools, with special interest to
examine whether classroom teachers use of ICT meets the requirement of differentiated
learning.
1.2 The Chinese Curriculum Reform in 2004
The expansion of global markets and the development of new ICTs have brought a series of
dramatic changes to education in todays world (Darling-Hammond, 2010). To catch up with
such a trend, Singapore also initiated curriculum reforms in various subjects and fields,
including the CL curriculum reform that was piloted and launched in 2006. Before 2006,
students in primary schools were taught with a relatively standardized curriculum. Within
each unit of the textbook, only one text was provided, and the learning focuses were equally
emphasized on the four language skills, namely, listening, speaking, reading and writing. The
pedagogical focuses were mainly on memorization and exam-oriented repetition (CLCPRC,
2004). The core pedagogical focus of the new curriculum (hereafter, the Modular Curriculum)
was to reorient the teaching practice toward a more individually customized and
communicative mode. It highlights a number of changes in pedagogies, among which the
modular approach and the use of ICT are two main focal points. The flexible modular
approach is designed to cater to students with varying levels of Mandarin proficiency.
Meanwhile, ICTs are promoted so as to enhance students communication skills (especially,
listening and speaking) as well as to increase students learning interest.
1.2.1

The Modular Approach

The Modular Approach is a pedagogy based on the concept of differentiated learning. It


intends to provide customized content for students from various backgrounds and with diverse
abilities. For example, different from previous textbooks, each unit in the new textbook series
consists of three different modules.

Figure 1. Structure of Chinese Language Modules for Different Learner Profiles (CLCPRC, 2004, p. 9)

Figure 1 shows the structure of module arrangement from Primary 1 (P1) to Primary 6
(P6). All students take the core module in each unit which focuses on the knowledge
needed for the Primary School Leaving Examination (PSLE). Students who enter schools with
little exposure to CL, typical of children from English dominant families, take the bridging
module, which emphasizes on the building of listening and speaking skills that would assist
them with picking up the core module so as to catch up with their peers. The bridging
module in the unit is replaced by reinforcement module during Primary 3 (P3) and Primary

4 (P4). For P5 and P6, the Basic Chinese course continue to serve the needs of linguistically
weak students. The bridging module and reinforcement module differ in the length of the
texts. The bridging module mainly introduces words useful for learning in the core module.
The reinforcement module in P3 and P4 includes a passage of a topic similar to, but easier
than those in the core module. Additionally, an enrichment module in the unit is added on
top of the core module for those who have the ability and interest to go beyond the core
syllabus.
The curriculum developers believe that through this modular approach, students with
different language capacities and home language backgrounds could receive customized
learning contents. Those who are weak in Mandarin can catch up with the average batch at
their own pace, while the stronger ones will further improve with an enrichment module.
1.2.2

The Use of ICT

The use of ICT was greatly promoted in the CL curriculum reform with the hope to enhance
students Chinese learning. More specifically, ICT application in CL pedagogy was to achieve
four goals, namely, to ignite students interest, to facilitate Chinese character writing, to assist
in independent learning and assessment, and to increase opportunities to use CL (CLCPRC,
2004). However, how the integration of ICT use can promote differentiated learning was not
stated clearly in the Report of the Chinese Language Curriculum and Pedagogy Review
Committee (CLCPRC, 2004), the major official document that spearheaded the curriculum
renovation. In other words, there were only general concepts and visions without detailed
guidelines and procedures for teachers to use ICT to enhance their teaching.
1.3 ICT and Chinese Language Education
Buang (2011) introduced the 10C Programme, a Web 2.0 based programme that enabled
students independent learning and peer interaction in learning MTLs. The preliminary
evaluation shows a definite advantage of the new approach (ICT approach) and the new
ICT-based pedagogy for the learning and teaching of MTLs is contributing to the fostering of
learner autonomy in language learning in primary schools in Singapore (Buang, 2011, p.
239). Lim and Tay (2003) reported how different types of ICT tools (informative, situating,
constructive, and communicative tools) were used to engage Singaporean students in higherorder thinking. They pointed out that different types of ICT tools were often used to
complement one another to achieve the teaching objectives. Moreover, the classification of an
ICT tool is based more on how it is used than its characteristics, and effective management of
digital instructional resources tends to ensure seamless and easy retrieval and supports the
integration of ICT into the curriculum.
Since the use of ICT and differentiated learning are two major components of the
modular curriculum, it is our focus to see how the use of ICT could be integrated into
differentiated learning, and whether CL teachers actual teaching practices would show any
module-wise variation with respect of the use of ICT. To be more specific, since the aim of
the modular curriculum is to provide learning contents tailored up for students with diverse
proficiency in CL, we expected that teachers would adopt customized pedagogies for students
in different modules. Thus, the use of ICT in different modules was expected to be different in
terms of both type of ICT tools and frequency of their use.

2.

Research Methods

2.1 Participants
This study adopted the method of class observation to investigate the use of ICT in different
modules in Singapore primary schools. The video recordings were obtained from a large-scale
ongoing research project at the Centre of Research in Pedagogy and Practice, National
Institute of Education, Singapore. In that project, three classes of different modules (bridging,
core, and enrichment) in twenty primary schools P2 cohort were randomly sampled to be
observed. For each class, one weeks worth of lessons were systematically observed and
recorded because one unit should be taught within one week according to MOEs curriculum.
Each lesson lasted from thirty minutes to 1.5 hours. In total, 207 lessons were observed,
including 56 bridging lessons, 68 core lessons, and 74 enrichment lessons.
During the observations, the researchers used the Singapore Chinese Pedagogy Coding
Scheme (SCPCS) to code each lesson. This coding scheme was developed on the Singapore
Pedagogy Coding Scheme (Luke, Freebody, Shun, & Gopinathan, 2005). The SCPCS was
modified to facilitate classroom observation, help capture the features of teachers
pedagogical practices, and examine the similarities and differences among different modules.
Each lesson observed was divided into several phases according to teachers teaching
activities. Each phase lasted at least three minutes, and within each phase, the pedagogical
focus, the teaching tools, code switching, the learning tools students used, and students
engagement were coded according to the coding manual. In the present study, we focused
only on the data of teachers teaching tools. We recorded exactly whether the teacher had
used any tools and what kind of tools the teacher adopted.
Since lessons were observed across three modules, it was expected that teachers use of
tools, especially ICT tools, would vary from module to module. To be more specific, students
in the three modules would be taught with diverse pedagogical approaches since they
possessed different CL capacities, and miscellaneous tools would be applied so that the aim of
differentiated learning could be achieved. In order to investigate the differences of the use of
ICT among the three modules, one-way between subjects ANOVA was conducted to compare
the effect of module type on various conditions.

3.

Preliminary Results

Since the study mainly focused on teachers use of ICT, this paper only provides the results of
the following categories, namely, Teachers Tool_Nil (no ICT usage), Teachers Tool
_ICT_total (including Teachers Tool_Traditional ICT and Teachers Tool_New ICT), and
Teachers Tool_Traditional (teacher used traditional tools such as textbooks, whiteboard,
worksheets, etc.). The use of traditional ICT such as PowerPoint, Audios and Videos was
coded within the sub-category of Teachers Tool_Traditional ICT, whereas the use of new
ICT such as one-to-one computer, interactive whiteboard and multi-media involving
interaction was coded under the sub-category of Teachers Tool_New ICT. This paper
examines the use of ICT across the three different modules (bridging, core, and enrichment).
One-way ANOVA results showed that for Teachers Tool_NiL, there was a significant
difference between Core and Bridging modules and between Enrichment and Bridging
modules, F(2, 195) = 6.577, p = .002. However, no significant differences were found across
modules, F(2, 195) = 1.057, p = .350 for Teachers Tool _Traditional ICT, F(2, 195) = 2.796,

p = .064 for Teachers Tool_New ICT, F(2, 195) = 1.057, p = .349 for Teachers
Tool_ICT_total, and F(2, 195) = .085, p = .918 Teachers Tool_Traditional.

4.

Conclusion and Discussion

The above result indicated a very intriguing finding, that is, how to integrate the use of ICT
into differentiated learning. Although the adoption of the modular approach and ICT
attempted to band the weak with the average through differentiated pedagogies, the
expectations did not seem to be well met when the intention of the curriculum reform was
translated and carried out into practices. The result implies that teachers use of ICT in
Singapore CL classrooms has not met the needs of differentiated learning. CL teachers
pedagogical practices in general, and ICT use in particular, remained the same even though
they were conducting lessons to students with different CL proficiency. Therefore, in order to
fulfill the purpose of differentiated learning, teacher training should consider reinforcing the
integration of ICT in classroom pedagogies to accommodate learners diverse learning needs.

Acknowledgements
The data used in this paper is derived from the project Effectiveness of the Chinese Modular
Curriculum in Singapore Primary Schools: An Evaluative Study funded to the third author
by the Office of Education Research, National Institute of Education, Singapore. Much
appreciation goes to Mr. Wang Chu Meng and Miss Janice Lim Xiaoyuan.

References
[1] Buang, A. B. (2011). ICT-Based teacher-facilitated and self-directed learning for Mother Tongue
Languages. In D. Gardner (Ed.), Fostering autonomy in language learning (pp. 233-239). Gaziantep: Zirve
University.
[2] CLCPRC. (2004). Report of the Chinese Language Curriculum and Pedagogy Review Committee.
Singapore.
[3] Darling-Hammond, L. (2010). The flat world and education. New York: Teachers College Press.
[4] Department of Statistics. (2010). Singapore census of population 2010: Advance census release. Retrieved
from http://www.singstat.gov.sg/pubn/census2010.html.
[5] Lim, C. P., & Tay, L. Y. (2003). Information and communication technologies (ICT) in an elementary
school: Students' engagement in higher order thinking. Journal of Educational Multimedia and
Hypermedia, 12(4), 425-451.
[6] Luke, A., Freebody, P., Shun, L., & Gopinathan, S. (2005). Towards research-based innovation and reform:
Singapore schooling in transition. Asia Pacific Journal of Education, 25(1), 5-28.
[7] Pakir, A. (1993). Two tongue tied: bilingualism in Singapore. Journal of Multilingual and Multicultural
Development, 14(1/2), 73-90.

Analyzing ICT literacy of German Teachers:


Focusing age and gender issues as well as
identifying knowledge levels
Johannes ZYLKA
Center for Research on Educational Quality and Evaluation,
Centre for International Student Assessment (ZIB),
German Institute for International Educational Research (DIPF),
Frankfurt, Germany
zylka@dipf.de

Abstract: Owing to the increasingly fast development of media and especially the
advancement of computer technology in recent decades, the topic of ICT integration in
classrooms is being intensively discussed on different levels and media educators argue
about the missing link between school education and the use of ICT in schools. In this
respect, ICT-related competencies of teachers are supposed to play an important role, but
approaches assessing ICT-related competencies in general as well as in terms of the
teacher profession rarely exist and, thus, extensive data on ICT literacy of teachers is
missing. The presented article focuses on identifying ICT knowledge of German teachers
and teacher trainees, taken as a central aspect of media and ICT literacy, and in this respect
especially addresses age-related and gender issues as well as different levels of ICT
knowledge.
Keywords: ICT literacy, Media literacy, ICT knowledge, teaching profession, gender issues

Introduction
Schools worldwide are confronted with a variety of requirements related to digital media,
for instance when looking at current developments such as mobile devices, cloud
computing, collaborative working, or Game-based Learning [1]. Taking into account these
developments, it is evident that the lacking integration of media literacy education in formal
education has reached a new level of significance and will continue to gain importance
[2].Thus, for teacher professionals in the 21st century, the ability to handle digital media in
different forms and scenarios is a central part of their every-day life [3].
Searching for studies focusing media or ICT literacy, it gets clear that this is an area of
research which has not been extensively analyzed so far. There are at least some approaches
discussing media use or media related attitudes of teacher professions [4], analyzing
computer availability of teachers at home [5], or discussing the necessity of media literacy
education on a theoretical level [6], but a minor amount of studies focuses ICT literacy of
teacher professionals [7], especially accounting for different groups of teacher
professionals: student teachers (initial training), teacher trainees (advanced training) as well
as experienced teachers. Therefore, the presented article will analyze teachers knowledge
on ICT as predictor of ICT competence and ICT literacy, enabling conclusions on the
readiness of teachers for the use of ICT in their profession. This article discusses
ICT-related competencies of teachers in context of ICT integration in schools.

1.

Integration of ICT in schools

In terms of school contexts, desktop or laptop computers still represent the most frequently
used devices. The PISA studies showed that nowadays most students are quite well
equipped with computers at home and that the school-related computer-per-student ratio has
significantly improved comparing the situations between 2000 and 2009 [8]. Besides ICT
availability at school, it is also important that teachers frequently use the available media,
especially in classroom contexts. Thus, as important as data on the school-related
availability of ICT is, it is not sufficient to allow statements on the implementation of ICT in
schools. Studies have shown that the use of ICT in schools by teachers as well as by students
differs a lot from the given possibilities. Based on these data, it can be shown that teachers in
those countries where schools are equipped with computers are not necessarily using them
in classroom contexts or other school-related activities. While in some ICT-equipped
countries there is a relatively intensive use of computers in school contexts, the situation in
other countries such as Germany differs: The level of computer availability is at least
sufficient, but the use of ICT in school contexts differs strongly from the given possibilities.
This is amongst others leading to a missing integration of ICT in learning scenarios as
well as a lack of media literacy education of students [9].
In this context, discussions lately noticed that ICT-related competencies of teachers are
quite important in terms of this gap and, therefore, in context of using ICT potentials in
school [10]. As mentioned above, researchers have mostly focused on the availability and
use of ICT in schools or on the curricular grounding of media literacy education in the last
decade. Searching for literature on teachers media and ICT competencies, it seems clear
that this topic has not yet been an area of major research interest [11].

2.

Discussing Media Literacy of teachers

Extensive data on media and especially ICT-related competencies in terms of teacher


profession are rare. There is a variety of reasons for the lack of research on this topic, for
instance the resistance of formal education against change and especially digital media or
the task of finding teachers who permit studies on their media literacy. Even if data is
provided, it often does not meet the requirements of objectivity, reliability and validity. In
the past, studies have mostly concluded experience-based that teachers lack media
education and media-related competencies [12] or have focused on ICT availability and use.
Hence, the most important challenge might be that the measurement of media competencies
or media literacy has only found little attention in the past. As a consequence, there are for
the best of the authors knowledge no instruments for adequately measuring media
literacy in a quantitative manner, especially talking about the teacher profession.
2.1 Accessing ICT Competencies of Teachers
Speaking of competencies in general, there is quite an extensive discourse on how these can
be measured. For instance, large scale studies such as PISA or TIMSS assume that
knowledge is the most essential basis of competencies. In this respect, declarative as well as
procedural knowledge can be seen as a predictor of competency. While the level of
knowledge is quite important from a theoretical perspective, empirical approaches to media
competencies of teachers in the European area have so far not focused on the level of
knowledge [13]. Therefore, a decision was made to develop a new instrument targeting ICT
knowledge, applicable in the given context. A first version consisted of nearly 400 multiple
choice items, comprehending extensive knowledge on most fields of ICT. Because the

instrument was developed on the basis of the ICT standard work of [14], it can be seen as
theoretically approved. After development, the questions were presented to experts
(originating from the teacher profession as well as from the field of media research) rating
the importance and the adequacy of the questions on a 5-point Likert scale. Finally, the
experts agreed on the high importance and good adequacy of 39 items.
2.2 Description of the study
Based on this newly developed questionnaire, which is named ITK.basic in its German
version, we conducted a study in the German federal state of Baden-Wuerttemberg, which
focused on teachers and teacher trainees, in total including N=393 test persons from the
different levels of teacher education. The smallest study group is the group of teachers (T)
with exactly N=100 test persons. Teacher trainees are differentiated by student teachers
(ST), who are studying teaching at university (initial training), and teacher trainees (TT) in
the second phase of their vocational training, as is common in German teacher education.
The studys results are by no means representative for the federal state of
Baden-Wuerttemberg or the country of Germany. The study was paper-based as well as
computer-based with an internet questionnaire, the assumption being that teachers as well as
teacher students of the second phase would hardly participate in the survey if their
questionnaires were not delivered paper-based. Thus, inquiry of TS was executed
web-based, of TT both web- and paper-based, and of T just using a paper-based version of
the ITK.basic. Since participation in the questionnaire (for the groups TS and partially TT)
was voluntary, it is supposed that students with at least by trend higher ICT knowledge
participated in the study. Thus, the empirical level of ICT knowledge is probably even
overestimated in the presented results.
However, the overall reliability of the questionnaire resulted quite well with
Cronbachs =0.914. A good validity can be assumed because of the expert rating, and
objectivity was achieved due to the study method which was based on a questionnaire.
Overall, no test person was able to solve all the questions of the questionnaire. The one
student with the lowest percentage of correct answers scored just about 5% correct answers,
those with the highest knowledge levels around 95% correct answers. The most difficult
items were solved by only 13 persons (3%), while the easiest question was solved by all of
the probands.
2.3 Central findings from the study
One of the recently discussed aspects related to ICT competencies not only speaking of the
teacher profession is age-dependence. It is mostly assumed that older persons do not know
very well how to use computers and especially the latest ICT devices, which is a matter of
concern because of the given age ratio in terms of the school systems in most developed
countries. Thus, as first focal point of this study it is supposed in the presented data that
older subjects score significantly less than younger ones. In addition, some authors argue
that students are just partially media literate and that students choosing to become teachers
mostly are not media literate. As teachers are expected to teach students ICT literacy as well
as use ICT in their teaching activities, their own ICT knowledge becomes an essential
predictor not only for their ICT literacy but also for their teaching competencies. Coming to
the presented study: While there is no significant correlation of the subjects age with their
ICT knowledge (r=-.099, p=.07, N=334), the heterogeneity from 20 years to beyond 60
years is immense. There were students who solved over 95% of the questions as well as
students solving below 20%. Therefore, data suggests that groups of teachers with
significant gaps in media knowledge can be found at all ages. In terms of data analyses,

there are four groups that can be differentiated amongst the test persons. The following
differentiation of groups should be understood as approximating in terms of practicable
visualization of data barriers between different groups do not represent fixed levels.
First, there is the group of expert users with profound ICT knowledge that solved more
than 80% of the items in the questionnaire. The members of this group can be characterized
by extensive knowledge in most areas of ICT. It therefore is assumed that subjects
belonging to this group are well equipped with knowledge to use ICT in a self-determined
way and are also able to use ICT in different pedagogical scenarios. It is also supposed that
these test persons are somehow engaged and interested in ICT. 59 out of 393 test persons
belong to this group, 56% of which are student teachers, 17% teacher trainees and 27%
teachers.
Members of the second group, which is referred to as standard users, scoring between
50% and 80 %, are supposed to have some ICT-related knowledge in basic ICT topics
necessary in every-day ICT use. Subjects belonging to this group are supposed to be mostly
standard users, being equipped with a minimum of ICT-related knowledge. This group
might be seen as target audience for on-the-job trainings in media education, because they
have a good basis for further media-related skills and abilities.171 test persons (43%) scored
between 50% and 80% correct answers, 61 (35%) of which are student teachers, 59 (35%)
teacher trainees, and 51 (30%) teachers.
Reaching between 25% and 50% of correct answers, the third group is labeled low
affinity users: Members use ICT as a tool if necessary. They generally have little
motivation to deal with ICT and minor expertise on standard tasks, but might be a good
audience for basic professional training. 134 subjects (34%) and therefore, the second
largest group in this test supposedly belong to the group of low affinity users. 56 (42%) of
these are teacher students, 54 (40%) teacher trainees and 24 (18%) teachers.
Test persons of the fourth and last group solved below a quarter of items and, therefore,
demonstrated a significant lack of substantial ICT knowledge in all areas of ICT. They
should be strongly advised to attend extensive media-related training. Training on-the-job
will mostly not suffice for these test persons, because they lack a basic understanding of
ICT. 28 probands (9%) can be defined as no affinity users, 25% of whom are teacher
students, 43% teacher trainees, and 32% teachers. The fact that this group does not only
consist of older teachers but also of younger teachers aged 20 to 30 shows that todays
students are not necessarily ICT literate.
Analyzing the group with quite low ICT knowledge it is clear that mostly female
students are included here: While there are19 (82%) female students in this group, just 4
(17%) male students possess minor ICT knowledge. Amongst the group of 59 students
which scored above 80%, there are 70% males and 30% females. These aspects lead to the
discussion of general gender-specific issues in terms of this article, which in context of
discussing media and ICT literacy is an aspect of major importance. Gender issues are
generally considered to be important in terms of the formal educational sector because of the
given gender ratio in primary and secondary education. Thus, presented data showed highly
significant differences between male and female probands in all of the groups: For student
teachers (r=.550, p=.000, N=157), teacher trainees (r=.382, p=.000, N=133) as well as
teachers (r=.395, p=.000, N=98). Gender differences are also significant in all included
topics, the strongest differences are recognizable in the knowledge field of data bases
(Mm=.66, SDm=.33, Nm=98; Mf=.36, SDf=.28, Nf=240) as well as Software (Mm=.76,
SDm=.32, Nm=98; Mf=.48, SDf=.29, Nf=240). The least but still highly significant
differences between males and females are extracted from the field of basic knowledge
(Mm=.79, SDm=.20, Nm=98; Mf=.67, SDf=.21, Nf=240) as well as from internet and web
(Mm=.70, SDm=.22, Nm=98; Mf=.52, SDf=.21, Nf=240).

3.

Conclusion and Future Work

Bearing in mind that the ITK.basic was rated by experts to just ask for basic knowledge on
ICT, the results mentioned here show that most teachers, teacher trainees as well as student
teachers do not possess extensive knowledge in ICT. Considering the statement that
presently there are major lacks in terms of measuring media literacy of teachers, the
presented study adds to international educational research. It gives additional information
on the addressed topics of gender issues. The focal point that older teachers generally are
at least compared to younger colleagues not that media and ICT literate could not be
proven in terms of this study. There are teachers older than 60 years who know lot more
about ICT than some students in their early 20s. The instrument that was developed in terms
of this study presents one step towards the development of an instrument which can
empirically measure ICT competencies (not only in terms of teacher education) in a
quantitative manner. Nevertheless, tasks such as the development of instruments focusing
other areas of ICT literacy, remain.

References
[1]
[2]
[3]

[4]

[5]

[6]
[7]

[8]
[9]
[10]

[11]

[12]

[13]

[14]

Johnson, L.; Levine, A.; Smith, R. and Stone, S. (2011). The 2011 Horizon Report. Austin, Texas: The
New Media Consortium
Frau-Meigs, D. & Torrent, J. (2009). Mapping Media Education Policies in the World. New York:
United Nations.
Hague, C. & Payton, S. (2010). Digital Literacy across the Curriculum. A Futurelab Handbook. Bristol:
Futurelab; Johnson, L.; Levine, A.; Smith, R. und Stone, S. (2011). The 2011 Horizon Report. Austin,
Texas: The New Media Consortium;
Vrasidas, C.; Pattis, I.; Panaou, P.; Antonaki, M.; Aravi, C.; Avraamidou, L.; Theodoridou, K. &
Zembylas, M. (2010). Teacher Use of ICT: Challenges and Opportunities. Proc. 7th International
Conference on Networked Learning 2010.
MPFS (Medienpdagogischer Forschungsverbund Sdwest)(2003). Lehrer/-innen und Medien 2003.
Nutzung, Einstellungen, Perspektiven (Teachers and Media 2003: Use, Attitudes and Perspectives).
Retrieved from http://www.mpfs.de/fileadmin/Einzelstudien/Lehrerbefragung.pdf on June 3rd, 2012.
Frau-Meigs, D. & Torrent, J. (2009). Mapping Media Education Policies in the World. New York:
United Nations.
Petko, D. (2012). Hemmende und frderliche Faktoren des Einsatzes digitaler Medien [Positive and
negative aspects of the use of digital media]. In: Schulz-Zander, R.; Eickelmann, B.; Moser, H.; Niesyto,
H. and Grell, P. (Hrsg.). Jahrbuch Medienpdagogik 9. Wiesbaden: Springer VS. pp.38.
OECD (2011). PISA 2009 Results: Students On Line. Digital Technologies and Performance (Volume
VI), p.146. Online: http://dx.doi.org/10.1787/9789264112995-en
Hague, C. & Payton, S. (2010). Digital Literacy across the Curriculum. A Futurelab Handbook. Bristol:
Futurelab.
Buckingham, D. & Domaille, K. (2009). Making Media Education Happen: A Global View. In: Cheung,
Ch.-K. (ed.). Media Education in Asia. Heidelberg u.a.: Springer. pp.19; Camps, V. (2009). Media
Education beyond School. In: Frau-Meigs, Torrent (Hg.): Mapping media education policies in the
world. Vision, Programmes and Challenges. New York: United Nations, pp.189;
Petko, D. (2012). Hemmende und frderliche Faktoren des Einsatzes digitaler Medien [Positive and
negative aspects of the use of digital media]. In: Schulz-Zander, R.; Eickelmann, B.; Moser, H.; Niesyto,
H. und Grell, P. (Hrsg.). Jahrbuch Medienpdagogik 9. Wiesbaden: Springer VS. pp.38.
Petko, D. (2012). Hemmende und frderliche Faktoren des Einsatzes digitaler Medien. [Positive and
negative aspects of the use of digital media]. In: Schulz-Zander, R.; Eickelmann, B.; Moser, H.; Niesyto,
H. und Grell, P. (Hrsg.). Jahrbuch Medienpdagogik 9. Wiesbaden: Springer VS. pp.38.
Zylka, J. (2012). Medienkompetenzen und Instrumente zu ihrer Messung. Entwicklung eines
Wissenstests zu informationstechnischem Wissen von Lehrkrften, Lehramtsanwrtern und
Lehramtsstudierenden. [Media competencies and Instruments for their measurement: Development of a
knowledge test on IT knowledge of student teachers, teacher trainees and teachers]Unpublished
PhD-Thesis: University of Education Weingarten.
Parsons, J.J. & Oja, D. (2010). New Perspectives on Computer Concepts 2011. Boston: Course
Technology/Cengage Learning.

Investigating how to design interactive


learning environments to support students
learning of upper secondary and university
math
Sanela MEHANOVICa*, Daniel SPIKOLb
a
Department of Mathematical Sciences, University of Gothenburg, Sweden
b
Department of Computer Science, Malm University, Sweden
*sanela.mehanovic@chalmers.se

Abstract: Students difficulties in learning of mathematics have for a long time, been
investigated by researchers in different fields. Within educational research there are claims
that technological tools appropriately integrated in students mathematical work can support
their understanding of a wide range of concepts in mathematics. This paper reports on the
initial investigation for the design of Interactive Learning Environments (ILE) to support
students learning of mathematics. The project is guided by the notion of Design Based
Research (DBR) and aims to explore how to design ILE that support students
understanding of integrals in particular. The initial study was conducted at a Swedish
university with 10 students in 4 groups. The study confirmed difficulties in students
understanding of integrals as reported in educational literature and provides a set of design
aims for the next iteration of the ILE to support the learning.
Keywords: mathematics, education, students, interactive learning environments

1.

Introduction

Society needs a well-educated population, who not only actively contributes to the shaping
of the society itself, but who also, as a broadly qualified work force is able to activate and
transfer school content knowledge, insights, and skills to a variety of situations and
contexts. Mathematics, from a societal perspective is recognized as one of the key
components in this process, has lately met considerable difficulties. Schools and universities
across the world meet with an increasing problem with young people having difficulties in
dealing with mathematical content. The use of computers in mathematics education has
often been an underlying goal of presenting mathematical concepts to students in a new and
dynamic way compared to previous learning environments. Some mathematical concepts
are difficult for students to understand when presented in the paper/pencil based teaching
lend themselves to computer representations as in the case with the integral concept [1].
Integrals have visual aspects that can be displayed on a computer screen along the other
representations such as algorithmic, symbolic, numerical, or natural language
representations.
With the use of mathematical software for visualization, the notion of integrals is more
easily adopted by students [2]. On the other hand, it also makes the didactical situation more
complex [3]. A technological tool that becomes a mathematic work tool in the hands of the
students is a process that has turned up unexpectedly complex [4]. The process causes
differentiation in students work with technological tools, meaning that different students

have different experiences and work differently with the same tool and within the same
environment. Furthermore, the work of Guin and Trouche [5] argues the more complex
environment the larger the differentiation of students work methods with these applications
can result in more diverse learning trajectories.
1.1 Research Aim
This paper reports on the initial investigation in the design of an interactive learning
environment (ILE) to support students understanding of integrals. The project is guided by
the notions of design-based
based research (DBR). In education DBR is used to develop and
investigate (content oriented) theories though iterative cycles of intervention and
refinement. DBR aims to combine the intentional design of interactive learning
environments with the empirical exploration of our understanding of these environments
and how they interact with the individuals [6].
The research aim is to explore how to design ILE that support understanding integrals
in particular.. In our opinion, a design of ILE should consider following two aspects: Firstly,
it should attempt to minimize issues related to students difficulties to optimally use
technological tools in their mathematical work. Secondly, the design should,
sho
in parallel,
deal with difficulties in students understanding of a particular mathematical content.

2.

Background

In the upper secondary education, the integral is generally defined in the following
way (see figure 1): Let f(x) be a continuous function
function in a closed interval [a, b] divided in
subintervals with equal length x. Then, for n subintervals we have the area
S f xi
xi. If we let n then xi 0 and it can be shown that
f xi
xi approaches a limit called the integral of f from a to b, which is denoted
f x dx.

Figure 1: Integral defined as an area under the function f(x).


f(x)
This definition based on a Riemann sum is difficult for students to understand.
Students difficulties with integrals are not a new behavior in the mathematics classroom
and had, for the last several decades
decade have been a subject for educational research [1,7,8]. In
the early eighties, Orton observed that students had difficulties while solving tasks related to
the understanding of integration as limit of sums [9]. In this study, students were able to
apply, with some facility, the basic techniques of integration
integration but further probing indicated
fundamental misunderstanding about the underlying concepts. Students interpreted the
integral as a procedure that transforms an input into some output.
output The
he same study revealed
that students technical ability could be quite strong, despite showing minimal conceptual
understanding. Apart from showing strong procedural skills the students were found to
demonstrate a strong reluctance to using geometric interpretations to complete an algebraic
process, and when possible, were more inclined to move to an algebraic context [8].

Another study from Orton (1980) revealed that students had problems with the integral
f x dx if f(x) is negative or b is less than a [9].
More recent studies specialized in mathematics education show that this concept is still
difficult for students to grasp; they are not able to write meaningfully about the definition
of a definite integral nor can they without difficulties interpret problems calculating areas
and definite integrals in wider contexts [10]. The students also intend to identify the definite
integral as an area [11].
2.1 Theoretical Framing
Students understanding of integrals can be discussed from the perspective of the cognitive
structure in their mind that is associated with the concept of integrals. Tall and Vinner [12]
formulate a distinction between the mathematical concept as formally defined and the
cognitive processes by which they are conceived by the students (p.1). The total cognitive
structure that is associated with the concept, including all the mental pictures and associated
processes, they name a concept image, and mean that a students image of a mathematical
concept may not be globally coherent and may have aspects which are quite different from
its formal mathematical definition (p.1). At different times, seemingly different conflicting
images may be activated. The conflicting aspects, that are a part of a students concept
image and/or a concept definition, are called cognitive conflict factors (p.3). As a student
does not necessarily see a conflict while using different methods in their mathematical
work, the student simply utilizes the method he or she considers appropriate on each
occasion [12]. The conflicting aspects that are a part of a students concept image and/or a
concept definition are called cognitive conflict factors (p.3). Only when conflicting aspects
are evoked simultaneously need there be any actual sense of conflict or confusion (p.2).

3.

Methods

Our initial study was conducted to investigate students concept image of integrals, here in
terms of the definite integral, in a way the concept is usually introduced to students (see
figure 1). The study was conducted at a Swedish university and considered an introductory
course in mathematics with 10 students. Four groups were self-created with 2 to 3 students
for the intervention. The participating students, who were just about to finish their
introductory course already had an image of the integral concept, were asked to take a test
containing integral tasks based on a previous research conducted by Rolka & Rsken [13].
This test was developed in order to investigate students understanding of the formal
mathematical definition of the definite integral, and focused on aspects in integrals known
to be difficult for students to grasp.
In the intervention, the students were supposed to within an hour, solve a test with eight
integral tasks. They were asked to solve tasks in the test as a group, and to write their
solution on a whiteboard while discussing a particular task. Their work was videotaped, and,
once they agreed on a solution to a particular task, we took a photograph of their whiteboard
notices. We are currently working on the analysis the video data and the solutions gathered
from observing the groups of students.

4.

Results

The initial study confirmed previous research within mathematics education. While solving
the integral tasks, students have not always been aware of their conflicting images of the

definite integral in relation to its definition. For instance, one of the images that has been
shown to be highly present in the experiment group was the perception of the definite
integral as an area, see figure 2.
In the task shown in Figure 2 (left image) students were asked to calculate the value of
the integral. The task considers the oriented area aspect of the integral and the results
confirmed Rolka & Rskens finding in which many students just equals the concepts of
integral and area [13]. None of the four groups in our initial study came with a correct
answer to this task. Three groups have chosen the option A as the final answer to the task.
Only students in the remaining group considered that the correct answer might be another
option than A. Indeed, they did suggest the correct option (B) although it has been suggested
as a second alternative (even this group had A as their first choice).

Figure 2: Examples of the students perceptions of integrals.


Another task that all participating groups had difficulty with, was to deal with the
problem illustrated in figure 2 (right side). The picture shows two areas A and B. What do
you think is correct for the relation between the areas?
The area of A is bigger than the one of B.
The area of A is smaller than the one of B.
Both areas are equal.
Without any function given explicitly, it is not possible to answer this question
None of the groups had answers that they were certain about, rather, they were
discussing different options having difficulties in choosing between the first and the third
option. What was of a particular interest for the study was the contrast between these two
options that seemed to cause a cognitive conflict for some of the students. The discussed
aspect was following: The first option feels true instinctively, if one thinks that the area of A
can be stretched outside of the interval (still keeping the same height). On the other hand, in
the definition of the definite integral as a Riemann, the area between two curves is
calculated as a sum of areas of infinitely thin rectangles. Only one student started to discuss
the Riemann sum which led the whole group to move their reasoning to what answer option
could be appropriate from the formal definitions perspective. In our opinion, this example
demonstrates how conflicting images evoked simultaneously in students work with a
mathematical concept can lead them to a deeper reasoning of the meaning of the formal
concept definition.

5.

Discussion

The results of our initial exploration point to design goals for the ILE that include further
investigations of the role of the technological tools for students mathematical work with
integrals. Misconceptions about the mathematical problem observed in the student groups

point to the need for support. From our investigation of students understanding of integrals
and from the supporting research literature we are able to identify some implications for the
next iteration of our ILE. In order to support the students to expose their conflicting images
of integrals and gain a deeper understanding of this mathematical concept the following
design goals have been identified:
The ILE needs to guide the students though a learning process that exposes their concept
image of integrals and then supports its development, while taking into consideration a
diversion in students individual perceptions.
This guiding process needs to provide support for individual learning exploration for the
student though some types of externalization like adaptive scaffolding and teachable
agents [14].
Minimize issues related to students difficulties to optimally use technological tools in
their mathematical work by acting as a support component for learning instead of
providing a wealth of features that can be seen at times to distract lower performing
students [4].
For the next steps, we are currently designing a low-fidelity prototype that will explore
teachable agent like qualities that we can test during a summer school mathematics class for
university students. These students will form a new test group for the next round of
participatory workshops with the teachers and researchers.

References
[1] Tall, D. (1991). Visualizing Differentials in Integration to Picture the Fundamental Theorem of Calculus.
Mathematics Teaching, 137, 29-32.
[2] Herceg, D. & Herceg D. (2009). The Definite Integral and Computers. The Teaching of Mathematics, 12
(1), 33-44.
[3] Blomhj, M. (2005). Villkor fr lrande i en datorbaserad matematikundervisning - elevernas
anvndning av avancerade matematikprogram. In B. Grevholm (Ed), Matematikdidaktik - ett nordiskt
perspektiv, (pp. 185-218). Lund: Studentlitteratur.
[4] Artique, M.(2002). Learning mathematics in a CAS environment: The genesis of a reflection about
instrumentation and the dialectics between technical and conceptual work. International Journal of
Computers for Mathematical Learning, 7(3), 245-274.
[5] Guin, D. & Trouche, L. (2002). Mastering by the teacher of the instrumental genesis in CAS
environments: necessity of instrumental orchestration. ZDM 2002 34 (5).
[6] Hoadley, C. M. (2004). Methodological alignment in design-based research. Educational Psychologist,
39(4), 203212. Routledge.
[7] Orton, A. (1980). An investigation into the understanding of elementary calculus in adolescents and
young adults. Cognitive Development Research in Science and Mathematics, 201-215. University of
Leads.
[8] Machin, M.C. & Rivero, R.D. (2003). Using Derive to understand the concept of definite integral.
International Journal for Mathematics Teaching and Learning, 1-16. ISSN 1 473-0111.
[9] Orton, A. (1983). Students understanding of integration. Educational Studies in Mathematics, 14, 1-18.
Dordrecht, Holland, and Boston, U.S.A.
[10] Rasslan, S. & Tall, D. (2002). Definitions and images for the definite integral concept. In Cockburn, A.
D. Cockburn Nardi, E. (Eds), Proceedings of the 26th Conference of the International Group for the
Psychology of Mathematics Education, (Norwich, UK), 4, 89-96.
[11] Attorps, I., Bjrk, K. & Radic, M. (2011). The use of mathematics software in university mathematics
teaching. Paperpresented at CERME 7, The Seventh Congress of the European Society for Research in
Mathematics Education, 9-13th February 2011. Rzeszow, Poland.
[12] Tall, D. & Vinner, S. (1981). Concept Image and Concept Definition in Mathematics with particular
reference to Limits and Continuity. Educational Studies in Mathematics, 12, 151-169.
[13] Rsken, B. & Rolka, K. (2007). Integrating Intuition: the Role of Concept Image and Concept Definition
for Students Learning of Integral Calculus. The Montana Mathematics Enthusiast, ISSN 1551-3440,
Monograph 3, 181-204. The Montana Council of Teachers of Mathematics.
[14] Blair, K., Schwartz, D., Biswas, G. and Leelawong, K.(2011). Pedagogical Agents for Learning by
Teaching: Teachable Agents. To appear in Educational Technology.

Exploring pre-service teachers perceptions of


and participation in online communities
Liping DENGa*, Nicole TAVARESb
Hong Kong Baptist University, Hong Kong
b
The University of Hong Kong, Hong Kong
*liping@graduate.hku.hk

Abstract: The study focuses on the participation of pre-service teachers in a course


website set up on Moodle and a website developed by students on their own initiative to
support their Teaching Practice. It explores PSTs experiences and perceptions of the two
different types of websites and how their online participation linked with the sense of
community. Utilizing questionnaire as the main source of data, the paper reveals that the
student teachers had more positive perceptions of the self-initiated website which was
regarded as a useful platform for exchanging teaching ideas, sharing resources, gaining
support, and maintaining communication with their fellow coursemates. The frequency of
reading the self-initiated website was also correlated with the sense of community.
Keywords: online community, perceptions, pre-service teacher

1. Introduction
The study focuses on the participation of pre-service teachers in a course website set up on
Moodle and a website developed by students on their own initiative to support their
Teaching Practice (TP). We seek to understand how students participated in and perceived
the teacher-initiated and self-initiated website and how their perceptions and participation
are associated with their sense of community. Adopting the case study methodology, the
study investigates the experience of 31 final-year student-teachers in the Faculty of
Education at an university in Hong Kong. The particular questions that guide the inquiry
are: (1) How do student-teachers perceive the course website and the self-developed
website for TP? (2) How are student-teachers perceptions associated with their sense of
community?

2. Literature review
The proliferation of web-based tools opens up new dimensions and brings new meaning to
the notion of community. A community is no longer conceptualized merely in terms of
physical proximity, but in terms of social networks [11]. [5] identified five dimensions of
an online community: commonality, computer system, interaction, social infrastructure,
and social relationships. Online communities could provide their members with multiple
resources including information, social support and emotional support [8]. For some
researchers, an online community is more closely associated with the participants sense
of community. According to [13], a sense of community is a feeling that members have
of belonging, a feeling that members matter to one another and to the group, and a shared
faith that members' needs will be met through their commitment to be together (p. 9). [4]

defined community as a general sense of connection, belonging, and comfort that


develops over time among members of a group who share purpose or commitment to a
common goal (p. 2).
There is a type of online community labeled as a blended community or hybrid
community which is enabled and supported by both online and offline methods of
communication [6]. Such communities are grounded in the rationale that the relationship
between online and offline communication can be supplementary (see [9]). In blended
communities, the existing social relationships, too tight or too loose, might have negative
effects on members engagement and commitment [5]. On one hand, a close community
with abundant opportunities for its members to interact through traditional ways might
make the additional online interaction redundant [1]. On the other hand, if there is only a
loose connection between community members, the incentive for extending
communication to online space will also be low.
A community cannot survive without active and ongoing participation of its members
[10]. Active participation takes the form of creating and consuming content [3]. [14] assert
that the more people participate in online group activities, the more likely they are to
develop online relationships. Likewise, [15] reports that the more time and effort people
invest in the community activities, the greater the chances are for their sense of
community to be ingrained. The reverse is also the case: a sense of community enhances
participation [17]. However, the degree of participation varies significantly among the
members of same community. [2] reported that both participation and achievement levels
were uneven in an online learning community based on voluntary participation. Since the
contribution will turn into public good, the temptation of enjoying the free-ride without
contribution is pretty high [10]. This stems from a social phenomenon known as social
loafing which refers to the tendency of exerting less effort in collective tasks than
individual tasks [12].

3. Methods
3.1 Research setting
The study includes a group of 31 final-year pre-service teachers enrolled in an English
educational program in a comprehensive university in Hong Kong. The course the study
focuses on is a core course with an eight-week Teaching Practice (TP). The instructor was
an enthusiastic and experienced user of educational technology. She set up a course
website on Moodle and tried to engage students in voluntary online discussion as an
extension of face-to-face teaching. She herself was active in facilitating online interaction
among students and responding to students questions. However, the overall level of
online participation was rather sporadic. During the TP, the students themselves took the
initiative to create a website for sharing teaching resources and lesson plans. This
stimulated our interest in the study that examines the phenomenon of student online
participation and their perceptions of the two types of websites.
3.2 Data collection and analysis
The study utilizes questionnaires as the main source of data. The purpose of the
questionnaire was to capture the overall picture of students online behavior and their
perceptions and attitudes towards the two websites. The questionnaire items were
developed based on the researchers previous work on online participation [18] and [16]s
instrument for measuring participants sense of community. The first section of the

questionnaire allowed the researchers to gather data about the participants general
computer skills and comfort level with the use of technology. In the second section
participants were invited to report on their online behaviors such as their frequency of
reading messages. The third section consisted of Likert-type questions that tapped into
participants perceptions, in particular, their perceived usefulness of the websites and their
sense of community (adapted from [16]). The questionnaire was administered on 31
students, yet one student did not complete it, which left the researchers with 30 valid
responses.

4. Results
According to the results from the questionnaire, this group of students was quite
comfortable with computer-mediated communication (Mean =3.17 on a scale of 4 with 1=
very uncomfortable and 4= very comfortable). Their average level of self-rated computer
skills was close to average (Mean = 1.9 on a scale of 3 with 1= weak, 2 = average,
3=high).
4.1 Descriptive data on online activities
Under Learning Forums on Moodle, there were 10 forums created. The first forum
(Forum 1) had the highest number of posts (18) followed by Forum 7 (10). The number of
students who took part in the online discussion as writers was also quite limited. There are
five students who posted in Forums 1, 2, and 7, four in Forum 3 and one in Forum 9. To
provide support and foster interaction among students during TP, the instructor set up a
forum on course Moodle (Forum 10). However, there was no activity in this space at all.
We also asked the students to report on their frequency of reading the content on course
Moodle on a scale of 5 ranging from never to always. 4 students (13%) often and,
16 (53%) sometimes read the content on course Moodle, but 10 (33%) rarely or
almost never (M=2.77) did so.
TP no worries is set up on Google Sites as a publicly accessible website. In
addition to the homepage, there are 12 pages created which are organized under two major
categories: Teaching Materials and Other reference. During TP, the student-teachers
shared quite a number of teaching resources among one another, including their lesson
plans, worksheets, audio and video files. The other section Other reference included
four pages that contained references such as bookmarks, video links, and useful evaluation
forms contributed by students as well. Through the questionnaire, they reported on the
frequency of reading the content on the TP No Worries website. 7 claimed that they had
never accessed the site, 10 (33%) rarely or almost never visited the site, 11
sometimes (37%), and 2 (6.7%) often did so (M=1.93).
4.2 Perceptions of usefulness
In the questionnaire, students were asked about their perceptions of the usefulness of the
two websites. For the course website on Moodle, the students basically thought of it as a
storage place for course materials. Less than one quarter of the students felt the course
website was useful for exchanging ideas and gaining insights into teaching. Students
responses on their perceptions of TPNW were more positive. 87% of them agreed to the
statement that TPNW fostered the exchange of teaching ideas and sharing of teaching
resources, and helped them get peer support from one another during TP. 83% also
acknowledged that the platform supported their communication among peers. 61% felt

that TPNW prompted them to reflect on their own teaching and 52% agreed the website
helped keep them connected despite being in different physical locations for the TP.
We also explored the relationships among the students perceived sense of
community, their online participation, and their perceptions through running the Pearson
correlation. The sense of community (SOC) score was obtained by computing the mean
value of the four items that aimed to gauge students sense of community. The results
show that SOC is correlated with the frequency of reading TPNW (r =.43, p< . 05), but not
with the frequency of reading on Moodle (r =-.1, p> . 05). No association was detected
between SOC and students perceptions of the course website on Moodle. Yet SOC was
strongly correlated with the perceived value of TPNW for exchanging ideas (r =.6, p<
.001), sharing teaching resources (r=.65, p< .001), enabling peer support (r = .55, p <.01),
prompting reflection on teaching practice (r=.46, p<.05), and fostering peer
communication (r= .47, p<.01).

5. Discussion and conclusion


The two websites in the study were developed and used quite differently. The course
Moodle site was set up by the instructor as a place for sharing course materials and
extending in-class discussions. The TP No Worries (TPNW) website was purely
designed, developed, and maintained by the students themselves to promote a stronger
sharing culture during TP. As to the locus of control, the course Moodle site was
perceived by students to be implemented in a top-down fashion with the instructor taking
the leadership and a supervisory role; the TPNW website was a student- initiated website
emerging from the students genuine personal needs for sharing and connection during
their TP. This echoes the findings in [6] about grass-root-initiated online communities. It
was thus interesting to note that the students were found to be less active readers on
TPNW than they were on the course website. This may be interpreted by the fact that the
students all needed to access the course materials on the website at some point in time
during the course. However, their frequency of accessing the course website was not
associated with their sense of belonging to the group. That implies that when CMS is used
mainly for storing information, it did not give students a sense of ownership and hence it
fails to generate a community spirit among them. In line with previous studies (e.g. [7]),
student-teachers in the present study did not perceive the Moodle CMS as a platform for
exchanging ideas and having insightful discussions, thus contributing to their limited
interest in online participation.
On the contrary, those who used TPNW responded very positively to its impact in
fostering peer sharing, communication and support. TPNW was regarded as a useful
platform for student-teachers to exchange teaching ideas, share resources, gain support,
and maintain communication with their fellow coursemates during their TP. It is worth
highlighting that although the results show a lower frequency of reading the content in
TPNW as compared to that in their course Moodle website, the frequency was correlated
with their felt sense of community. Those who were active in TPNW tended to feel a
stronger sense of community. When students visited such the website with lesson plans,
teaching materials and resources all shared by their peers, their sense of belonging was
strengthened and their inclination to access the website heightened. This may have
implications for enhancing students skills in developing and maintaining websites for
their own learning.

Acknowledgement
The study is funded by the seed funding provided by Hong Kong Baptist University.

References
[1] Ardichvili, A., Page, V., & Wentling, T. (2003). Motivation and barriers to participation in virtual
knowledge-sharing communities of practice. Journal of Knowledge Management, 7(1), 6477.
[2] Bruckman, A. (2004). Co-evolution of technological design and pedagogy in an online learning
community. In S. A. Barab, R. Kling, & J. H. Gray (Eds.), Designing for virtual communities in the
service of learning (pp. 239255). Cambridge, UK; New York: Cambridge University Press.
[3] Butler, B., Sproull, L., Kiesler, S., & Kraut, R. (2002). Community effort in online groups: Who does
the work and why? Leadership at a Distance. Mahwah, NJ: Erlbaum.
[4] Conrad, D. (2005). Building and maintaining community in cohort-based online learning. Journal of
Distance Education, 20(1), 120.
[5] Deng, L., & Yuen, H. K. (2007). Connecting adult learners with an online community: Challenges and
opportunities. Research and Practice in Technology Enhanced Learning, 2(3), 1-18.
[6] Gaved, M., & Mulholland, P. (2005). Grassroots initiated networked communities: A study of hybrid
physical/virtual communities. Presented at the 38th Hawaii international Conference on System
Sciences, Hawaii: IEEE.
[7] Hamuy, E., & Galaz, M. (2010). Information versus communication in Course Management System
participation. Computers & Education, 54(1), 169177.
[8] Haythornthwaite, C., Kazmer, M. M., Robins, J., & Shoemaker, S. (2000). Community development
among distance learners: Temporal and technological dimensions. Journal of Computer-mediated
Communication, 6(1). Retrieved from http://jcmc.indiana.edu/vol6/issue1/haythornthwaite.html
[9] Koku, E., Nazer, N., & Wellman, B. (2001). Netting scholars: Online and offline. The American
Behavioral Scientist, 44(10), 17521774.
[10] Kollock, P., & Smith, M. (1996). Managing the virtual commons: Cooperation and conflict in
computer communities. In S. Herring (Ed.), Computer-mediated communication: Linguistic, social,
and cross-cultural perspectives (pp. 109128). Amsterdam: John Benjamins.
[11] Kollock, P., & Smith, M. A. (1999). Communities in cyberspace. In M. A. Smith & P. Kollock (Eds.),
Communities in cyberspace (pp. 325). London: Routledge.
[12] Ling, K., Beenen, G., Ludford, P., Wang, X., Chang, K., Li, X., Cosley, D., et al. (2005). Using social
psychology to motivate contributions to online communities. Journal of Computer-Mediated
Communication, 10(4). Retrieved from http://www.blackwell-synergy.com/doi/abs/10.1111/j.10836101.2005.tb00273.x
[13] McMillan, D. W., & Chavis, D. M. (1986). Sense of community: A definition and theory. Journal of
Community Psychology, 14(1), 623.
[14] Parks, M. R., & Floyd, K. (1996). Making friends in cyberspace. Journal of Communication, 46(1),
8097.
[15] Roberts, T. L. (1998). Are newsgroups virtual communities? CHI98 Conference on Human factors in
Computing Systems (pp. 360367). Los Angeles, CA.
[16] Rovai, A. P. (2002). Development of an instrument to measure classroom community. The Internet and
Higher Education, 5(3), 197211.
[17] Yoo, W. S., Suh, K. S., & Lee, M. B. (2002). Exploring the factors enhancing member participation in
virtual communities. Journal of Global Information Management, 10(3), 5571.
[18] Yuen, H. K., Deng, L., Fox, R., & Tavares, N. J. (2009). Engaging students with online discussion in a
blended learning context: Issues and implications. In F. L. Wang, J. Fong, L. Zhang, & V. S. K. Lee
(Eds.), Hybrid Learning and Education, Lecture Notes in Computer Science (LNCS 5685) (pp. 150
162). Berlin Heidelberg: Springer-Verlag.

Supporting Creativity Learning in Digital


Storytelling with Tablet Computers: A Peer
Assessment Approach
Chen-Chung Liua, Yu-Feng Wua*, Kuan-Hsien Lua, Chia-Ching Lina
a
Graduate Institute of Network Learning Technology, National Central University, Taiwan
*
yfwu@cl.ncu.edu.tw

Abstract: Peer assessment (PA) has been widely adopted in an educational context with ample
evidence suggesting the potential to promote learning. We argue that although the process of
asking learners to involve in assessing achievement of the others might be a beneficial
manipulation for learning, the effectiveness is still unclear for young pupils when creative
learning tasks such as storytelling are involved. Creativity self-efficacy might be interfered with
peers feedbacks. Thus, the presented study proposes an approach to support creativity learning
through peer assessment aided by Tablet computers. This study sought a deeper understanding of
the relationship between creative self-efficacy (CSE) and creativity product among 54 sixth
graders on completing a digital storytelling project. Findings and discussions are included.
Key terms: Creative self-efficacy, Peer-assessment, Digital storytelling

1. Introduction
Who does what to whom We tell stories in our entire lives for that it is one of the
most familiar patterns to make sense of our surrounding world [5]. The process of telling a
story requires reasoning and elaborating where the minds immerse in the scenario,
interchange ideas with personal and cultural values [9], while negotiating and
renegotiating meanings within experiences [3]. Therefore, educators and researchers
consider storytelling as an important pathway to fulfilling instructional objectives.
However, scholars did not recognize storytelling with its educational potential until later
where constructivists started to approach instructional design with learner-centered
instruction, situated learning, learning styles, and so forth [5]. Thus, investigations on
practicing the ability to generate and tell stories should be seen as rehearsals of skills to
effective learning. Robin [10] defined digital storytelling as a combination of multimedia
with computer-based graphics, recorded audio, computer-generated text, video clips, and
music that allows users to become creative storytellers through the traditional process of
selecting a topic, conducting some research, writing a script, and developing an interesting
story (p. 222). However, most of children seem to lack of the opportunity to learn how to
create and tell stories. To tell stories digitally, storytellers need to understand the
fundamental knowledge and evaluate what are good stories based on the knowledge.
However, recent studies of creativity indicated that teaching students to learn fundamental
knowledge may limit the levels of creativity self-efficacy. Thus, there is a need for a
pedagogical approach to enhance students' knowledge about stories and self-efficacy for
the creative activity.
Peer assessment may be a potential approach to address the above issue as literatures
suggested that the practice of assessing one anothers work can help form the ability of

recognizing key performance that is associated to high quality work [12; 16]. Peer
assessment has complicated influence on creativity learning as learners cognitive,
affective, and self-efficacy status may have interfered with the entire learning process [7;
14]. On one hand, by evaluating others' storytelling works, one may know the fundamental
knowledge about high quality stories. On the other hand, previous studies might have
underpinned the complexity of the forming and the effect of peer assessment [13]. Strijbos,
Narciss, & Dnnebier [11] pointed out peer assessment from the more able peers may led
to a negative effect, and it might negatively influence their creative tasks [13]. This study
thus aims to develop a pedagogical approach with peer assessment enabled by Tablet
computers. It is hoped that through the aid of Tablet computers and peers assessment,
students not only can develop fundamental knowledge about storytelling but also sustain
their self-efficacy toward the creative activity.

2. Method
2.1 iPad Storytelling Application
Portable and easy to share make a Tablet computer distinguishable from a desktop
computer, so as to enable the progress of peer assessment and interaction among peers.
Thus to support creativity learning in the storytelling, this study developed an iPad
application. The system allows the storyteller to draw (e.g., using pencil, color picker,
eraser, cleaner, and stamp), to tell (e.g., using voice recorder and background music), and
to frame (e.g., using new frame), in order to produce a digital storytelling project (see
Figure 1)

Figure 1. Interface of the iPad storytelling application.


2.2 Creativity Self-efficacy (CSE) Questionnaire
In order to better understand how peer assessment influence ones creative self-efficacy,
this study adopted a CSE questionnaire that was originally developed by Hung [4]
examining CSE with three dimensions: (1) self-efficacy on creative performance
(CSE-performance), (2) self-efficacy on creative strategy (CSE-strategy), and (3)
self-efficacy on the attitude toward significant negative feedbacks from others
(CSE-Nfeedback). Additionally, the summation of these three factors represents an overall
level of CSE. Cronbachs Alpha reported internal consistencies of the CSE questionnaire
on the overall CSE (.825), CSE-strategy (.78), CSE-performance (.64), and
CSE-Nfeedback (.74), indicating a moderate level reliability.
2.3 Peer Assessment (PA) Form

The current study aimed at a peer assessment process that would facilitate learning of
creativity in the context of storytelling. Based on a previous study [6], we proposed a
5-items-criteria that focuses story grammar, which consisted: (1) general settings, (2)
storyline, (3) creativity, (4) technical performance, and (5) overall presentation of the story.
Such PA was designed for testing our assumption that applying PA could promote
students creativity performance.
2.4 Grading Rubrics and Process of Storytelling Product
Since the Draw&Tell application turns stories as slow motion frames (i.e., slideshows), the
current study adopted a widely used checklist for movie production as the grading rubrics
[15]. The rubrics examine the final production within nine emphases: (1) transitions &
edits 2 items, (2) planning & storyboarding 4 items, (3) action & dialog 2 items, (4)
accuracy of information 2 items, (5) originality & creativity 2 items, (6) sound usage
2 items, (7) drawing 2 items, (8) camera picturing 2 items, and (9) framing 2 items.
Grades were calculated per item from five (i.e., the highest points) to one (i.e., the lowest
points). The grades from these nine categories with total 20 items were calculated together
to indicate an overall performance. Additionally, we would look more closely at one
emphasis of originality & creativity than the others. Two raters used this rubric to grade all
of the 54 final products. The results were tested by correlational analyses and the result
yielded to a range between .539 and .848 (p < .01) which indicated the grading process is
highly reliable.
2.5 Participants & Procedures
Two classes consisted with 54 sixth graders from an elementary school in northern Taiwan
were randomly selected to participate in the current study.
Eight consecutive classes with one hour weekly meeting were scheduled. At the 1st
class, the pre-CSE was administered, and orientation for storytelling and the iPad
Draw&Tell application. Students were then had the opportunity to practice a story for the
1st and 2nd week. From the 3rd to the 8th week, students were assigned a new topic, Saving
the ecological environment, as the formal project. The experimental group started to
review their peers works during the 6th and 7th weeks. Rather than grading on them, they
review works with a PA handout. Students then submitted the handouts to the instructor
without further process regarding PA. As a comparison, the controlled pupils did not know
and process the peer assessment. After the eighth weeks, all students were asked to fill the
post CSE questionnaire at home in the 9th week. In the 10th and 11th weeks, 12 pupils from
both the groups were randomly selected to be interviewed. These data were treated rather
as anecdotal data than qualitative data. Finally, the entire treatment was completed.

3. Results
T-test did not report any difference on the overall creative self-efficacy (CSE),
CSE-strategy, CSE-performance, and CSE-Nfeedback between the 2 classes pre-CSE
questionnaire, suggesting an equality of homogeneity on the 2 groups pre-CSEs.
In regards to the post-CSEs, T-test reported some significances between the 2 classes
post-CSE: overall CSE, t(52) = 2.163, p = .035; CSE-strategy, t(52) = 2.230, p = .030.
In regards to creative performance, T-test reported some significance on final scores
of students creative products between the two groups: overall score, t(52) = 2.591, p
= .012; transitions & edits, t(52) = 3.769, p < .001; planning & storyboarding, t(52) =

3.495, p = .001; accuracy of information, t(52) = 3.230, p = .002; drawing, t(52) = 2.620, p
= .012. Selected significant items are listed in Table 1 with descriptive statistics.
Table 1. CSE and Final scores
Group

PA

NonPA

28

26

Creative
Self-Efficacy
Performance
Strategy*
Nfeedback
Overall*
Performance
Strategy*
Nfeedback
Overall*

Mean

S.D.

Selected Final Scores

Mean

S.D.

3.36
3.76
3.87
3.64

.70
.61
1.02
.49

3.10
3.43
3.65
3.37

.68
.44
.66
.43

Originality & Creativity


Transitions & Edits*
Planning & Storyboarding*
Accuracy of information*
Overall*
Originality & Creativity
Transitions & Edits*
Planning & Storyboarding*
Accuracy of information*
Overall*

2.87
3.60
3.20
3.35
2.98
2.36
2.77
2.39
2.42
2.39

1.09
.89
.94
1.12
.90
.93
.71
.74
.98
.74

Note. * indicates significant difference between the 2 groups

4. Discussion
Hypothesis 1. Peer assessment has a positive effect on performance of digital storytelling
utilizing Draw&Tell Tablet computer application among elementary schoolers. PA has a
positive effect on participants overall score. Hypothesis 1 was retained as the result
indicated that PA had a positive on participants final score. Such finding is consistent with
the existing literature that PA promotes peer learning [12]. Furthermore, the finding
supported our assumption that a simplified PA could eliminate a potential negative effect
on creativity outcomes and promote creative outcomes. Bandura [1] suggested that instead
of applying traditional types of PA (e.g., grading, commenting, feedbacks, etc.), a PA
process to be more informing and less evaluating-orientated might produce a positive
effect on creative outcome.
Hypothesis 2. Peer assessment has a positive effect on creative self-efficacy (CSE) of
digital storytelling utilizing Draw&Tell Tablet computer application among elementary
schoolers. PA has a positive effect on participants levels of CSE. Hypothesis 2 was
retained as the result indicated that PA had a positive effect on participants levels of CSE.
Literature suggested a raise on the level of CSE might indicate a higher quality of creative
outcome [8], as well as the potential on the academic success [2]. The finding echoes that
a personal belief on creativity could be enhanced by training (i.e., PA could be seen as a
reinforcement of informing personal belief on creativity) [8].
In order for a better understanding of participants thoughts behind the data, we
randomly interviewed 12 students from both the groups as anecdotal data. Unlike in the
experimental group where students had the opportunity to learn from their peers, students
in the controlled group mostly revealed that I dont feel my work is special, or, I feel
others might have done better than me. When we took one more step by asking Better
on? Or worse on what? Students could not specify what they mean about the differences.
This echoed our finding that a peer assessment process was helpful for students to be
appreciative of ones own works, and thus then increase the level of personal belief on
creative performance (i.e., CSE). We heard similar reflections when asking what they
would have felt if negative feedbacks were given. Most students from both groups
revealed that they would not give up their ideas. This might explain a PA did not constrain
self-efficacy on creativity, and why students final score correlated positively with
CSE-Nfeedback. It was conjectured that students who possessed a higher level of creative
self-efficacy on dealing with significant feedbacks, there was a potential that they could
perform a higher quality of the task.

5. Conclusion
The current study investigated a peer assessment effect on students creative work (i.e.,
storytelling) and creative self-efficacy in a creativity learning context. The findings
suggest implementations in elementary education that peer assessment may promote
students creativity performance and creative self-efficacy. Additionally, the current study
re-tested the assumption that creative self-efficacy could be enhanced by reinforcing the
beliefs about creative performance. The reinforcement was carried with a peer assessment
process in the presented study. As a result, students performed better quality of their
creativity works as well as reflected a higher level of creative self-efficacy. Such findings
may inform an implementation for educational settings.

Acknowledgement
The presented study is supported by the National Science Council in Taiwan, NSC
100-2631-S-008-004.

References
[1] Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.
[2] Beghetto, R. A. (2006). Creative self-efficacy: Correlates in middle and secondary students. Creativity
Research Journal, 18(4), 447-457. doi: 10.1207/s15326934crj1804_4
[3] Bruner, J. (1990). Acts of meaning. Harvard University Press: Cambridge.
[4] Hung, S.-P. (2003). A social cognitive model of significant other's feedback, creative self-efficacy,
internal motivation and external motivation on creative behavior: A structure equation modeling study.
Master Thesis, National Chiao-Tung University, Taiwan.
[5] Jonassen, D. H., & Hernandez-Serrano, J. (2002). Case-based reasoning and instructional design: Using
stories to support problem solving. Educational Technology Research and Development, 50(2), 65-77.
doi: 10.1007/bf02504994
[6] Liu, C.-C., Chen, H. S. L., Shih, J.-L., Huang, G.-T., & Liu, B.-J. (2011). An enhanced concept map
approach to improving childrens storytelling ability. Computers & Education, 56(3), 873-884. doi:
10.1016/j.compedu.2010.10.029
[7] Lu, J., & Law, N. (2012). Online peer assessment: effects of cognitive and affective feedback.
Instructional Science, 40(2), 257-275. doi: 10.1007/s11251-011-9177-2
[8] Mathisen, G. E., & Bronnick, K. S. (2009). Creative self-efficacy: An intervention study. International
Journal of Educational Research, 48(1), 21-29. doi: 10.1016/j.ijer.2009.02.009
[9] Polkinghorne, D. (1988). Narrative knowing and the human sciences. Albany, NY: SUNY Press.
[10] Robin, B. R. (2008). Digital storytelling: A powerful technology tool for the 21st century classroom.
Theory into Practice, 47(3), 220-+. doi: 10.1080/00405840802153916
[11] Strijbos, J.-W., Narciss, S., & Dnnebier, K. (2010). Peer feedback content and sender's competence
level in academic writing revision tasks: Are they critical for feedback perceptions and efficiency?
Learning & Instruction, 20(4), 291-303. doi: 10.1016/j.learninstruc.2009.08.008
[12] Topping, K. (1998). Peer assessment between students in colleges and universities. Review of
Educational Research, 68(3), 249.
[13] Topping, K. (2010). Methodological quandaries in studying process and outcomes in peer assessment,
Editorial,
Learning
&
Instruction,
pp.
339-343.
Retrieved
from
http://0-search.ebscohost.com.source.unco.edu/login.aspx?direct=true&db=ehh&AN=48891870&site=eh
ost-live
[14] Tseng, S.-C., & Tsai, C.-C. (2007). On-line peer assessment and the role of the peer feedback: A study
of high school computer course. Computers & Education, 49(4), 1161-1174. doi:
10.1016/j.compedu.2006.01.007
[15] Via, S. (2002). http://its.ksbe.edu/dst/PDFs/Rubrics/rubric.pdf.
[16] Yu, F. Y., & Wu, C. P. (2011). Different identity revelation modes in an online peer-assessment learning
environment: Effects on perceptions toward assessors, classroom climate and learning activities.
Computers & Education, 57(3), 2167-2177. doi: 10.1016/j.compedu.2011.05.012

An Online Platform for Problem-Based


Learning with Operational Concepts Map
Chung-Cheng TSENGa*, Po-Yao CHAOb, Chih-Yuen CHOUa & K. Robert LAIa
a
Department of Computer Science & Engineering, Yuan Ze University, Taiwan.
b
Department of Information Communication, Yuan Ze University, Taiwan.
*cct0403@saturn.yzu.edu.tw, poyaochao@saturn.yzu.edu.tw, cychou@saturn.yzu.edu.tw,
and krlai@cs.yzu.edu.tw

Abstract: Problem-based learning (PBL) encourages learners to acquire new knowledge


through cognitive model development and hypotheses-deduction process. Graphical
modeling tools, such as concept map, can help learners to exteriorize the process of
knowledge construction and visualize learner's cognitive model to support meaningful
learning. But few of those tools integrate hypothesis-reasoning mechanism into the process
to support the discovery of knowledge deficiencies and foster in-depth reflection. To that
end, this paper proposes a novel graphical tool, operational concept map (OCM), for
supporting the PBL. In OCM, it combines the merits of conceptual modeling tool and
computer-based simulation. The OCM not only inherits all basic functions of concept map,
also facilitates the learning-by-doing approach for problem solving. Application of the
proposed idea is illustrated in learning a well-known sort algorithm, bubble sort, in the sort
problem.
Keywords: Problem-based learning, Concepts map, Operational concepts map, Simulation,
Learning by doing.

Introduction
Problem-based learning (PBL) has been seen as an active learning approach to promote
meaningful learning [1, 2]. This active approach encourages learners to acquire new
knowledge through an iterative hypotheses-deduction process when solving a problem.
During the process, a learner analyzes the problem by identifying key fact with the problem
scenario, proposing hypotheses and solutions, and evaluating the hypothesis by monitoring
outcomes of the solutions. Among this process, research shows that the development of
cognitive model and the evaluation through experiments can help a learner to identify the
key concepts and evaluate the hypothetical relationship among these concepts [3, 4].
Cognitive model [5, 6] is a representation of key elements with relationships among
these elements in the specific phenomena or knowledge. When solving a problem, cognitive
model can provide a learner with reference framework to facilitate the thinking of a
problem[7]. Some graphical modeling tools, such as concepts map[6], have demonstrated
their usefulness in construction of learners cognitive models. With a concept-mapping tool,
a cognitive model can be represented as a well-organized graphical map composed of verbal
and symbolic elements. This useful tool provides scaffolds to assist a learner to develop and
represent the structure of knowledge as personal cognitive model regarding a problem or
topic [8].
While, in problem solving, it often requires a learner to reflect and then refine his/her
own cognitive model by iteratively evaluating the models effectiveness, the mechanism for
evaluating hypothetical relationship among key concepts to facilitate learning by

experimenting is seldom integrated with a concept-mapping tool. To this end,


computer-based simulation[9] seems to provide an environment in which the learners can
evaluate their hypotheses by experimenting. The simulation functions assist learners to
develop critical skills including identify key concepts of a problem, generate hypotheses,
and propose solutions for the problem[10]. These simulation environments not only
facilitate the development of cognitive model by evaluating hypotheses, but also provide
learning goals for learning by doing activities. Based on the necessity of evaluation
mechanism of graphical modeling tool and the need of learning by experimenting in PBL,
this paper proposes a novel graphical tool, namely operational concept map (OCM), which
incorporates the basic functions of concept map with additional hypothesis evaluation
mechanism. The intent is to foster effective problem solving through developing cognitive
models and evaluating hypothesis in a computer-based simulation environment.
The remainder of this paper is organized as follows. Section 1 introduces a conceptual
framework of OCM. In this framework, learners can construct a knowledge model with
hypotheses-reasoning mechanism. Then, Section 2 describes how the OCM can enhance the
process of PBL. In this section, we present an example to illustrate how the OCM can be
applied for learning the concept of bubble sort in the sort game followed by a conclusion in
Section 3.

1.

Operational Concept Map

1.1 Overview
Operational concept map (OCM) is a graphical modeling tool for organizing and
representing knowledge with deductive-reasoning mechanism. An OCM is composed of
concepts, hypothesis links among these concepts, and propositions. Firstly, concepts, which
are enclosed in circles as shown in Figure 1, are perceived regularity in events or objects, or
records of events or objects, designate by label. Secondly, the hypothesis link, indicated by
rectangle with round corners and several connecting lines linking concepts, is a possible
causal relationship among these concepts. Each link contains a hypothetical statement that
describes nominal, logical or mathematic formulations. Finally, a proposition contains one
hypothesis link connecting two or more concepts to form a meaningful statement. To
implement hypotheses-reasoning mechanism, each operational concept and corresponding
hypothesis links in OCM are bound with specific set of problem instances. These problem
instances are collected to provide real world problem data that helps learners to evaluate the
effectiveness of an OCM. Then, learners apply different sets of problem instances to ensure
the satisfaction of hypothetical statements in hypothesis links.

Figure 1. A simple operational concept map


1.2 Key Features

1.2.1 Development of cognitive model and hypotheses evaluation


Like concept map, the OCM assists learners to incorporate new information into the
relevant concept framework of which they have already built when a learner generates
his/her cognitive model for a problem. The OCM helps the learner to identify general
concepts held by the learner prior to instruction and on more specific concepts, and anchor
the new knowledge into the conceptual framework. Moreover, the relationship among these
key concepts in OCM represents not only the semantic relationship, but also the rule of
constraints. The rule can be represented as a mathematic formulation, such as A=B+C, or
logical representation, such as IF A > B, then Swap(A,B). For hypothesis evaluation,
hypothesis links represented as a constraint serve not only as semantic description of model
to be simulated, but also as rules telling that certain conditions must be satisfied [11-13]. A
learner maintains the satisfaction of their hypothesis constraints by iteratively revising their
problem cognitive models. Moreover, in OCM, the just-in-time feedbacks of hypotheses
evaluation are offered immediately as simulation outcomes. A learner refines his/her
cognitive model according to the just-in-time feedbacks.
1.2.2 Construction process of OCM
The construction process of OCM is one kind of externalizing internal and hidden mental
model. Some kinds of modeling process [14, 15] facilitate the learner to construct cognitive
model with verbalizing and writing. But, in this process, OCM is an external memory tool to
facilitate the cognitive process by visualizing. Because the thinking process of
problem-solving is represented as organization of concepts and hypothesis links in OCM,
the simulation then can not only facilitate the exploration of what-if situations and also
support the hypothesis-reasoning process for identifying the knowledge deficiencies.
Finally, the visualization with graphical cognitive model facilitates this constructive process
with guidance of what to do and how to do.

2.

Application of OCM to learn a concept of Bubble sort in the sort problem

To demonstrate the application of OCM in supporting PBL, a well-known learning issue in


an introductory college computer science course, the sort problem, is adapted for this study.
A learner will learn various sort algorithms to solve the sort problem. In this example of sort
problem, the bubble sort algorithm is chosen because it is a simple algorithm that compares
each pair of adjacent items and swaps them through the list to be sorted until no swaps are
needed. Figure 2 displays the role of instructor and learner played in the process of applying
the OCM in the sort problem.

Figure 2. The process of applying the OCM in the sort game.


2.1 Analyzing problem scenario by Instructor
In order to apply OCM to sort problem, the instructor has to analyze a problem scenario at
first. In the example of sort problem scenario, four values are randomized and placed
individually in the four cells in sequence. A learner has to sort this set of values in ascending

order. The instructor has to analyze this problem to identify the key concepts and possible
relationships among these key concepts. In this case, the key concepts are the position of
each cell. Therefore, the instructor uses four cells of array B to represent the key concept.
So, in the figure 3, these key concepts (the four cells of array, B(1) to B(4)) are included in
the Concepts Repository for learners to construct their own OCMs, and the Simulation
Panel will represent the sort process and result of this array for evaluation. The instructor
then identifies the relationships between two adjacent cells and constructs possible
reference models to evaluate learners cognitive model.
2.2 Learner develops his/her cognitive model
In this phase, the learner has to construct his/her personal cognitive model for the sort
problem. But, at first, the learner has to read the description of bubble sort algorithm which
is prepared by the instructor. Then, the learner has to realize the learning goal and key
concepts of a sort problem. The learning goal is defined by the instructor to tell learners that
some conditions they have to satisfy. The goal of sort problem is that the unsorted list has to
be sorted in ascending order. The key concepts have been identified in the concepts
repository.
After realizing the problem scenario, the learner starts to develop his/her cognitive
model by OCM. In this example of sort problem, the learner will describe each step of the
bubble sort algorithm in sequence. In this case, the learner places two adjacent elements,
B(1) and B(2) in the OCM Construction Panel, to determine hypothesis link among them.
The possible relationship between B(1) and B(2) is that if the value in B(1) is larger than the
value in B(2), these two values have to be swap. Therefore, the learner places a hypothesis
link, named swap, and establish connection lines between two concepts and hypothesis
link to form a proposition. Next, the rule of swap is established to represent a logical
representation: if B(1) > B(2), then swap(B(1), B(2)). Then, this hypothesis can be
evaluated by simulation. In the simulation, the learner observer the result generated from the
problem scenario for this hypothesis. If this hypothesis link is work, the learner can observer
that value in B(1) will be swap for value in B(2) (if B(1) is larger than B(2)). Then, the
learner will apply this hypothesis link to the next two adjacent concepts repeatedly. The
cognitive model of bubble sort algorithm is represented as Figure 3.

Figure 3. OCM Construction Environment


2.3 Learner acquires and applies new knowledge
When personal cognitive model is completed, another scenario of sort problem is applied to
evaluate it. If this model cannot reach the learning goal of new scenario, it has to be revised.

For example, an unsorted list consist of 100 elements cannot be solved by the model for four
elements of list. The learner may develop various models for different size of lists, or deduct
a general model for different scenario by using the iteration construct. This is an adaptive
process that personal cognitive model will be adapted by various problem scenario.

3.

Conclusions

This paper has proposed an operational concept-mapping approach for supporting PBL. In
OCM, it combines the merits of conceptual modeling tool and computer-based simulation.
During the iterative construction process, OCM can foster in-depth learning by
experimenting with the proposed game simulation environment. Learners can have better
understanding of what they do and why they do in the problem-solving. While the
preliminary results look interesting, more issues remain to be further investigated, including
application of OCM for supporting various learning through problem solving, and
development of better online environment for supporting PBL with OCM.

References
[1]
[2]
[3]
[4]
[5]

[6]

[7]
[8]

[9]
[10]
[11]
[12]
[13]

[14]

[15]

Hmelo-Silver, C. E. (2004). Problem-Based Learning: What and How Do Students Learn? Educational
Psychology Review, 16(3), 235-266.
Liu, C.-C., Cheng, Y.-B., & Huang, C.-W. (2011).The effect of simulation games on the learning of
computational problem solving. Computers & Education, 57(3), 1907-1918.
Macklin, A. S. (2001). Integrating information literacy using problem-based learning. Reference
Services Review, 29(4), 306-313.
Torp, L., & Sage, S. (2002). Problems as Possibilities: Problem-Based Learning for K-16 Education (2
ed.): Association for Supervision & Curriculum Development.
Novak, J. D. (2002). Meaningful learning: The essential factor for conceptual change in limited or
inappropriate propositional hierarchies leading to empowerment of learners. Science Education, 86(4),
548-571.
Novak, J. D., & Caas, A. J. (2008). The Theory Underlying Concept Maps and How to Construct and
Use Them, Technical Report IHMC CmapTools. Florida Institute for Human and Machine Cognition:
Florida Institute for Human and Machine Cognition.
Ormrod, J. E. (2010). Educational Psychology: Developing Learners (7 ed.): Prentice Hall.
Rendas, A. B., Fonseca, M., & Pinto, P. R. (2006). Toward meaningful learning in undergraduate
medical education using concept maps in a PBL pathophysiology course. Advances in Physiology
Education, 30(1), 23-29.
Schank, R. C. (1995). What We Learn When We Learn by Doing, from
http://cogprints.org/637/1/LearnbyDoing_Schank.htm
Savery, J. R. (2006). Overview of Problem-based Learning: Definitions and Distinctions. The
Interdisciplinary Journal of Problem-based Learning, 1(1), 9-20.
Lai, K. R., & Lan, C. H. (2006). Modeling Peer Assessment as Agent Negotiation in a Computer
Supported Collaborative Learning Environment. Educational Technology & Society, 9(3), 16-26.
Lan, C. H., Graf, S., Lai, K. R., and Kinshuk (2011). Enrichment of Peer Assessment with Agent
Negotiation, IEEE Trans. on Learning Technologies, Vol. 4, No. 1, pp. 35-46.
Mitrovic, A. (2010). Modeling Domains and Students with Constraint-Based Modeling. In R. Nkambou,
J. Bourdeau & R. Mizoguchi (Eds.), Advances in Intelligent Tutoring Systems (Vol. 308, pp. 63-80):
Springer Berlin / Heidelberg.
Collins, A., Brown, J. S., & NewMan, S. E. (1989). Cognitive Apprenticeship: Teaching the Crafts of
Reading, Writing, and Mathematics. In R. Lauren B (Ed.), Knowing, Learning, and Instruction: Essays
in Honor of Robert Glaser (pp. 453-494). Hillsdale, New Jersey: Lawrence Erlbaum Associates,
Publishers.
Pedersen, S., & Min, L. (2002). The Transfer of Problem-Solving Skills from a Problem-Based Learning
Environment: The Effect of modeling an Expert's Cognitive Processes. Journal of Research on
Technology in Education, 35(2), 303.

Small Group Learning with Digital Pens in


High School Physics
Ming-Hong CHIUa, Chi-Jen LINb*
National Lo-Tung Senior High School, Taiwan
b
Department of Learning and Digital Technology, Fo Guang University, Taiwan
*cjlin@mail.fgu.edu.tw
a

Abstract: Physics learning was known to be hard for high school students and it was known
that there was a gender difference in this perspective. One important method to promote
student learning achievement in physics is to increase the opportunities to receive feedbacks
during problem-solving practices, and small group learning is one of such a good method.
Another method is to provide multiple viewpoints to assist student learning, and digital pens
is a potential tool to achieve this goal. The experimental results showed that female students
are more benefited from small group learning and male students may suffer in learning
retention when digital pens were used in small group learning setting.
Keywords: small group learning, digital pen, physics concept

Introduction
Physics has been perceived as a hard subject by most high school students in Taiwan. One of
the reasons that make physics hard to be mastered was that most students did not know how
to solve problems with physics concepts properly, thus they tended to acquire superficial
problem solving techniques [1]. Without interacting with physics teachers, students were
unaware whether their learning was superficial or not. The authors believe that increasing
student opportunities to receive feedback during their problem-solving practices at their
early learning stage is crucial to improve their learning. However, due to limited time in the
classroom, a physics teacher can spend time only with partial of the students in a class, and
the rest of the students in the class are, unfortunately, on their own. The solution to this
situation is the well-known two sigma problem [2], which is a search of group instruction
methods that are as effective as one-to-one tutoring.
In a physics class, besides the teacher, more capable students are the potential helpers to
enhance student learning. Peer instruction [3] is an instruction method that matches students
pairwise in order that the interaction between the two students helps their learning.
However, it was found that if the two students in a pair were both less capable students, the
learning outcomes might be worse than they studied alone. With careful assignment of the
group leaders, the discussion in the small groups may result in learning enhancement. Thus,
it is interesting to investigate whether such a small group learning method will enhance
physics learning. Another method to improve student learning is provide students with
multiple viewpoints. We use digital pens to achieve this goal. The effects of introducing
digital pens will also be investigated to see if there exists any further enhancement. As
gender was also known to be significant factor in physics learning [4] [5] [6], we will also
investigate how the gender factor influence student learning under relevant situation.

1.

Characteristics of Digital Pens

The usage of digital pens is quite the same as those of ordinary pens, even their appearance
is the same (as shown in Figure 1). The main differences between a digital pen and an
ordinary pen are that a digital pen comes with an internal camera to capture what was
written using the digital pen and a wireless transmitter to transmit what the camera captured
to a whiteboard or a computer. The configuration of transmission target is up to the teacher.
Thus, a teacher can monitor answers of all the student groups and react accordingly at one
location when all the groups use digital pens (teacher time saving feature). The monitoring
screen can also be displayed to all students to demonstrate various solutions of a problem.
The benefit of introducing digital pens is that it does not require additional training for
students to use them. However, to ensure the proper capturing function of the cameras,
specially prepared paper, which comes with invisible printed dots, must be used for writing.
In this study, we use digital pens as a tool for demonstrating alternative solutions in order to
investigate whether students learning will be further improved in the small group learning
format.

Figure 1. Digital pen(bottom) and ordinary pen(above).

2.

Two-staged Experiment

Due to the nature of this study, a two-staged experiment was planned. The goal of the first
stage experiment was to investigate whether the small group learning format outperformed
the individual learning format, and the goal of the second stage experiment was to
investigate whether the introduction of digital pens did further improve physics learning in
the small group format.
2.1 Experimental settings
The learning topic of the first stage experiment is the unit of gravity on earth, while the
learning topic of the second stage experiment is the unit of satellite movement. Each topic
was taught for two contiguous weeks with a total of eight classes. The class delivery format
was consisted of three parts: concept introduction, problem-solving demonstration, and
student problem-solving practices. The learning format of the control group and
experimental group differed only at the third part, that is, the student problem-solving
practice part. In the individual learning setting, or the control group of the first stage
experiment, each student solved problems by themselves, while the teacher walked through
the classroom to assist those who need help. In the small group learning setting, which
includes the experimental group of the first stage and the control group of the second stage,
a class of students was divided into eight groups. Each group was required to work out a
solution and their solutions were reported to the whole class after all the groups completing

their solutions. The group leader was chosen by the teacher according to their previous
physics learning performance. Eight top ranked students were divided into the eight groups
and were assigned as the group leader. The group leader was in charge of coordinating the
group solution among the group members and reporting the group solution to the whole
class. The teacher played the role of commentator during their reports. In the digital pen
learning settings, students also learn in the group format, except that group leaders use
digital pens as their written tools and group solutions were shown on a common display that
was visible to the whole class.
2.2 The participants
The subjects come from two classes of high school students of eleventh grade. One of the
classes served as the control group and the other served as the experimental group. There are
44 students in the control group with 13 female students and 31 male students, and 37
students in the experimental group with 23 female students and 14 male students. Before the
two-staged experiment, a formal learning performance assessment was performed for the
two classes. The results showed that there were no significant statistical differences between
the previous physics learning performance of the two classes, with the average of learning
performance of the control group slightly better than that of the experimental group. The
two classes were taught by the same teacher.
In each stage of the experiment, a pretest and a posttest of the learning unit were
conducted. In the second stage, a delayed test was also performed to investigate the retain
effects of the intervention of digital pens. The scores of these tests were the number of
problems the students solved correctly. There were totally ten problems in each test.

3.

Experimental Results

3.1 Small group may suffer from classroom order degeneration


Before the formal two-staged experiment, a prior experiment, which was arranged in the
same setting as the first stage experiment, was conducted for the same subjects. To our
surprise, the results showed that the learning performance of students in the individual
learning setting was significantly better than that of in the small group learning setting.
After reviewing the classroom process, it was found that the subjects chatted much more
often in the small group learning setting than their previous classes. That is, classroom order
degenerated during the experiment. For the rest of the classes the student attended, their
format was individual learning. The discussion nature of the small group was novel to the
students and they misused the opportunities of learning discussion by chatting. However, it
was also noticed that one of the groups that was close to a video recorder chatted much less
than the other groups. Consequently, in the experiment, we set up a video recorder for each
group in order to prevent unnecessary chatting. This strategy worked well in the experiment.
3.2 Small group learning outperforms individual learning
Table 1 denotes the student performance in the first stage experiment. The numbers in the
table denote how many problems the student correctly solved in the tests. The result of
t-statistics of the pretests indicates that there is no significant difference between the student
performance in the control group and the experimental group. On the other hand, the
students in the experimental group outperform the students in the control group. That is,
small group learning is better than individual learning. This result is a support that more

capable students in the small group do help enhance the learning of the other group
members.
Table 1. Student performance in the first stage experiment (n=81)
Control group
Experimental group

4.

pretest
posttest
pretest
posttest

N
44
44
37
37

Mean
3.30
6.23
3.08
7.16

SD
1.81
1.66
1.48
1.48

Learning gain
2.97
4.08

Conclusions

In this study, we investigated the effectiveness of small group physics learning by assigning
more capable students as group leaders, whether the introduction of digital pens in the small
group learning can further improve student learning, and whether gender difference exists in
these investigations. The results showed that such a small group learning method did
improve student learning and the learning gains of female students were greater than those
of male students. However, introducing digital pens in the small group learning format,
which was a means to provide students with multiple viewpoints, did not produce additional
learning gains.

References
[1] Crouch, C.H. & Mazure, E. (2001). Peer Instruction: Ten years of experience and results. American
Journal of Physics, 69(9), 970-977.
[2] Bloom, B. S. (1984).The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as
One-to-One Tutoring. Educational Researcher, 13, 4-16.
[3] Mazure, E. (1997) Peer Instruction: a Users Manual. Prentice Hall, New Jersey.
[4] Cavallo, A. M. L., Potter, W. H., & Rozman, M. (2004) Gender Differences in Learning Constructs,
Shifts in Learning Constructs, and Their Relationship to Course Achievement in a Structured Inquiry,
Yearlong College Physics Course for Life Science Majors. School Science and Mathematics, 104(6),
288-300.
[5] Hazari, Z., Tai, R. H., & Sadler P. M. (2007) Gender differences in introductory university physics
performance: The influence of high school physics preparation and affective factors. Science Education,
91(6), 847-876.
[6] Kost, L. E., Pollock, S. J., & Finkelstein N. D (2009) Characterizing the gender gap in introductory
physics. Physical Review Special Topics - Physics Education Research, 5(1), 010101.

Aiding Digital Natives Learn Positive


Learning Behaviors through Reflection
Roberto LEGASPI*, Paul Salvador INVENTADO, Rafael CABREDO
& Masayuki NUMAO
The Institute of Scientific and Industrial Research, Osaka University, Japan
*roberto@ai.sanken.osaka-u.ac.jp

Abstract: Commonly attributed to digital natives is the ability to quickly, yet effectively,
shift from one task to another. However, several works have debunked this assumption by
showing that multitasking even among digital natives led to poor learning performance and
productivity. Our aim is to provide a tool to help digital natives be self-aware of desirable,
while curbing undesirable, learning behaviors. Our tool is infused with self-annotation and
feedback mechanisms that allow students to reflect upon their entire learning history. Our
results indicate that the annotation process with the tool helped students understand their
learning behaviors better and identify ways in which their behaviors can be improved.
Keywords: Digital natives, learning behaviors, productivity annotation tool, user modeling

Introduction
For a time, the notion that digital natives [17], also called Gen M (for multimedia) [19] or
Net Gen [17,19,22], are capable of switching tasks [8,15,18] prevailed until other
researchers started to level this assumption. Digital natives are technology-savvy and carry
out many of their activities on the web [18]. They engage with and communicate socially
through various multimedia [8,15]. Since the tools they use to study, get entertained and
socialize are equally accessible at any time, digital natives tend to switch from learning
(e.g., viewing tutorial videos) and non-learning (e.g., engage in game or social network
websites) tasks. However, there is evidence to suggest that task-switching behavior can lead
to poorer performance and learning outcomes [11] and result to time lost that increases with
the complexity of, and unfamiliarity to, the task [20]. It is plausible that productivity
deteriorates when alternating tasks.
Our objective is to help the digital native learner become aware of problems resulting
from learning and non-learning task-switching and help him/her self-regulate towards
increased learning productivity. This paper reports the results we obtained from subjects
who experimented with our tool designed to raise self-awareness of effective learning
behaviors - to discover routines that lead to productive learning and become cognizant of
personal factors that support positive study by constantly aiding reflections on activity
choices made that helped achieve (or otherwise) the desired learning outcomes.

1.

Productivity Annotation Tool

Being aware of ones own learning behavior is metacognitive and students who are able to
step back, reflect in retrospect, and adjust their thinking are able to adapt and monitor their
behaviors to know what they need to study and how to do it [10]. We have designed a tool

basically to this end. We infused in its design a self-annotation scheme that will allow
students to review and reflect on their past learning sessions.
When using our tool, students identify their goals for the current learning session and
then start the tools data collection process to run in the background. Once the session is
over, a timeline representing their entire learning session is shown to the students. When the
mouse hovers over this timeline, screenshots of the desktop and webcam are shown to help
students recall what happened at any point in time. Apart from using their mouse for
annotation, an image player can also be used to select time partitions, much like how a video
player is used, together with a selection button to indicate when to begin and end the
selection. After selecting a time span, students can then provide their annotation.
Students annotate three aspects of their learning session, namely, intention, activity,
and affective state. The intention is either goal-related, i.e., the activities relate to reaching
any of the defined goals, or nongoal-related. Activities can be labeled as primary or
secondary. A primary activity refers to what the student was focused on during the selected
time span (e.g., using chrome.exe, reading a technical paper, or viewing a tutorial video).
Secondary activities are those done together with the primary (e.g., listening to music,
drinking coffee and eating, aside from others). The students also input the affective states
they experienced as they performed the activities, i.e., delighted, engaged, confused,
frustrated, surprised, afraid and neutral for goal-related activities [5], and angry, disgusted,
sad, delighted, afraid and neutral [6] for nongoal-related ones.
After annotating the entire session, students assess their learning session in terms of the
percentage of goal completion, activities that helped achieve their goals, and their perceived
productivity level (in a scale of 1-5). The tool then provides statistics regarding the students
learning sessions in terms of the amount of time they spent in goal- and nongoal-related
activities, together with the emotions they felt during each type of activity. The tool then
produces a log file that contains timestamps, intentions, activities and affective states for a
learning session. The productivity and goal completion ratings will be stored in a separate
log file. All these will be used by the software for analysis of student learning patterns.
We designed our tool while taking cues from prior works. Goals enhance
self-regulation, self-motivation, self-efficacy, and self-evaluation of progress [4,21]. When
goals are defined, there is the tendency for students to do less the other tasks since they are
aware of what they are supposed to do. Secondly, when students annotate their activities,
they are actually performing self-monitoring. Seeing in retrospect how they spent their time
learning, they get an idea of what comprises their study habits [10]. They can identify what
behaviors they think are helpful to them as well as those that cause distractions. It will also
help them see how their affective states actually influenced their learning process [5,16].
For example, emotions such as boredom and frustration can cause students to engage in
non-learning tasks or stop learning altogether [12]. The tool can therefore help students
identify and retain good study habits and mitigate unwanted learning behaviors.

2.

Related Works

The importance of students setting their own goals, selecting the appropriate strategies,
applying these strategies and monitoring their effects in order to modify them when
necessary has spawned research in providing support tools for such metacognitive tasks
while learning with computer-based learning environments, such as intelligent tutoring
systems and hypermedia-based environments (e.g., [2,3,14]). Furthermore, some learning
environments keep track of student actions to help maintain the student model (e.g., [1,13]).
However, we see the need to analyze student behavior outside a learning environment
and the importance of providing feedback for these instances to support learning. In our

previous work, we reported that when students study on their own, they do not only engage
in goal-related tasks but also in nongoal-related activities. We reported that the digital
natives who participated in our experiments in a span of two weeks spent 47.8% of their
time learning and the rest not learning [9]. Furthermore, although most systems track
student activities by looking at what they do on the computer, many of their activities are
also done without the use of a computer [7].
Our motivation is to provide support to student metacognitive tasks and actions when
learning using the computer and without it. Our students may not be constraint to a learning
environment or particular domain knowledge. They may use our tool given their chosen
topic of study and the tools they will use to study.

3.

Experimentation

We are interested in observing learning behaviors in an environment with no guidance from


teachers and where students have complete control over their learning tasks, hence, can pose
as potential source of distractions. We identified foreign scholars performing their graduate
research projects in Japan as good source of data since they are expected to retrieve, process,
and analyze information on their own while managing their own time and effort. Since they
have full control of their activities, playing games, watching videos, sending SMS and
chatting can always pose distractions.
Four subjects participated in our experiments, specifically: two females and two males,
who are Filipino, Indonesian, Thai and Vietnamese, respectively, with an average age of
25-years. Everyone fits as digital natives. We aimed for the sessions to be as natural as
possible with the subjects deciding when to use the tool, for how many times and for how
long. They were taught about the definition of goal and how to provide it to the system. For
the annotation, they were taught about the definitions and specifics of intentions, activities
and affect. Specific to affect, they were taught to indicate the strongest emotion in cases
where they seem to have felt more than one emotion.
At the end of every session, the subjects were asked to fill up a questionnaire to help
assess the influence of the tool to their reflections. We wanted to know if they discovered
anything interesting about their learning behavior, whether positive or negative, during the
annotation process, and based on the statistics provided by the system, whether they see the
need to improve their learning behavior.

4.

Results and Analyses

4.1 User Feedback on the Tools Assistance


The subjects agreed that the tool helped point out what is both positive and negative in their
learning behaviors. One subject, for example, felt good when after annotating, he realized
that he actually got absorbed into work and got distracted less than he thought. He also
realized that even though it was very easy for him to fall asleep when confused or frustrated,
most of his time was actually spent on goal-related activities. In his last session, after
previously having reflected on his past behaviors wherein he spent on the average a third of
his time on nongoal-related activities, he still allowed himself to enter into nongoal-related
states while making sure he spent still most of the time in learning. Another subject realized
that it was a disadvantage for her to switch between tasks - that when she attempted reading
three papers simultaneously, none of the goals she set for the session was completed and she
gave herself a low productivity score. She also realized that when she got confused, her

tendency was to switch to non-goal related tasks rather than stay engaged in learning. Yet
another subject realized that he spent more time watching game video streaming and felt bad
after. And one reported that standing up every now and then or drinking coffee would help
stay focused. Lastly, one subject noticed that the urgency of his goals (i.e., submission
deadline) coerced him to spend more time learning than usual, and that the lack of
urgency or importance of the goal and the lack of desire to accomplish it resulted in less
productive learning.
All of the above show positive effects of the tool because the students were able to
reflect and monitor themselves through the annotation process. They became more aware of
the activities they performed and actively tried to avoid moving away from being
goal-engaged thereby learning to improve their learning behavior.
4.2 Self-assessed Productivity
As previously mentioned in the first section, students were asked to assess their learning
sessions in terms of how productive they felt they were. It is interesting to note that even
though the subjects performed nongoal-related activities only 24% of the time at the
average, their average self-assessed productivity is only 71.3%, with the amount of time in
goal-related activities not necessarily directly proportional to productivity. Table 1
illustrates this point using snapshots of all the sessions for all subjects, where %TG, %TNG
and %Pr refer to percentage of time spent in goal- and nongoal-related tasks and the
percentage of self-productivity, respectively. The norm is that if a subject stayed most of the
time in goal-related states, productivity assessment is supposed to be high. However, for the
second sessions (S2) of subjects B and D, this did not hold. According to B, she felt that the
time she spent in nongoal-related activities in S2 is higher than S1, and on that basis alone,
she gave herself a lower productivity score. For D, he could have given a higher
productivity score (resulting to %Pr > 80) but he felt that most of his time learning was spent
thinking about the problem and not finding the solution. The implication here is that
students rate themselves based on the standards they set for themselves and the tasks they
set for the sessions. Our aim is that in their process of self-reflection, the students should be
able to see the effects of what they impose on themselves in relation to their goals.

%TG
%TNG
%Pr

5.

Table 1. Self-assessed productivity of each subject per session


Subject A
Subject B
Subject C
Subject D
S1 S2 S3 S1 S2 S3 S4 S5 S1 S2 S3 S4 S5 S1 S2 S3
89 46 84 83 76 90 70 89 93 51 64 71 80 90 83 64
11 54 16 17 24 10 30 11 7 49 36 29 20 10 17 36
80 60 80 60 40 80 80 80 80 40 60 60 60 100 80 100

Conclusion

Although digital natives switch from one task to another as afforded by the advanced digital
technologies they grew up with, it does not follow that they can be academically productive
in it all the time. The tool we developed allowed our experiment subjects to realize both the
positive and negative aspects of their learning behavior.
Our results are clearly preliminary. We aim to find possible generalizations of learning
behaviors across learners, which would mean significantly increasing the population of our
subjects as well as the amount of time we expose them to our tool. Secondly, we aim to
lessen the cognitive load being imposed upon the user by the annotation process. This may
mean automating to some effective degree the inference of intention and affective states.

Acknowledgements
This research work is supported in part by the Management Expenses Grants for National
Universities Corporations from the Ministry of Education, Culture, Sports, Science and
Technology of Japan and by the JSPS KAKENHI Grant Number 23300059.

References
[1]
[2]
[3]

[4]
[5]
[6]
[7]
[8]
[9]

[10]
[11]
[12]
[13]
[14]
[15]
[16]

[17]
[18]
[19]
[20]
[21]
[22]

Aleven, V. (2010). Rule-based cognitive modeling for intelligent tutoring systems. Advances in
Intelligent Tutoring Systems, pp. 33-62.
Azevedo R. & Feyzi-Behnagh R. (2011). Dysregulated learning with advanced learning technologies.
Journal of e-Learning and Knowledge Society, English Edition, 7(2), 9-18.
Azevedo, R., Johnson, A., Chauncey, A. & Burkett, C. (2010). Self-regulated learning with MetaTutor:
Advancing the science of learning with MetaCognitive tools. New Science of Learning: Computers,
Cognition, and Collaboration in Education, pp. 225247.
Bandura, A. (1997). Self-efficacy: The Exercise of Control. New York: Freeman.
Craig, S. D., Graesser, A. C., Sullins, J., and Gholson, B. (2004). Affect and learning: An exploratory
look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241-250.
Ekman, P. (1999). Basic emotions. The Handbook of Cognition and Emotion, 45-60.
Foehr, G. (2006). Media multitasking among American youth: Prevalence, predictors and pairings.
Kaiser Family Foundation.
Hartman, J., Moskal, P. & Dziuban, C. (2005). Preparing the academy of today for the learner of
tomorrow. Educating the Net Generation, pp. 6.1-6.15.
Inventado, P.S., Legaspi, R., Suarez, M. & Numao, M. (2011). Investigating transitions in affect and
activities for online learning interventions. Proceedings of 19th ICCE (pp. 571-578). Chiang Mai,
Thailand: Asia-Pacific Society for Computers in Education.
Kerka, S. (2007). Study Skills. What Works - Evidence-based strategies for youth practitioners.
Learning Work Connection, The Ohio State University.
Kirschner, P.A. & Karpinski, A.C. (2010). Facebook and academic performance. Computers in
Human Behavior, 26(6), 1237-1245.
Luo, M. M., Nahl, D. & Chea, S. (2011). Uncertainty, Affect, and Information Search. Los Alamitos,
CA, USA. IEEE Computer Society.
Mitrovic, A. (2012). Fifteen years of constraint-based tutors: what we have achieved and where we are
going. User Modeling and User-Adapted Interaction, 22, 39-72.
Moos, D.C. (2010). Self-regulated learning with hypermedia: Too much of a good thing? Journal of
Educational Multimedia and Hypermedia, 19(1), 5977.
Oblinger, D. & Oblinger, J. (2005). Is it age or IT: First steps toward understanding the Net Generation.
Educating the Net Generation, pp. 2.1-2.20.
Picard, R. W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., Machover, T., Resnick,
M., Roy, D. & Strohecker, C. (2004). Affective learning - A manifesto. BT Technology Journal, 22(4),
253-269.
Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1-6.
Roberts, G.R. (2005). Technology and learning expectations of the Net Generation. Educating the Net
Generation, pp. 3.1-3.7.
Roberts, D.F., Foehr, U.G. & Rideout, V. (2005). Generation M: Media in the lives of 8-18 year olds.
Kaiser Family Foundation, Menlo Park, CA.
Rubinstein, J.S., Meyer, D.E. & Evans, J.E. (2001). Executive control of cognitive processes in task
switching. Journal of Experimental Psychology: Human Perception and Performance, 27(4), 736-797.
Schunk, D. H. (1995). Self-efficacy and education and instruction. Self-efficacy, Adaptation, and
Adjustment: Theory, Research, and Application, 281-303.
Tapscott, D. (2009). Grown up digital: How the Net Generation is Changing Your World. McGraw-Hill.

Variation Based Discovery Learning Design in


1 to 1 Mathematics Classroom
Euphony F. Y. YANG*, Hercy N. H. CHENG, Emily CHING, Tak-Wai CHAN
Graduate Institute of Network Learning Technology, National Central University, Taiwan
*
euphony@cl.ncu.edu.tw

Abstract: Learning retention is an important issue for instruction. However, traditional


mathematics teachers tend to use direct instruction to tell students a formula, and ask them to
keep in mind. But students may forget it very soon. This study adopts a variation based
discovery learning strategy, which controls the similarity and difference among examples
and problems to guide students to discover the critical features of mathematical concepts
through personal observation and inference. A one-to-one technology-enhanced learning
system is therefore designed to provide cognitive tools and scaffolding mechanisms for the
guidance of discovery. To investigate the learning effect, a pre-test and a post test were
conducted. The result of post test showed that the average score of the experiment group
were significantly higher than that of the control group. The interview data revealed that the
perspectives of some students showed higher self-confidence and learning motivation.
Keywords: Guided discovery learning, critical feature, one-to-one technology-enhanced
learning, elementary mathematics

1.

Introduction

The traditional mathematics education emphasizes on revealing and applying mathematical


theorems. In other words, teachers tell students a formula and assign them exercises to make
them keep in mind. But telling students a correct mathematical operator might hamper their
ability of recall the operator [1], and also makes math a boring or daunting course.
Comparing with traditional direct instruction, education experts widely believed that
discovery learning, which is different from providing complete explanation of concept in
direct instruction, was an effective way of profound and lasting understanding for students
learning [9] [15] [10].Bruner [2] proposed four benefits of discovery learning better than
direct instruction: (1) Growing intellectual. (2) Rewarding from the initiative discovered
process and getting satisfaction. (3) Learning the "discovery" method, the capacity of
analogy and independent learning. (4) Memorizing knowledge longer. McDaniel and
Schlager [16] also pointed out that students in discovery learning were more able to utilize
and expand knowledge. Students must discover and induct themselves.
However, only in discovery learning with proper guidance, students may learn better
than in direct instruction [15]. The reason perhaps is that students tend to try aimlessly if
discovery learning activities have no or insufficient guidance or students lack of precise
objectives and discovery skills. Even if students are engaged in learning tasks, there are not
much knowledge constructions. Furthermore, lacking clear instruction, students who have
less prior knowledge are hard to get basic information and become frustration.
Besides, inappropriate guidance may force students followed the instruction one by
one to complete the task; guiding too much also reduced the discovery ingredients.
Therefore, guiding instance design and choices were extremely important. Related literature
have discussed that discovery learning needed guidance or not (e.g. [10][15]), and what

kinds of guidance were effective [5][19][7][17][18]. However, how to design and present
learning materials in the discovery learning environment were limited, and the using of
computer-aided discovery learning in mathematics was only sporadic [6].
This study proposes a discovery learning environment design which guide properly to
promote students to find and organize the critical concepts through summarizing the
mathematics text description. Therefore, this study has two research questions:
1) How is the learning effectiveness of variation-based-discovery learning?
2) Could the variation based discovery learning facilitate students' learning
motivation?

2.

Methods

2.1 Participants
This study held an experiment on a formal mathematics class for one year. Each week, the
experiment was conducted three times, and each time lasted 40 minutes. The participants
were the third grade primary students in north Taiwan. There were two groups. One was
experiment group (n = 26), which used the guided discovery learning approach supported
with one to one devices to learn mathematics [4]. The control group (n = 26) used traditional
direct instruction.
2.2 Design Structure of Learning Activity
Many researches put into the development of guided tools in guided discovery learning [20].
Comparing with those guided tools, this study designs cognitive scaffolding tools to help
students dealing with the task and go beyond their original extent [8] [11]. More
specifically, the tools had the following characteristics:
1. Using focus to guide students discovering the critical features from examples for
reducing the cognitive loading and establish the important attributes of concept [13].
2. Using learning content itself as the guidance to reduce the extra guidance and
controlling the similarities and differences between examples to naturally highlight the
critical features. In other words, seeking common ground in diversity and seeking
difference in similarities may be easier noticed.
The design structure of learning material was divided into three parts: observation,
identification, and generalization. In the observation step, the students focused on the
relationship between examples and questions. The design of observation highlighted the
critical point of concepts. In the identification step, students might check their possible
assumption after completing the related questions. If students answer incorrect, they have to
go back and observe the differences and similarities between examples and questions again;
otherwise the next question would come out for students to make sure the possible manner
and answer the next question on the correct base. In the generalization part, our design used
short sentences and symbols as the algebra to help students describe the critical feature in a
summarized statement for ensuring their mathematical concepts.
2.3 Theoretical Structure
This study based on the variation theory which was originated from Bruners [2] discovery
learning and proposed by Marton, Runesson, & Tsui [14]. Considering the primary school
mathematical concepts of relevant learning theory and the assistive role computer played in
the classroom, we controlled the similarity and differences of examples and established

theoretical structure to help students learn mathematical concepts. Variation theory


highlighted the key point which students needed to notice and identify from the overall
phenomena. It was only when students noticed and linked more than one thing change in
appearance at the same time, they could experience the similarities and differences [12]
[14]. As for how to change was based on four modes of variation theory: (1) Contrast:
Provide positive and negative examples in the same type to compare. (2) Generalization:
Present different concept facet for students extracting the general rule. (3) Separation:
Change the results corresponding to the value of key property, but other properties held
constant. (4) Fusion: Multiple properties changes at the same time.
2.4 Data Collection
This study considered two parts to evaluate the learning effect of variation based discovery
learning: for the academic ability, learning effectiveness, pre-test in beginning of the first
semester before first mathematics instruction and post test after one week of the midterms
test were used and to prevent the practice effect, post-test using a parallel test and also
changed the questions order and numbers; for the affection conservation, such as students
mathematics motivation. We interviewed teacher and six high, medium, and low
performance students of experiment group about their learning perspective.

3.

Result

3.1 Comparison of Learning Performance


Using variation based discovery learning approach to learn the concept comprehension of
mathematics, students needed to observe examples, imitate examples and words
summarized to explain to the learning concepts. From the result of a t test to compare the
pre-test scores of this approach with the traditional teachers direct instruction, t (2, 51) =
0.447, p =0.657>.05, there was no significant difference. However, after the experiment of
half year, the average score of post test in direct instruction group was 63.5 points and that
of variation based discovery learning approach group was 75.92 points. The average score
difference of the two groups was 12.42 points. There was significant difference between
two group, t (2, 51)=-2.429, p=0.019<.05, so the learning performance of variation based
discovery learning approach was better than the direct instruction.

Figure 1. Pre-test and post-test scores of


discovery learning and direct instruction

Figure 2. Post-test scores histogram of


discovery learning and direct instruction

Furthermore, discovery group not only improved significantly, but their standard
deviation also dropped from 18.24 points in the pre-test to 13.95 points in the post test.
However, the standard deviation of direct instruction increased from 16.23 points to 21.90
points. It meant that using one to one computer supported discovery learning approach
could lessen the students learning difference.

3.2 Learning Interest


Compared to traditional teachers teaching, students of low and medium achievement
preferred variation based discovery learning, and they also cultivated the concentrative and
quiet habits of self-directed learning in mathematics. A medium-achieving S1 said: "I prefer
the discovery learning more. I want to think. The teacher always teaches one after another,
I feel that it was too fast. Low-achieving students generally preferred the discovery
learning. A low-achieving S11 said: "I like to see, discover and find out the key points. I do
not like the teachers telling me about what is the key point, although sometimes it is difficult
to discover on my own. Students feel more solid and less pressure to control their learning
speed and proceed to the next concept after they real understand the critical feature rather
than pursuit the teachers fast instruction, to hurry completing a lesson but did not real know
the learning content. So discovery learning enhanced the self-confidence and
accomplishment of medium and low achieving students.
But not everyone liked computer-guided discovery learning. For example, the
high-achieving students had different opinions. S5 responded that he liked the teacher
instruction. He said computer instruction was boring, there were only math problems. It
always asked me to finish the math questions. Unlike teacher would give us practice after
teaching." High achieving students could follow the teacher's teaching pace and relatively
had no learning difficulties in the learning process. However, the variation based discovery
learning only guided through questions, which was unable to meet the students emotional
needs and wanted a real teacher to promote their learning motivation and enthusiasm.
Some students were frustrated because some questions they tried many times yet not
found the critical features. Some students needed additional guidance in variation based
discovery learning to successfully find the critical feature between the questions presented.
The teacher said: "1/3 children were very excited and want to surpass themselves, but 2/3
children encountered bottlenecks. In variation based discovery learning, students had no
way when they couldnt pass in one stage. So they would be very frustrated and afraid of this
course. Even if the variation based discovery learning had cognitive scaffolding and
prompted students to solve problems, it still unlike real teacher who could find out students'
individual problems and provide suitable instruction. The current system couldnt do so
precise detections of the problem difficulties. Students couldnt pass some stage by using
variation based discovery learning cause them to stay put and feel depression.
Overall, most students expected teachers oral encourage. Some students remarked that
they were used to the traditional instruction which teachers and textbooks directly told the
concepts and answers, even if they could answer by observing examples from the computer
and quite had achievability, but still hoped to acquire knowledge from the direct instruction.

4.

Conclusion

This study focused on the design method, learning effect, and learners interesting in
variation based discovery learning in 1:1 mathematics. This study proposed a guide strategy
based on variation theory to control the similarities and differences between examples to
facilitate students to be aware of the critical features of concepts for further analysis,
reasoning and inductive to learn knowledge. According to this strategy, this study design
computer-aided function to help students follow a certain learning process and discovery
activities to ensure in correct exploring direction, and provide scaffolding to facilitate
students' discovery process and describe results in a summarized form. This guide strategy
gives students effective guidance and is able to retain the opportunity of self-discovery for

students. Therefore, this study may bring new thinking direction for the research community
of discovery learning.

Acknowledgements
The authors would like to thank the National Science Council of the Republic of China,
Taiwan, for financial support (NSC 101-2811-S-008-009, NSC 101-2631-S-008-003, NSC
100-2511-S-008-013-MY3, & NSC 99-2511-S-008-002-MY3), and Research Center for
Science and Technology for Learning, National Central University, Taiwan.

References
[1] Bernstein, D. A., Clarke-Stewart, A., Roy, E. J., & Wickens, C. D. (2003). Psychology. Boston:
Houghton-Mifflin.
[2] Bruner, J. S. (1961).The act of discovery. Harvard Educational Review, 31, 2132.
[3] Brunstein, A., Betts, S. A., & Anderson, J. R. (2009). Practice enables successful learning under minimal
guidance. Journal of Educational Psychology, 101 (4), 790-802.
[4] Chan, T.-W., Roschelle, J., Hsi, S., Kinshuk, Sharples, M., Brown, T., et al. (2006). One-to-one
technology-enhanced learning: An opportunity for global research collaboration. Research and Practice
in Technology Enhanced Learning Journal, 1, 3-29.
[5] de Jong, T. (2005).The guided discovery principle in multimedia learning. In R. E. Mayer (Eds.), The
cambridge handbook of multimedia learning (pp.215-218). New York: Cambridge University Press.
[6] de Jong, T., Hendrikse, H. P., & van der Meij, H. (2010). Learning mathematics through inquiry: a large
scale evaluation. In M. Jacobson, & P. Reimann (Eds.), Designs for learning environments of the future:
International perspectives from the learning sciences (pp.189-205). New York: Springer Verlag.
[7] Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and Achievement in
Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational
Psychologist, 42 (2), 99-107.
[8] Jonassen, D.H. (1999). Computers as mindtools for schools: Engaging critical thinking. Upper Saddle
River, NJ: Prentice-Hall.
[9] Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of
direct instruction and discovery learning. Psychological Science, 15 (10), 661-667.
[10] Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why Minimal Guidance During Instruction Does Not
Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and
Inquiry-Based Teaching. Educational Psychologist, 41(2), 7586.
[11] Lajoie, S., & Derry, S. (1993). Computers as cognitive tools. NJ: Erlbaum: Hillsdale.
[12] Lo, M. L., Pong, W. Y., & Chik, P. P. (Eds.). (2005). For each and everyone: Catering for individual
differences through learning studies. Hong Kong: Hong Kong University Press.
[13] Tuovinen, J. E., & Sweller, J. (1999). A comparison of cognitive load associcated with discovery
learning and worked examples. Journal of Educational Psychology, 91 (2), 334-341.
[14] Marton, F., Runesson, U., & Tsui, B. M. (2004). The space of learning. In F. Marton, A. B. Tsui, P. P.
Chik, Y. P. Ko, M. L. Lo, D. F. Ng, et al., Classroom discourse and the space of learning. Mahwa, NJ:
Erlbaum.
[15] Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? The case for
guided methods of instruction. American Psychologist, 59 (1), 14-19.
[16] McDaniel, M., & Schlager, M. (1990). Discovery learning and transfer of problem-solving skills.
Cognition and Instruction, 7, 129-159.
[17] Moreno, R. (2004). Decreasing cognitive load in novice students: Effects of explanatory versus
corrective feedback in discovery-based multimedia. Instructional Science, 32, 99-113.
[18] Prasad, K. S. (2011). Learning mathematics by discovery. A Multidisciplinary Journal, 1 (1), 31-33.
[19] Van Joolingen, W. (1999). Cognitive tools for discovery learning. International Journal of Artificial
Intelligence in Education, 10, 385-397.
[20] Varnhagen, C., Elio, R., Henry, A., Braga, J., Dyer, C., Haag, M., & Sundararaj U. T. (2009). Teaching,
Research,
and
Discovery
Learning.
Retrieved
from
http://www.psych.ualberta.ca/~varn/Documents/TeachResearchCLEJan10.pdf.

Explaining My Solutions: An Integrated


Model of Peer Tutoring for Facilitating
Mathematical Communication Abilities
Hercy N. H. CHENG*, Euphony F. Y. YANG, Ying-Yen HUANG, & Tak-Wai CHAN
Graduate Institute of Network Learning Technology, National Central University, Taiwan
*hercy@cl.ncu.edu.tw

Abstract: For elementary students, mathematical communication ability is a fundamental


learning objective. Previous studies have showed that self-explanation could allow students
to inspect what they have learned in the learning process, while peer tutoring could facilitate
students to reorganize their learning knowledge as well as to express their idea. As a result,
both could enhance students learning performance. Therefore, this study aims to design a
system that may increase the mathematical communication abilities of elementary students.
The study also conducts a learning activity which incorporates representation generation,
self-explanation and peer-explanation in mathematical word problem solving. Preliminary
evaluation shows that the integrated model may facilitate students mathematical
communication abilities.
Keywords: mathematical communication, self-explanation, peer-explanation, peer-tutoring

1.

Mathematical Communication

Even if children have not learnt mathematics, they live in a world with numbers and shapes.
Mathematics helps people understand the world by simplifying complex problems, solving
them reasonably, and conveying the solution to other people persuasively. However, our
primary education about mathematics focuses too much on problem solving and ignores the
importance of mathematical communication. Mathematical communication involves
adaptive reasoning (Kilpatrick, Swafford, & Findell, 2001, p. 170) and even argumentation
(Andriessen, 2006).
In terms of adaptive reasoning, students have to acquire the ability to think logically, to
explain a mathematical concept or procedure, and to justify their own or others assertions.
Adaptive reasoning also relates to the usage of representation (English, 1997). The ability to
use appropriate representation can facilitate conceptual understanding, and problem
solving. In terms of argumentation, students have to elaborate what they think, and to debate
with sufficient evidences (Toulmin, 1958). When students attempt to build arguments, they
aim to produce their mathematical ideas. For doing so, they may direct themselves to learn
new concepts and procedures.
Self-explanation (or think aloud) is a domain-general learning strategy (Chi, de Leeuw,
Chiu, & Lavancher, 1994), which emphasizes the linkage between prior knowledge and
new one (Chi & van Lehn, 1991). Previous research has shown that successful problem
solvers can generate more explanation (Chi, Bassok, Lewis, Reimann, & Glaser, 1989).
Compared with self-explanation, peer-explanation is an interactive explanation strategy,
which can be applied in a natural and social learning environment. Among various
peer-explanation pedagogies, peer instruction is a widely adopted and effective pedagogy,

which allows students to explain their own ideas for reducing misconceptions (Mazur,
1997).
Furthermore, students may benefit from tutoring others (Cohen, Kulik, & Kulik, 1982;
Rohrbeck, Ginsburg-Block, Fantuzzo, & Miller, 2003) as well as preparing teaching
materials (Ching, Chen, Chou, Deng, & Chan, 2005). Additionally, peer teaching facilitates
spontaneous and appropriate use of diagrams in order to solve mathematics word problems
(Uesaka & Manalo, 2007; 2011). Therefore, this study aims to design a system to support a
peer tutoring model, which integrates generating representations, self-explanation and
peer-explanation for facilitating students mathematical communication ability.

2.

Activity Design

As shown in FIGURE 1, the model consists of three main phases: material preparation, peer
teaching and public teaching.

FIGURE 1. Activity Model


In the first phase, every two students are paired as a group and receive two similar but
different mathematical word problems. They are told that they have to teach each other one
of the two problems, and thus have to prepare their teaching materials. For doing so, they
should solve their own word problems in a way to draw representations, to formulate
expressions, and to calculate their answers on their own tablet PCs. They are also asked to
practice their teaching by self explaining.

FIGURE 2. Interface
In the second phase, they reciprocally play as a tutor and a tutee. For this reason, the
tutor has to teach his/her tutee how he/she solve the word problem. The tutor, more
specifically, have to elaborate their representations and expressions to his/her tutee. After
the elaboration, the tutee has to ask his/her tutor questions, and the tutor should answer the
tutee.
In the third phase, the teacher conducts a session of public tutoring. The teacher may
allow several tutor students to teach their own word problems. As the tutoring practices in
the second phase, the tutor students have to elaborate their representations and expressions

to the whole class. The other tutee students have to ask the tutor students questions related to
the tutoring. The tutor students have to response these questions.

3.

Preliminary Evaluation

3.1 Research questions


This study focuses on a main research question whether the peer tutoring model can
facilitate students mathematical communication ability. More specifically this research
question can be divided into three sub-questions:
1. Can the peer tutoring model facilitate students ability to explain self mathematical
procedure?
2. Can the peer tutoring model facilitate students ability to explain others mathematical
procedure?
3. Can the peer tutoring model facilitate students ability to explain others mathematical
statements?
3.2 Settings
The participants were two second-year classes (N1=25, N2=26), in which students had
similar mathematical communication abilities (see 3.3 for more details). One of the two
classes was assigned as the experimental group, in which the integrated model of peer
teaching was conducted for eight weeks. Another class was assigned as the control group, in
which students received traditional courses of word problem solving.
The teacher in the experimental group could conduct the activity one or two sessions in
a week and each session took 80 minutes. In this experiment, students were participated in
the activity thirteen times in total. The materials were mathematical word problems, which
involved addition, subtraction and multiplication. More specifically, in the first seven
sessions, the word problems were about addition and/or subtraction, while they further
involved multiplication in latter six sessions.
3.3 Measures
In this study, the dependent variable was the mathematical communication ability. For this
purpose, a test on mathematical communication was conducted. This test, developed on the
basis of a (Lin, & Lee, 2004), consisted of three sub-abilities: the ability to explain self
procedures, the ability to explain others procedure, and the ability to explain others
statements.
Independent t tests show that there are no significant differences between experimental
and control groups in terms of explanation for self procedure (t(49)=0.879, SE=0.362,
p>0.05), explanation for others procedure (t(49)=0.861, SE=0.759, p>0.05), and
explanation for others statements (t(49)=0.541, SE=0.466, p>0.05). Therefore, the
mathematical communication abilities of experimental and control groups are similar.
3.4 Results
FIGURE 3 illustrates the results of students mathematical communication ability. First, in
terms of the ability to explain self procedure, a two-way ANOVA reveals that there is a
significant interaction between groups and time (F(1, 49)=7.441, MSE=2.222, p<0.05). As
shown in FIGURE 3(a), both of the experimental and control group performed significantly

better in the post-test than in the pre-test (the experimental group: t(24)=7.955, SE=0.362,
p<0.05; the control group: t(25)=2.744, SE=0.463, p<0.05). An independent t test on the
post-test further indicates that there is a significant difference between the experimental and
control group (t(49)=3.788, SE=0.509, p<0.05). This result shows that the integrated model
of peer tutoring can facilitate the ability to explain their own procedures. Although the
ability of the control group increase as well, the students who have the experience of peer
tutoring can improve more.
Second, in terms of the ability to explain others procedure, a two-way ANOVA shows
that there is a significant interaction between groups and time (F(1, 49)=19.831,
MSE=3.947, p<0.05). Interestingly, while the experimental group improved significantly
(t(24)=2.206, SE=0.508, p<0.05), the control group performed significantly worse in the
post-test than in the pre-test (t(25)=-3.990, SE=0.598, p<0.05). The reason is probably that
the pre-test asked students to distinguish and explain a wrong procedure, and the post-test
asked student to distinguish and explain a correct procedure. The results may raise a further
question can students explain more about a wrong procedure than about a correct one,
which need further investigation.
Third, in terms of the ability to explain others statements, a two-way ANOVA shows
that there is a significant interaction between groups and time (F(1, 49)=27.583,
MSE=2.229, p<0.05). Further analysis indicates that while the experimental group
significantly improved their performance in the post-test (t(24)=6.157, SE=0.448, p<0.05),
the performance of the control group did not change significantly (t(25)=-0.892, SE=0.388,
p>0.05). The results suggest that the experience of peer tutoring can facilitate students to
understand and explain others statements.

(a) explanation for self procedure

(b) explanation for others


procedures

(c) explanation for others


statements

FIGURE 3. The effect on mathematical communication abilities

4.

Concluding Remarks

This study aims to incorporate representation generation, self-explanation and


peer-explanation into an integrated peer tutoring model in order to facilitate students
mathematical communication abilities. The results showed that the integrated model may
significantly improve students ability to explain self procedures, others procedures, and
others statements. Furthermore, this study also revealed that students could explain
procedures in a more complete and more contextualized way. Besides, students could
generate more and more abstract representations, when they prepared teaching materials for
teaching their classmates. Students were also found that they became more enthusiastic and
confident about teaching in public. These findings suggested that the integrated model could
help not only students cognition, but also their affects.

Acknowledgements
The authors would like to thank the National Science Council of the Republic of China,
Taiwan, for financial support (NSC 101-2811-S-008-009, NSC 101-2631-S-008-003, NSC
100-2511-S-008-013-MY3, & NSC 99-2511-S-008-002-MY3), and Research Center for
Science and Technology for Learning, National Central University, Taiwan.

References
[1] Andriessen, J. (2006) Arguing to Learn. The Cambridge Handbook of the Learning Sciences, Chapter 26,
Edited by Sawyer R. K., Published by Cambridge University Press.
[2] Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: how
students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
[3] Chi, M. T. H., de Leeuw, N., Chiu, M. H., & Lavancher, C. (1994). Eliciting self-explanations improves
understanding. Cognitive Science, 18, 439-477.
[4] Chi, M. T. H., & van Lehn, K. A. (1991). The content of physics self-explanations. The Journal of the
Learning Sciences, 1(1), 69-105.
[5] Ching, E., Chen, C. T., Chou, C. Y., Deng, Y. C., & Chan, T. W. (2005). P3T: a system to support
preparing and performaing peer tutoring. In C. K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds),
Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed
Technology. (pp. 768-770). Amsterdam, The Netherlands: IOS Press.
[6] Cohan, P. A., Kulik, J. A., & Kulik, C. C. (1982). Educational outcomes of tutoring: a meta-analysis of
findings. American Educational Research Journal, 19(2), 237-248.
[7] English, L.D. (Ed.). (1997). Mathematical Reasoning: Analogies, Metaphors and Images. Lawrence
Erlbaum & Associates.
[8] Kilpatrick, J., Swafford, J., & Findell, B. (Eds.). (2001). Adding it up: Helping children learn
mathematics. Washington, DC: National Academies Press.
[9] Lin, Y. H., & Lee, Q, Y. (2004). A study of ability on mathematics communication for pupils of
elementary schools. Journal of Research on Measurement and Statistics, 12, 233-268. (in Chinese)
[10] Mazur, E. (1997). Peer instruction: a users manual. NJ: Prentice Hall, Upper Saddle River.
[11] Rohrbeck, C. A., Ginsburg-Block, M. D., Fantuzzo, J. W., & Miller, T. R. (2003). Peer-assisted learning
interventions with elementary school students: a meta-analytic review. Journal of Educational
Psychology, 95(2), 240-257.
[12] Toulmin, S. E. (1985). The uses of argument. Cambridge: Cambridge University Press.
[13] Uesaka, Y., & Manalo, E. (2007). Peer instruction as a way of promoting spontaneous use of diagrams
when solving math word problems. In D. McNamara & J. Trafton (Eds.), Proceedings of the 29th Annual
Cognitive Science Society (pp. 677-672). Austin, TX: Cognitive Science Society.
[14] Uesaka, Y., & Manalo, E. (2011). The effects of peer communication with Diagrams on students math
word problem solving processes and outcomes. In L. Carlson, C. Hoelscher, & T. Shipley (Eds.),
Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 312317). Austin, TX:
Cognitive Science Society.

The Effects of STEAM by RST Instruction


Using Writing on Elementary School Students
Problem-Solving Skills
abcd

SuRyun Jeona*, SunHwoi Kimb & DoongSoo Namc &TaeWuk Leed


Dept. Computer Education, Korea National University of Education, Korea
*ynot1178@nate.com

Abstract: Problem- solving skills are mandatory for everyone in the 21st century. The goal
of STEAM also is to increase the problem-solving skill based convergence thinking. So, in
this study, I proposed a study program of the STEAM by RST(Reverse Science from
Technology) for improving problem-solving skills. The topics are hi-tech products which
are the convergence product of Science, technology, culture and art like Refrigerators and
mobile phones. This program is starting from the exciting technology and conversely
learning basic science, mathematics, art and etc. That learns technology in high-tech
products which were subject of curiosity, and then learn mathematics and science, which are
the basis of the technology. So, learners will learn how to use basic studies to their life and it
increases understanding of course content as well as the interest in learning. Also, that
provides learning opportunity of Language art by utilizing writing in the learning process
and learners can recognize and solve problem on their own to found and modify their
thinking.
Keywords: Problem-Solving Skill, STEAM, RST, Convergence Education, Writing

Introduction
The topic of education in todays information society is the problem-solving skills.
Problem-solving skills are mandatory for everyone in the 21st century by Partnership for
21st century skills, DeSeCo(Definition and Selection of Key Competencies) report of
OECD, SCANS(Secretary's Commission on Achieving Necessary Skills). The complex
problems in 21st centurys information and knowledge-based society require
problem-solving skills based convergence thinking. STEAM is also aimed at improving
problem-solving skills based convergence thinking. STEAM is convergence with Science,
Technology, Engineering Mathematics and Art. This study proposed a study program of the
STEAM by RST(Reverse Science from Technology) using writing for improving
problem-solving skills. This program is starting from the exciting technology and
conversely learning basic science, mathematics, art and etc. The topics are hi-tech products
which are the convergence product of Science, technology, culture and art like refrigerators,
mobile phone, etc. These products are used very frequently in everyday life of learners.
Furthermore, the program provides the opportunity for reflective thought by using writing
in learning process. So, learners can recognize and solve the problem by themselves

1.

STEAM by RST

STEAM is one of convergence study, an advanced concept of STEM. STEM is a

convergence with Science, Technology, Engineering, and Mathematics. STEAM is adding


=to Art to STEM. This Art means Fine Art, Liberal Art, Language Art and so on. [Figure 1]
is the STEAM Pyramid. RST is an abbreviation of the Reverse Science from Technology.
STEAM by RST teaches the principles of high-tech products such as mobile phones, TV,
car, etc and integrated system. That increases learning interest and achievement by
understanding how basic science is used in every life. Furthermore, learners can recognize
the truth how to increase the value of the product by design and User Interface.

2.

Problem- Solving Skills

General Problem-solving skills are procedural knowledge which is procedure and method.
These can be generally applied to solve a widespread and various problems in the real
world. Creative problem-solving skills refer to every kind of process that an individual or a
group thinks creatively to solve a certain problem or to such efforts. These are concepts
which emphasize process than outcome. So, the education for improving problem-solving
skills should be focused on the process of solving the problem.
The common process of problem-solving by OECD 2004, Polya can be summarized as
follows: First, understand and analyze the meaning of the problem. Second, solve the
problem. Third, reflect and evaluate the ideas. So, this program was designed that learners
proceed the process of problem-solving and recognize and solve the problem by themselves.

3.

Writing

The process of writing is very similar to the process of solving the problem. Many subjects
such as Mathematics, Science, Social Studies, Art, etc focus on these similarities and use
actively the writing. Based on these previous studies, the educational effects of writing can
be summarized as follows: First, writing is a tool for finding problems that is beginning of
solve the problem. Second, writing provides the opportunity for reflective thinking. So,
learners can be helped to solve the problem by themselves. Third, Learners are to participate
actively in their learning. Fourth, Effective communication is possible by writing. So,
learners can review the idea of solving problems with peers. Fifth, writing provides the
opportunity to think deeply about the representation of the target. That can be useful
STEAM education including art.

4.

Methodology of the STEAM by RST using Writing Study Program

4.1 Directivity and select content of Study program

Prepare standards and emphasis on STEAM by RST, writing, the process of


problem-solving.
Consider elementary school students interest study level, learning ability.
Choose the theme which learners can meet easily in real life

4.2 The concept model of STEAM by RST using writing

Figure 1. STEAM Pyramid

Figure 2. Concept model of STEAM by


RST using writing

4.3 Design the model for teaching


Table 1. The model for STEAM by RST using writing and content
Polyas problem solving
Understanding the
problem

STEAM by RST
Presenting the problem
Analyzing the problem

Devising a plan

Devising a plan

Carrying out the plan

Carrying out the plan

Looking back

Looking back

5.

Writing Content
- Question and answer for Motivation
- Draw a picture of the subject and write the role of
the component
- Compare with peers and modify
- Write experience about the topic
- Explorer and write scientific principles on the topic
- Write the uncomfortable points during using the
topic and writing solution
- Write algorithm flowchart
- Discuss the scientific principles on the topic
- Creative Design
- Presentation
- Review the idea

Conclusion and Future Work

In this study, we developed the STEAM by RST using Writing study program to improve
problem-solving of elementary school. This program could increase learning interest and
problem-solving by hi-tech products. In the learning process, writing will help learners
reflective thinking. To inspect the effects this program, we have to apply this method to real
class. Developed STEAM by RST program will be taught to students in class and well
observe the students learning activities and analyze outcomes of learning.

References
[1] OECD. (2004). Problem solving for tomorrows world : First measures of Cross-Curricular Competencies
from PISA 2003.
[2] The Secretarys Commission on Achieving Necessary Skills. (1991). What work Requires of School.
[3] Gorgette Yakman. (2007) STEAM Education : an overview of creating a model of integrative education.
[4] Polya, G.(1957). How to solve it.
[5] L. Flower. (1998). Problem solving Strategies for Writing in College and the Community.
[6] Kwang-Hyung Lee. (2011). http://www.sciencetimes.co.kr/article.do?atidx=0000048348&todo=view

Facilitating Story Revision Through Peer


Questioning on a Scribble Environment
Siou-Lan WANG*, Calvin C. Y. LIAO, Tak-Wai CHAN
Graduate Institute of Network Learning Technology, National Central University, Taiwan
*shirley@cl.ncu.edu.tw

Abstract: For the past 1 1/2 years, eight second-grade Taiwanese classrooms (n = 207) have
used BookTalker to draw pictures and write stories. Although, this allows for easy sharing
of ones story, students still seem to lack an understanding of what constitutes a quality
story or how to self-improve. The study proposed in this paper aims to extend the use of
BookTalker to enhance the quality of students stories by incorporating scaffolding through
the process of asking questions regarding the content of peers stories, critical analysis of
asked questions, and story revision.
Keywords: story writing, peer questioning, drawing, writing revision, scaffolding

1. Introduction
In traditional elementary school Chinese writing classrooms, students passively execute
writing tasks assigned by teachers with the goal of merely turning in written homework
assignments containing bits of knowledge acquired in past lessons. To approach this
problem, interactive peer questioning on a digital environment is considered as a strategy to
increase students analytical skills in order to improve their written Chinese. For the past 1
1/2 years, eight second-grade Taiwanese classrooms (n = 207) have used BookTalker, an
application of Crazy Brush [3] developed to allow students to create stories through
drawing and writing. The main purpose of the proposed study is to facilitate students story
revision through peer questioning on the scribble environment BookTalker. In addition,
this study attempts to observe specifically what about peer questioning enhances story
quality.
1.1 Sharing and Revising Stories
In traditional Chinese classrooms, computers and the Internet provide potentials of making
such stories meaningful and interactive. By drawing and writing on the computer and then
sharing these creations online with classmates, more interactive opportunities can be created.
In the present study, the researchers present a scenario in which students need to solve
problems (questions) generated by peers.
1.3 Peer Questioning
Several studies have been carried out investigating question generation [1, 2, 6]. Saddler [5]
reports that students with learning disabilities increased their story-writing ability through
self-regulated strategies. Although much interaction took place in the previously reviewed
studies, but do to the fact that this interaction was conducted face-to-face, this interaction
was extremely limited. Hence, the study proposed in this paper provides students with an

extra period of time for interaction through the online-based peer interaction afforded by
BookTalker.

2. Peer questioning to enhance story quality


Based on previous research, we designed a three-session activity to stimulate students peer
questioning: (1) Writing a story; (2) Questioning peers regarding their stories; and (3)
Revising ones own story. In session one, students literacy skills will be improved through
writing practice during the story creation process. Besides, students will be able to gain the
knowledge of the elements that constitute a good story. In session two, students are
empowered to judge and give comments about peers stories. In session three, revising
stories enables students to gain ownership of their stories. At the same time, their literacy
skills can be improved by revising errors.
The teacher will first introduce the system and all BookTalker functions including
how to draw, write, give titles, submit a story, choose anothers story to ask questions,
submit questions, and revise ones own story (see Fig. 1). Then the teacher will illustrate
examples of the seven elements that generally occur in a good story and how the
peer-questioning task should be completed through the use of a sample story included in
BookTalker. Afterwards, an entire class peer questioning story practice using a second
sample story from BookTalker will be conducted to confirm if students can take
story-specific questions into practice. In addition, an instructional video and an image laden
textual description as references are available to students while interacting on BookTalker.

Fig. 1 BookTalker Drawing Interface


First, students choose and read two classmates stories on BookTalker. The drawings
and writing associated with a story is unavailable for preview. Instead students are given a
list of story titles to choose from. Once a story is chosen, the story writers name will be
appended to the story title and the story will be removed from the list of available stories; the
story writers name is shown after selection to aid face-to-face interaction that takes place
during the story revision process.
Second, students are able to propose questions below each picture or on the webpage
that presents all four pictures of the story. Each student is asked to propose at least two
questions for each story chosen. A five-minute countdown begins when a student starts the
process of posing questions on a classmates story; if two questions have not been generated
after the five minutes are up the student will receive links to the instructional video and
reference material as well as the question prompts. After submitting two questions for the
first story BookTalker will then send the student to the second story where the process is
repeated. After all students have proposed questions regarding two classmates stories, each
student revises ones own drawing or writing according to peers questions. Before revision,
students must determine if the questions contribute to the revision. Students decide which
questions to take into consideration or to debate with the question givers face-to-face until a
consensus is reached. After the story is revised, individuals resubmit through BookTalker.

3. Discussion
The purpose of the proposed study is to develop a three-session activity to guide students in
generating questions regarding peers stories in order to improve students Chinese writing
and story creation. Students are expected to improve from three different perspectives; they
are: social, psychological, and cognitive.
Social perspective: Empowerment in the classroom. In this study, the role of reviewer
shifts from teachers to peers. With this empowerment [8], students performance in story
writing is expected to improve allowing students to produce more authentic creative quality
stories.
Psychological perspective: The Ownership of work. Creating and revising stories is not
merely to meet teachers requirements, but also to foster ownership [7]. During the
processes of generating questions and revision, students become deeply involved in their
own and peers story creation to not only improve story quality but also analytical skills.
Cognitive perspective: The practice of literacy skills. Compared with explicitly
teaching students the knowledge of literacy skills, a meaningful writing task of creating and
revising stories enables students to take these skills into practice [4]. Peers questioning
plays the role of facilitating the process of story revision.
The progress of this proposed study so far has been the construction of BookTalker.
The upcoming work is to run the experiment in a third-grade Chinese writing classroom as a
pilot.

Acknowledgements
The authors would like to thank the National Science Council of the Republic of China,
Taiwan, for financial support (NSC 101-2811-S-008-009, NSC 101-2631-S-008-003, &
NSC 99-2511-S-008-002-MY3), and Research Center for Science and Technology for
Learning, National Central University, Taiwan.

References
[1] Andr, M. E. D. A., &Anderson, T. H. (1978). The Development and Evaluation of a Self-Questioning
Study Technique. Reading Research Quarterly, 14(4), 605-623.
[2] King, A. (1990).Enhancing Peer Interaction and Learning in the Classroom Through Reciprocal
Questioning. American Educational Research Journal, 27(4), 664-687.
[3] Lee, Y. C., Liao, C. C. Y., & Chan, T. W. (2010). Crazy Brush: Designing the Scribbles Environment to
Improve Children Interest in Writing In S. L. Wong et al. (Eds.), Proceedings of the 18th International
Conference on Computers in Education (pp.73-75).
[4] MacArthur, C. A., Graham, S., Schwartz, S. S., & Schafer, W. S.(1995). Evaluation of a Writing
Instruction Model That Integrated a Process Approach, Strategy Instruction, and Word Processing.
Learning Disability Quarterly, 18(4), 178-291.
[5] Saddler, B. (2006). Increasing Story-Writing Ability through Self-Regulated Strategy Development
Effects on Young Writers with Learning Disabilities. Learning Disability Quarterly, 29(4), 291-305.
[6] Singer, H., & Donlan, D. (1982). Active Comprehension: Problem-Solving Schema with Question
Generation for Comprehension of Complex Short Stories. Reading Research Quarterly, 17(2), 166-186.
[7] Urdan, T., & Schoenfelder, E. (2006). Classroom effects on student motivation: Goal structures, social
relationships, and competence beliefs. Journal of School Psychology, 44, 331-349.
[8] Zimmerma, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into practice, 41(2),
64-70.

Three Methods of Applying Digital


Technologies to Enhance Teaching and
Learning Traditional Chinese Paintings
Shih-Lun CHEN, Yuan-Hsun CHUANG
Faculty Director, China University of Technology, Taiwan
Assistant Professor, China University of Technology, Taiwan
ricead@gmail.com
Abstract: Digital technologies have enormous impacts on all aspects of art education. By
applying powerful computer technologies in the classroom, various activities become
possible for introducing traditional Chinese painting, which is generally considered dreary,
pedantic and too far removed from modern life by young students today. In this research,
three methods that were applied in a fine art class, especially in primary school and
secondary school, are presented, to describe how digital technologies can be adopted for
relevant pedagogic practices. The three methods include three-dimensional animation,
digital storytelling and interactive modes. It is concluded that these three methods are
powerful and meaningful ways to enhance teaching and learning traditional Chinese
paintings for students of the digital generation.
Keywords: Digital technology, fine art education, pedagogic practice, Chinese painting

1.

Previous Works

Digitalizing oriental fine art such as traditional Chinese painting and calligraphy has
accomplished a remarkable evolution in the past two decades, and has been discussed
intensively. From the pioneering projects of Saito & Takahashi, Haeberli, Hertzmann, and
Freeman et al., systems of generating, rendering or translating painterly brush works were
invented and introduced (Saito & Takahashi, 1990; Haeberli, 1990; Hertzmann, 1998;
Freeman et al., 2003). Many computer devices such as tablets and styluses, and software,
are able to simulate or generate vivid brush works (Xu et al., 2008). From translating,
simulating, generating to animating, technology did digitalize traditional Chinese painting,
and utilized it mostly in the fields of art creation/recreation, commerce or entertainment.
Now people are surprised to see amazing animations with the style of Chinese painting and
enjoy the oriental aesthetic brush works that often appear in daily life via web pages, MTV,
TV commercials, various graphic designs and so on. However, digital applications,
particularly for traditional Chinese painting, still lack attention and need further research in
respect of education.

2. Research Approaches
In order to assist educators to apply appropriate digital technology for pedagogic practices,
three methods are proposed in this section. Each method solves issues of the modern
audiences difficulty with understanding traditional Chinese Painting. Additionally, many
concrete examples are given, to introduce how the methods address the issue.

2.1 Method One: Three-dimensional animation


Traditionally, most Chinese painting is composed on two-dimensional hanging scrolls, both
vertical and horizontal formats, and the spatial effects are more subtle and metaphorical,
rather than the sharp and dramatic effects of Western painting. However, the students of the
digital generation are more familiar with realistic or three-dimensional space and with
moving images, due to the influence of digital media such as TV and video games.
Also, the Qi, atmospheric phenomena throughout the painting, seems too abstract for
modern students to comprehend. Therefore, it is very difficult to invite students to place
themselves in the picture and appreciate the scenes, stories and aesthetics that the art work
conveys. To address this issue, the techniques of modeling, layering and animating were
utilized to transform two-dimensional Chinese painting into three dimensions with moving
actions. In three-dimensional images, it is easier for teachers to interest students in the
picture and to explain the theme. Three-dimensional animations also allow viewers to go
behind or around the objects to look closely, if the brush works are too abstract to
understand. For the philosophical images of Qi and empty space in paintings, it is more
possible for students to experience the mystical aesthetics if they can play around inside the
picture just like they are used to doing in computer games
2.2 Method Two: Digital storytelling
Many traditional Chinese paintings illustrate historical stories or persons, and mark the
content with calligraphic scripts in the painting. However, as time goes on, new generations
are no longer familiar with those events and with handwriting; thus, they have difficulty in
accessing them. To address this issue, multimedia technology was utilized to add story lines,
conversations or role-plays into paintings. Students were able to understand the background,
history and story behind the art more directly and effectively. Some traditional paintings
lend themselves to plots about popular and current issues that are especially applicable to
the daily life of modern times. Figures 1 shows a digital application that adds a story line
about cosmetic skills to introduce a traditional portrait.

Figure 1

Figure 2

2.3 Method Three: Interactive mode


In traditional times, Chinese painting was created and appreciated by scholars and gentry,
and presented elitist art forms that conveyed cultural, historical, political and philosophical
values, as well as artistic skills. It is very challenging for modern people without such an
elite background to fully and truly understand this art. It would not be too much of an
exaggeration to say that to study each traditional painting requires a certain level of
knowledge, a knowledge bank.
To address this issue, an interactive function is invented for traditional Chinese
painting, that served as a powerful database. With the interactive function, students can
actively browse the parts of painting that they are interested in, zoom in for details, react

with the images, or link to more instructions for relevant information. These kinds of
interactive painting were publicly exhibited recently, and received tremendous feedback.
Audiences found it was great fun to play with the art instead of just looking at it (CNN,
2010). The interactive version of Along the River During the Ching-ming Festival (Figure
2) is a great example of digitalizing traditional painting. It is a portrayal of prosperity and
affluence along the Bian River in Kaifeng, capital city of the Northern Sung dynasty. This
collaborative piece painted in 1736 measures 35.6 cm in height and 1152.8 cm in length.
The meticulously rendered painting features unique customs of the Ming and Ching
dynasties, providing a wonderful first-hand account of contemporary lifestyles. In the
interactive version, viewers can browse the entire image and select a section for more detail,
which comprises three-dimensional animations of the architecture, scenes and people.

3. Conclusion and future work


Digitalizing traditional Chinese painting has made tremendous progress in the past two
decades and has been utilized for graphic design, game design and many other
entertainment functions. Now, how to utilize it in classrooms, enrich pedagogic practices
and consequently benefit the teaching and learning of Chinese paintings, is the main
concern. Three methods of digital applications have been introduced and summarized in this
research. All of these applications are amazing in visual effects and received very positive
feedback from audiences in exhibitions. They are also relevant and appropriate pedagogy
because each method addresses the issue of the lack of attractiveness of traditional Chinese
painting for modern students. Furthermore, while it is proposed that these three methods of
digital application be used to enhance the teaching and learning of traditional Chinese
painting, further research is needed to quantify the effectiveness of digital applications for
students in different age ranges and cultural backgrounds. Also, questionnaire of students
can be used for further research to get detail feedbacks from students regard the
effectiveness of the proposed methods.

References
[1] C. Chan, E. Akleman, and J. Chen, Two Methods for Creating Chinese Painting. Proceedings of Pacific
Graphics'02, Beijing, P. R. China, 2002.
[2] J. Lambert, Digital Storytelling: Capturing lives, creating community. Berkeley, CA: Center for Digital
Storytelling, 2002.
[3] N. J. Livo & S. A. Rietz. Storytelling: Process and practice. Littleton. CO: Libraries Unlimited, 1986.
[4] D. Meadows, Digital storytelling: Research-based practice in new media. Visual Communication, 2(2),
189193(2003)
[5] J. Silbergeld, Chinese painting style: media, methods, and principles of form (Seattle and London:
University of Washington Press, 1982).
[6] S. H. Xu, F. Lau, and Y. Pan, A Computational Approach to Digital Chinese Painting and Calligraphy.
(Hangzhou: Zhejiang University Press and Berlin, Heidelberg: Springer 2009).
[7] W. Cohn, Chinese Painting (New York: Hacker Art Books 1978).
[8] Kozma, R. B. (1991). Learning with media. Review of Educational Research, 61, 179211
[9] Roxana Moreno, Alfred Valdez, Cognitive Load and Learning Effects of Having Students Organize
Pictures and Words in Multimedia Environments: The Role of Student Interactivity and Feedback.
ETR&D, Vol. 53, No. 3, 2005.
[10] CNN NEWS, Queue for hours to get tickets to Along the River During the Qingming Festival:
http://www.cnngo.com/hong-kong/life/200000-tickets-sold-out-first-day-glimpse-along-river-during-qi
ngming-festival-666493
[11] Ukiyo-Hakkei, ODD JOB. http://www.youtube.com/watch?v=WDdlEj6UUNQ
[12] Happiness Two. Lee Lee Nam. 2012. http://logoartproject.naver.com/artist5
[13] Happiness Four. Lee Lee Nam. 2012. http://logoartproject.naver.com/artist5

Using Self-generated Drawings to Support


Writing and Storytelling in Language
Learning
Calvin C. Y. LIAO*, Yi-Chian LEE, Maureen WU, Tak-Wai CHAN
Graduate Institute of Network Learning Technology, National Central University, Taiwan
*Calvin@cl.ncu.edu.tw

Abstract: Previous study indicated that self-generated drawings strategy possible will play
a key role on students learning to speak and write from emergent literacy to conventional
phase. Hence, this study developed a self-generated drawings environment to support young
children writing and storytelling. We implemented an activity integrated of drawing,
writing, and storytelling. In particular, each student used the tablet PC to share a story with
drawing and writing product. Besides, some implications about three future directions were
also discussed.
Keywords: self-generated drawings, writing, storytelling, language learning

1.

Introduction

In the last three decade, the relationship between drawing and writing has been explored in
relation to childrens literacy development (Caldwell, & Moore, 1991; Norris, Mokhtari, &
Reichard, 1998). Some studies have suggested that student-generated drawings can
supported for a variety of learning activities (van Meter, & Garner, 2005). In particularly,
these suggestions included construction of knowledge representation (van Meter, & Garner,
2005), preparation for narrative writing (Caldwell, & Moore, 1991), and pre-writing
strategy (Norris, Mokhtari, & Reichard, 1998). On the other hand, for a long time, oral
narrative is considered important for interpersonal interaction and social communication
(Vygotsky, 1986). Some studies considered that the oral narrative about the drawing and
writing could scaffold childrens topical understanding, such as using drawings to planning
and discussion (van Meter, & Garner, 2005). That means that oral narrative about the
drawing and writing allow children to share more detailed and accurate information with
each other; they become acutely aware of the contrast between the pre-drawing-and-writing
and the post-drawing-and-writing. In particularly, children determine whether modify their
original illustrations, recreate new drawings, or add new text that more accurately reflects
their drawing and writing. Above learning activities mean that students self-generated
drawings strategy have potential value for oral language and written language of primary
children. In other words, self-generated drawings strategy possible will play a key role
which students learn to speak and write from emergent to conventional phase. By contrast,
few empirical studies of drawing construction on speaking and writing aspect have been
systematic implementation in formal school, and much remains unknown about this
learning process. Moreover, children in Taiwan lack the opportunity to practice oral
narration and express their opinions (Chang, & Ku, 2008). Norris et al. (1998) also indicated
that there has been limited formal study about the role of drawing in the writing process of
children in primary grades 1-3. If we could combine drawing, writing, and storytelling

which will be more meaningful for childrens language learning. Hence, this study attempts
to support students drawing, writing, and telling their products in order to improve the
writing skills and storytelling abilities of children.

2.

Through Self-generated Drawings Strategy to Support Young Children Writing


and Storytelling

Previous study developed an on-line drawing environment to support young children


writing and to share their products, entitled Crazy Brush (Lee, Liao, & Chan, 2010). We
implemented a drawing, writing, and storytelling activity. These activities in the digital
drawing process can simplify automatic sketch segmentation, which can be used to support
students in creating products from drawings. In particular, each student used the tablet PC to
share a story with drawing based on text of writing. We hope the Crazy Brush system could
let students draw their scribble and have fun, and meanwhile students would write a story
about their scribble. In other words, students would find the interesting element from the
process of drawing their scribble and transfer into writing, and presenting their products.
The main sources of products include two ways: one is reading and the other is life
experience. The Crazy Brush system would provide a writing channel and students could
write a story through describing others scribble. Through the system, students could
mutually share ideas and maybe they could get more creative inspirations and ideas which
will not be limited to reading or experience. Crazy Brush system utilizes three strategies to
enhance students learning and motivation. We hope system design is flexibility and it could
support various activities, whether in the classroom or after school. The Crazy Brush system
function contains three parts: drawing, writing, and portfolio (for storytelling).
2.1 Drawing, Writing, and Storytelling
Drawing: The system provides the most basic functions, for example: brush, eraser, color,
and etc., see Figure 1. Using the eraser tool can easy clear unwanted drafts and the undo tool
can quickly return to the previous step. Paper cannot provide these functions. The button
design principle is intuitive which could avoid children's cognitive overload. Students could
scribble through system function to name their scribble. The system will also investigate
creator the sources of ideas: (1) imagination, (2) their own experience, (3) reading
experience, and (4) other. After the completion of product, the students could choose
whether to share their scribble picture with others and the scribbles could become one of
storytelling resources.
Writing: Students have to write a story in light of choosing their own scribble.
Students can use keyboard input words or handwrite on the screen. Students have to input a
story topic and the system will search related words. Students can write a story using related
words. In this part, the system provides video and sound recording functions and students
could telling a story according to their write story. After story writing or telling, the students
could choose whether to share with others and to publish to portfolio.
Storytelling: The portfolio function would collect students scribble and story, and
students could review all their products. They could press the button and watch others
creative writing or speech. Paper cannot record students scribble process, but the system
provide this function. In portfolio, students can see others scribble process, and they can
learn from each other. In addition, the system also provides recommend feature, students
can vote their favorite creation. We hope to make writing more fun and let students do the
initiative writing and speaking.

2.2 The Design of Activity Flow: Drawing, Writing, and Storytelling


This section describes the Crazy Brush system how to use in the classroom learning
activities. In the activity, teachers provide certain topic of activities. Through these topics,
students need to paint and write a story, see Figure 1. The learning activity includes 3 steps.
Step 1: students need to think and organize the ideas in mind, and then draw pictures. Step 2:
students need to describe and explain these pictures, and then write down some scripts
according to previous ideas. Step 3: students need to share and present these scripts and
pictures with others.
Step 1: Drawing

Step 2: Writing

Step 3: Storytelling

Figure 1. Drawing, Writing, and Storytelling in Crazy Brush.

3.

Remarks

This study developed a self-generated drawings environment to support young childrens


writing and storytelling. We implemented a drawing, writing, and storytelling activity. We
come up with three future directions in which deepening the creation, sharing, and
assessment of story making could be implemented. First, we should focus on story creation
in order to increase the content and structure of story. Secord, we should focus on story
sharing in order to provide the opportunity and interaction of students. Finally, we should
focus on story assessment in order to enhance the sentence usage and article structure.
Acknowledgements
The authors would like to thank the National Science Council of the Republic of China,
Taiwan, for financial support (NSC 101-2811-S-008-009, NSC 101-2631-S-008-003, &
NSC 99-2511-S-008-002-MY3), and Research Center for Science and Technology for
Learning, National Central University, Taiwan. In addition, the authors would like to be
grateful for suggestions and comments provided by Wan-Chen CHANG.
References
[1]
[2]
[3]

[4]
[5]

[6]

Caldwell, H., & Moore, B. H. (1991). The Art of Writing: Drawing as Preparation for Narrative Writing
in the Primary Grades. Studies in Art Education, 32(4), 207219. doi:10.2307/1320873
Chang, W. C., & Ku, Y. M. (2008). Children's oral narrative evaluation and assistance. Journal of
Education Research, 173, 74-84.
Lee, Y. C., Liao, C. C. Y., & Chan, T. W. (2010). Crazy Brush: Designing the Scribbles Environment to
Improve Children Interest in Writing. In S. L. Wong et al. (Eds.), Proceedings of the 18th International
Conference on Computers in Education (pp.73-75). Putrajaya, Malaysia: Asia-Pacific Society for
Computers in Education.
Norris, E., Mokhtari, K., & Reichard, C. (1998). Childrens use of drawing as a pre-writing strategy.
Journal of Research in Reading, 21(1), 6974. doi:10.1111/1467-9817.00043
van Meter, P., & Garner, J. (2005). The Promise and Practice of Learner-Generated Drawing: Literature
Review
and
Synthesis.
Educational
Psychology
Review,
17(4),
285325.
doi:10.1007/s10648-005-8136-3
Vygotsky, L. S. (1986). Thought and language. Cambridge, MA: MIT Press.

Mobile Learning Numeracy in Ontario


Serene Jin Yan HE, Maiga CHANG, & KINSHUK
School of Computing and Information Systems, Athabasca University, Canada
*maiga@ms2.hinet.net

Abstract: The growth of mobile technologies and devices has penetrated the world
substantially over the last 5 years, and a large percentage of these users are young users.
This introduces profound changes to traditional teaching practices which may initially be
seen as low-key, but it will eventually create visible side effects in the education domain that
cannot be ignored. As we push for effective instruction to be differentiated, we can use
mobile technologies to enhance learning. This research will present the mobile device
functions suitable for mobile learning and map it to learning topics of Ontario's K-6
Numeracy Curriculum.
Keywords: Mobile learning, instructional strategy, e-learning, Numeracy curriculum, grade
K-6, Ontario Ministry of Education

Introduction
Recent research is more focused upon mobile learning and the huge potential it has in the
language learning field (Mahruf, Shohel and Power, 2010), it also enhances student learning
of mathematics (Rains and Clark, 2011). Although there are many researches on mobile
learning, the educational value in the use of mobile technology is still limited (Morgan,
2010) there is no standard or model that formally addresses these relationships. The
objective of this research will seek to identify which mobile technologies are most suitable
to be integrated in instructional strategies to deliver knowledge and content.

1.

Ontario's K-6 Numeracy Curriculum and Instructional Strategy

The numeracy domain has always been an area of strong focus because it an essential part of
our life. The learning goals of numeracy as stated by the Ontario Ministry of Education
(2003) is to achieve beyond computation skills, instead students are to be able to solve
problems, process information from various sources and access and use quantitative
information to make knowledgeable decision. Table 1 maps out the learning topics within
the domain of numeracy according to the curriculum set up by the Ontario Ministry of
Education (2003, 2008a, 2008b, 2008c, 2008d).
Table 1. Numeracy Learning Topics
Learning Domain
Numeracy

Learning Topic
Counting, Operational Sense, Quantity, Relationships, Representation,
Geometry and Spatial Sense, Data Management and Probability,
Measurement, Patterning/Algebra

Ontario MoE (2003) identified that effective instruction is crucial to the successful
learning of mathematical knowledge and skills. There are five categories of instructional

strategy: Direct instruction, Indirect instruction, Experiential learning, Independent study


and Interactive instruction. These five categories are inter-related as the instructional mode
can be used in more than one strategy. Table 2 shows a snippet of the instructional strategies
and its instructional modes.
Table 2. Instructional Strategies and its Instructional Mode (Partial)
Instructional Strategy
Direct Instruction
Indirect Instructions
Experiential Learning

2.

Instructional Mode
Structured Overview, Lecture, Demonstrations, Guided and Shared
Problem Solving, Case Studies, Reading for Meaning, Cloze Procedure
Field Trips, Narratives, Conducting Experiments, Simulations

Discovering Relations

Educators recognize the potential of mobile technologies as a learning tool in education. We


need to establish how these mobile device functions relate to instructional modes and how
the relationship is useful to educators. There are different instructional strategies with its
own set of instructional modes that can overlap with each other. Mobile device functions
can be used to assist in the delivery of these instructional modes. New and improved devices
hardware capabilities, speed and innovative program applications using Web 2.0
technologies (e.g., wikis, Twitter, YouTube, MySpace) have made mobile devices more
promising as an educational learning tool (Park, 2011).
Educators can deliver the learning topics like measurement using GPS to see distance
traveled from point A to B, or using GPS to calculate length of time before reaching
destination. Students can use the camera to capture a chosen location with human traffic
during different times of the day as part of a data management topic. Figure 1 below
demonstrates the mapping concept for the relation between Learning Topics, Learning
Styles, Instruction Mode and Mobile learning mode.

Figure 1. Mapping of Learning topic (Counting), Learning Styles,


Instructional Mode and Mobile Learning Mode

3.

Discussions

In Figure 2 below, a tally of mobile learning mode across all instructional modes was total
and charted. There is clear indication based upon the list of instructional modes; there are
some mobile learning modes that are more frequently considered useful. The chart has
Bluetooth at a zero count, it indicates that it not considered useful for any delivery of
instructional mode or learning domains.

Figure 2. Mobile Learning Mode by Instructional Mode

Acknowledgements
The authors wish to thank the support of NSERC, iCORE, Xerox and the research related
gift funding provided to the Learning Communities Project by Mr. Allan Markin.

References
Mahruf, M. , Shohel, C., & Power, T. (2010). Introducing mobile technology for enhancing teaching and
learning in Bangladesh: teacher perspectives. Open Learning: The Journal of Open, Distance and
e-Learning, 25(3), 201-215.
Morgan, H. (2010). Using handheld wireless technologies in school: Advantageous or disadvantageous?
Childhood Education, 87(2), 139-142.
Ontario Ministry of Education. (2003). Early Math Strategy: The report of the Expert Panel on Early Math in
Ontario. Ministry of Education, Ontario, Canada.
Ontario Ministry of Education. (2008a). A Guide to Effective Instruction in Mathematics, Kindergarten to
Grade 6 - Patterning and Algebra. Ministry of Education, Ontario, Canada.
Ontario Ministry of Education. (2008b). A Guide to Effective Instruction in Mathematics, Kindergarten to
Grade 6 - Measurement. Ministry of Education, Ontario, Canada.
Ontario Ministry of Education. (2008c). A Guide to Effective Instruction in Mathematics, Kindergarten to
Grade 6 - Geometry and Spatial Sense. Ministry of Education, Ontario, Canada.
Ontario Ministry of Education. (2008d). A Guide to Effective Instruction in Mathematics, Kindergarten to
Grade 6 - Data Management and Probability. Ministry of Education, Ontario, Canada.
Park, Y. (2011). A Pedagogical Framework for Mobile Learning : Categorizing Educational Applications of
Mobile Technologies into Four Types. International Review of Research in Open and Distance
Learning,
12(2),
78-102.
Elsevier.
Retrieved
June
4,
2012,
from
http://www.irrodl.org/index.php/irrodl/article/view/791/1788
Rains, J., & Clark, L. (2011). A Brief Overview on Using Technology to Engage Students in Mathematics.
Current
Issues
in
Education,
14(2).
Retrieved
June
4,
2012,
from
http://cie.asu.edu/ojs/index.php/cieatasu/article/view/786/215

The Exploration of Online Engagement Data


in LMS as Predictors to E-Learning Outcomes
Ching-Rong LEE
National Open University, Taiwan
*crlee@mail.nou.edu.tw

Abstract: In this study, it is proposed an approach to utilize the students online engagement
data, in terms of the counts, collected by the LMS. Data about 364 students who learned
online throughout a semester was analyzed. Due to the skewed and peaked distribution, the
negative binomial regression was applied to the data analysis. The test scores and time spent
in e-learning produce the significant effects on the log of the counts of the LMS login, the
counts of course studying, as well as the counts of the e-pages read. It was shown that using
the count outcome variables can form the relationships with the predictors in a linear model.
Keywords: E-learning, Online engagement, Learning Management System

Introduction
American National Survey of Student Engagement (ANSS) addresses the concern of the
amount of time and effort that students devoted to their studies and other educationally
purposeful activities. The key concept of Engaged Learning can also be applied to
e-learning (Thurmond, Wambach, and Connors, 2002). The disengaged e-learners is a
challenge for the instructors who try to make extensive use of various pedagogies (Hiltz,
2004). If the e-courses had numerous students, the measures of the student online engaged
behaviors became time consuming. Prior studies in e-learning adopt the survey-oriented
approaches (Robinson & Hullinger, 2010), such as exploring students attitudes about
e-learning, overall students satisfaction, and the participative effects on learning outcomes.
It is argued that the most reliable evaluation of educational programs effectiveness is
derived from performance-based measures as a data-driven approach (Kirkpatrick, 1994).
This argument is well supported by the fact that learners' sense of engagement with courses
is more dependent on their connection with learning materials than with the instructor or
peers (Conrad, 2002). In this study, the way of students online learning behaviors counted
and tracked by the LMS (Learning Management System) is applied to be a data-driven
method for e-learning educators.

Purpose and Research Questions


LMS is capable of meeting three pedagogical features: (a) a repository of course materials,
(2) communication facilities, and (3) a platform for communication over the Internet. One
advantage is often ignored by the instructors is the functions of data collection, in terms of
counting, tracking, and recording of students online behaviors. Most of the LMS counts the
data, such as the occurrence of the login, the occurrence of the studying course materials, the
occurrence of the pages read, the time spent in e-learning, and so on. When the dependent
variable is a count variable, the Poisson or negative binomial distributions are commonly

used to represent its distribution. It is in the hope that significant predictors could be
identified by using this new analytic methods. Three research questions were formed: (1)
which engagement data can be useful for prediction; (2) what the differences existed
between the predictors impacts, (3) which a regression model has a good fit of the data.
Research Methods: A Case Study and Regression Model
A course entitled Introduction to Information Management was offered by Taiwan
National Open University Taiwan (TNOU). In spite of attending face-to-face group tutoring
of total 8 hours on four weekends, e-learners studied via an e-learning platform through 18
weeks. Both of the midterm and the final exams in the format of the paper-pencil test were
administered in proctor-based classrooms nationwide.
The Descriptive Data about the E-learners Online Behaviors
There were 443 distance students nationwide enrolled this course. 54% were males. In the
end of the semester, 64 students dropped, accounting for 14% of the drop rate. 15 students
records were excluded as outliers. Table 1 shows a great variability of the students data.
On the average, a student accessed this course for study 17 times, 83 pages, and 13 hours
throughout 18 weeks. The counts of LMS login accumulated the prior and the login of other
e-courses. It is noted that the positive skew and peaked kurtosis failed to represent a normal
distribution (see Table 1 & Figure 1).
Table 1: The descriptive data of the students backgrounds and online behaviors
N=364
Minimum
Maximum
Mean
Std Dev.
Skewness
Kurtosis

Age
18.0
72.0
39.9
9.9
0.3
-0.1

Level of
Midterm
Years
Scores
1.0
8.0
19.0
100.0
3.9
58.8
3.9
16.2
2.3
-0.2
5.0
-0.1

Final
Count of LMS Count of Freq. in
Count of
Scores
Login
Studying
Pages Read
2.0
0
0
0
100.0
1210.0
147.0
634.0
72.1
134.7
17.0
83.8
19.6
176.2
22.0
113.2
-0.6
2.4
2.6
1.9
-0.4
7.0
8.4
4.2

Time(hr) Spent
E-learning
0
186.9
13.3
22.5
2.7
10.9

Figure 1: Frequency histogram of engagement data (LMS login, course study, pages read)

Negative Binomial Model: An Over-Dispersion Poisson Regression


Negative binomial regression is known as a log-linear model, with the dependent variable, a
count variable, has a Poisson distribution. The logarithm of its expected value can be
modeled by a linear combination of the predictors. Over-dispersion is occurred when the
problems of excess zeros in which a subgroup of respondents who would never display the
behavior are included in the sample. In this study, the regression analyses were performed
for the three models, in which each included a different dependent variable as follows: Y1 is
the counts of LMS login, Y2 is the counts of course studying, and Y3 is the counts of pages
read, respectively. Age, gender, level of college years, time spent in e-learning, the midterm
test scores, and the final exam scores served as the predictors.

Results and Discussions


In Table 2, the parameter estimates shows the negative binomial regression coefficients for
each of the predictor variables along with their standard errors. Across the three models, the
predictor variables, the midterm scores, the final scores, and time spent in e-learning, are
statistically significant. It explained that, for instance, each one-unit increase on midterm
scores, the expected log count of the Y1, Y2, Y3 increase by 0.01 times, 0.014 times, and
0.011 times respectively. The midterm scores have a stronger affect (B =0.014) on the
counts of course studying (i.e., Y2) than Y1 and Y3. Each one-unit increase on time spent in
e-learning, the expected logarithm of the pages read (i.e., Y3) increase by 0.044 pages. The
categorical variable Level=1st year has a coefficient of -1.195, which has statistically
significant effect only on Y1. As compared to senior students, the count of LMS login for
the freshman was less. The time spent in e-learning for the male students is decreased by
0.274 hours, less than the female. The LMS login can be well predicted by the factors of age,
midterm, final exam, time spent, and the levels of the college years. The college years may
affect only on the frequency of using the LMS. Test scores and time spent in e-learning can
be well used for predicting the students online engagement.
Table 2: Parameter estimates of negative binomial regression by three dependent variables
Std
Wald
Std
Wald
Std
B of Y3
B of Y2
Error Chi-Square
Error Chi-Square
Error
(Intercept)
1
2.845 0.321
78.42***
0.296 0.352
0.71
1.330 0.334
Gender=male
1
0.007 0.108
0.00
-0.001 0.112
0.00
-0.274 0.109
Gender=female
0
0
0
Level=1st year
1 -1.195 0.150
63.20*** -0.068 0.160
0.18
0.052 0.154
Level=2nd~3rd yr
1 -0.161 0.122
1.74
-0.089 0.131
0.46
-0.047 0.124
Level=4th year
0
0
0
Age
1
0.018 0.006
9.56**
0.010 0.006
2.70
0.023 0.006
Midterm scores
1
0.010 0.004
7.10**
0.014 0.004
13.16*** 0.011 0.004
Final scores
1
0.007 0.003
4.92*
0.009 0.004
6.80**
0.007 0.003
Time spent
1
0.022 0.003
53.25***
0.027 0.003
75.52*** 0.044 0.004
Note: *** stands for-value less than the significance level at 0.001; ** : 0.01; *: 0.05.
Parameters N=364 df B of Y1

Wald
Chi-Square
15.81***
6.38**
0.11
0.14
14.31***
7.47**
4.92*
152.90***

Conclusion
This study has analyzed the data about e-learners engaged behaviors in use of a negative
binomial regression. The data collected by LMS in the case study shows that the effects of
the predictors, such as age, test scores, and time spent on the counts variables. The findings
provide a good fit of the new method and the count data types.

References
[1] Conrad, D. (2002). Engagement, Excitement, Anxiety, and Fear: Learners' Experiences of Starting an
Online Course. American Journal of Distance Education, 16(4), 205-226.
[2] Hiltz S. R. (1994). Learning Together Online. Lawrence Erlbaum Association. New Jersey.
[3] Kirkpatrick, D. (1994). Evaluating Training programs. Berrett Koehler, San Francisco, CA.
[4] Robinson, C. & Hullinger, H. (2010). New Benchmarks in Higher Education: Student Engagement in
online learning. Journal of Education for Business, 12,3, 100-120.
[5] Thurmond, V., Wambach, K., & Connors H. (2002). Evaluation of student satisfaction:
Determining the impact of a Web-based environment by controlling for student characteristics. The
American Journal of Distance Education, 60, 169189.

Vous aimerez peut-être aussi