International Journal of Computer Trends and Technology (IJCTT) - volume4 Issue5May 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page 1450

Effect Analysis of a Multi-Material Approach to a
Problem-Solving Learning System

Chun Fu Lin
#1
, Yu Hsin Hung
#2
, and Ray I Chang
#3
*1#3
Department of Engineering Science and Ocean Engineering, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd.,
Taipei 10617, Taiwan (R.O.C.)
#2
Department of Computer Science and Information Engineering, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd.,
Taipei 10617, Taiwan (R.O.C.)


AbstractWith the rapid development of multimedia,
various types of materials used in the design of problem-
solving activities have attracted a considerable amount of
attention from researchers. The adaptive materials
employed in learning activities could enhance learning
performance, and motivate learners to seek the solution to
a problem. Personalizing a learning environment to
optimize individual learning has recently become a
popular trend in e-learning. Therefore, this study was
carried out to explore how the learning materials of a
problem-solving activity influence learning performance;
thus we created a multi-material-based learning system
(MBLS) consisting of animated game-based material and
static text material. The MBLS develops a series of
problem-solving tasks to train the learner how to deal with
risk. Each task was developed based on the problem-
solving model, and the presentation of each tasks scenario
was classified as either static or dynamic. To achieve the
goal of personalized learning, learners could choose the
most suitable material for the problem-solving activity.
Teachers were included in an expert validation test in this
study to examine the performance and reliability of the
learning material in the MBLS. The experimental results
show that, when the multi-materials are employed, the
participants have a 95% probability of obtaining an
above-average score using stratification, which suggests
that the employed multi-material technique can be a good
vehicle for providing adaptive learning that is related to
problem solving.

KeywordsPersonalize learning, multi-material, e-
learning animation, problem solving

I. INTRODUCTION
In recent decades, rapidly developing web technology and
multimedia has been applied widely to teaching and learning
methods [1][2]. The main reason is that the learners interact
with an online learning platform and portfolios can be
dynamically retrieved based on the framework of information
and communication technology (ICT) [3]. As a result,
deciding which kind of learning material to employ in
learning activities has become more interesting than ever, and
learners can adjust the learning material to personalize the
learning activities. Recently, some studies have indicated that
a learning system should provide personalized and adaptive
learning programs to improve learning [4]; the multi-materials
employed in the learning activities are effective for meeting
the requirements of different learners.
The incorporation of web-based learning into dynamic
materials such as games has been applied widely to e-learning
systems. Notably, the game mechanism can improve the
learners learning performance while maintaining the
enjoyable nature of the games [5]. Therefore, problem-solving
activities have been aimed toward the use of the e-learning
platform rather than conventional paperwork. The
dramatically increasing amount of learning materials has also
increased the complexity and difficulty of problem-solving
activities, making it difficult to enhance the learners ability to
find a solution. This study aims to reduce this difficulty and
improve learning performance through undertaking problem-
solving activities; the static text or dynamic tool can be
integrated as a multi-material system which can assist in
adapting the system to learners preferences or needs.
Accordingly, we developed a multi-material-based learning
system (MBLS) for problem-solving activities, in which static
textual presentation, animated game techniques, the problem-
solving model, and inspiring theory are integrated to facilitate
the process of problem-solving activities. Web techniques are
used to enable the learning system to interact with and
successfully attract learners interest while efficiently improve
their learning performance.
II. RELATED WORK
Determining the correlations between learning performance
associated with different materials has engaged the attention
of many researchers [6]-[8]. The learning materials affect
learning performance in different ways. For example, dynamic
materials have a more positive effect on attracting the
learners attention and arousing their interest than static
textual material. The exploration of the effects of different
multimedia materials plays an important role in problem-
solving activities. In particular, game-based learning is
defined as one of the learning materials that aims to improve
International Journal of Computer Trends and Technology (IJCTT) - volume4 Issue5May 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page 1451

learners problem-solving abilities by adopting games in
educational settings [9]. Applying game techniques in learning
activities aims to enhance learning motivation. The related
literature shows that games can be a stimulating motivator for
students of all ages [10]. Creating game-based learning
systems has been considered as an innovative development in
digital learning. Therefore, game techniques are included
frequently as a positive component of individual learning.
Various types of materials applied in problem-solving
activities have attracted researchers attention. Mayer et al.
proposed that visual material could promote problem-solving
transfer and enhance problem-solving comprehension [11][12].
Some studies show that the developed visual style may be
congruent to a certain degree [12]. Animation provides
explicit external representations, thus the cognitive style is the
main difference between animation and static material [12].
For learners, animations are employed to not only provide
learners with compensatory illustration with less pronounced
visual styles [13], but also to enable a mental representation
with a less developed visual style[14]. To summarize,
animation is effective for helping learners to imagine
processes and conduct a mental representation. Accordingly, it
is considered a good tool for problem-solving activities. This
study employed animated material in its game-based problem-
solving activity to impress the learner with its dynamic
display of text and picture.
III. METHODOLOGY
In the MBLS, theories of problem-solving education,
learner behavior theory, web technology, and multimedia
were employed to develop the adaptive learning system. The
architecture of MBLS is illustrated in Fig. 1.

Fig. 1 The architecture of MBLS.
The multi-media materials we designed were classified as
static and dynamic types in the MBLS. The textual description
was integrated as the static material, and the animated
interactive game applied in the MBLS was defined as the
dynamic material. The MBLS includes a series of solution-
inferring tasks concerned with problem-solving ability
training based on the static text-based or animated-based game
material. When learners accomplish the problem-solving task,
the results of the game are considered the learning profile and
are entered into a learning database for data analysis. The
purpose of developing education intelligence is to analyze
learner preferences of the materials and optimize learning
activities. Knowledge management (KM) is employed in the
process of data analysis s. More specifically, the web-based
architecture acts as a medium between the system and the
learner in the MBLS. To enhance the portability of the system,
we programed the MBLS to ensure compatibility with mobile
device software. Learners can receive immediate feedback
even on a mobile device (See Fig. 2), allowing the learner to
obtain real-time information. Accordingly, the MBLS can
personalize service procedures and provide suggestions based
on the inputted information.

Fig. 2 The result of the MBLS executed on a mobile device.

A. System design of the MBLS

Fig. 3 depicts the user interface of the MBLS, which
consists of three parts: static text material, animated game-
based material, and an interacting mechanism, which consists
of a message board and suggestion forum. The MBLS
establishes a multi-environmental analysis including static and
dynamic characteristics.
International Journal of Computer Trends and Technology (IJCTT) - volume4 Issue5May 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page 1452


Fig. 3 The interface of MBLS

The scenarios of the game motivate the learner to use
skills such as observation, logic analysis, mathematic
computation, and associating ability. The problem-solving
model collaborates with the goal of personalized learning for
development of the learning scenario. In the problem-solving
model, the nature of the problem should first be identified.
The learner should then start to plan problem-solving
strategies and choose optimal resources. The problem-solving
process could be monitored by the solver [15]. The MBLS
applied the above procedure of the problem-solving model in
the scenarios, and displayed them using static text material
and animation-based games. Fig. 4 shows the representation
of static text material in the MBLS, the subject of each
scenario has clear instructions with textual descriptions.

Fig. 4 The representation of static text material.

To compare with static text material, we employed the
animated-based game in the MBLS for the same scenarios.
Fig. 5 demonstrates the interface of the animation-based game,
which incorporates the problem-solving model into the game
scenario design. The learner in the gaming process was asked
to observe the conversation in the animation and find the
solution to the problem by observing the gameplay. Learners
can monitor the problem via the board which records the
procedure of the game. Each game was developed based on
four main training subjects: observation, logic analysis,
mathematic computation, and associating ability (See Fig. 6).

Fig. 5 The interface of an animation-based game
International Journal of Computer Trends and Technology (IJCTT) - volume4 Issue5May 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page 1453



Fig. 6 The game scenarios and training items

B. Information interaction mechanism in the MBLS

Iterative learning can enhance learners motivation;
knowledge sharing and appropriate feedback is helpful to
make the learning activity adapt to train the learners problem-
solving skills. Therefore, the message board and contact
function are included in the MBLS. Users such as learners or
teachers can share their experiences, suggestions, even the
suggestions concerned with learning materials, knowledge and
information by interacting on the message board (See Fig. 7).
To increase the efficiency of the learning system, users can
give feedback, make suggestions, or share ideas about the
system by sending an email (See Fig. 8). After the message
has been received by the system, well immediately respond to
the user based on their needs or suggestions for the purpose of
increasing the user stratification and learning intention.

Fig. 7 The interface of the message board in the MBLS


Fig. 8 The function of the contact systemin the MBLS

IV. EXPERIMENT
A. Experimental design

This study aims to develop a multi-material learning system
that includes a series of problem-solving tasks. Accordingly,
the data source of an expert validation test is essential for
examining the performance and reliability of the MBLS. We
collected data from eight teachers whose teaching experience
averaged 4.7 years. The data were saved in the learning
database as preference analysis guidelines. The collected
information is displayed in Table 1. For the purpose of
adapting the analysis results in real cases, the performance
evaluation based on the successful information system model
[15] was included in the research questions. All participants
completed the experiment in approximately half an hour, and
they were paid US$5 dollars to participate.
TABLE I
THE VARIABLES OF THE LEARNER PREFERENCE SURVEY.













International Journal of Computer Trends and Technology (IJCTT) - volume4 Issue5May 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page 1454


B. Analysis of system performance validation by expert test

The integral evaluation included the following six factors:
information, system quality, service quality, intention to use,
user satisfaction, and net benefits, which ranged from
strongly agree (1 point) to strongly disagree (5 points).
Table 2 shows the results of the MBLS performance
evaluation based on the successful information system model,
each dimension obtained a good evaluation. Teachers felt the
proposed approach could assist teachers in teaching problem-
solving activities and improve both the students learning
performance and learning motivation.

TABLE II
The integral performance evaluation
The degree of agreement
Information quality 2.0
System quality 2.1
Service quality 2.0
Intention to use 1.8
User satisfaction 1.8
Net benefits 2.3

In order to explore whether animation-based interaction
game material influences teaching performance or not, we
carried out an in-depth investigation to determine the
correlation between animation-based games and teaching
performance. Fig. 9 demonstrates that animation-based games
improved students problem-solving ability and had a positive
effect on teaching efficiency.


Fig. 9 THE REALATIONSHIP BETWEEN ANIMATION MATERIALS
AND TEACHING EFFICIENCY
The following ten items were included in the
performance evaluation of the MBLS and user cognition:
The animation-based game of the MBLS could stimulate
the ability of problem solving (Q1); The animation-based
material could enhance learning efficiency in the current
task (Q2); The user interface provides a "user-friendly"
experience (Q3); The function of the system is well
defined(Q4); Have intentions to use the MBLS for
problem-solving activities (Q5); Animation-based
material could enhance the ability of problem solving(Q6);
The MBLS enhances learning motivation (Q7); The
MBLS is useful for problem-solving activities(Q8); In
the MBLS, information acquisition and sharing could assist
the performance of teaching (Q9); The incorporation of
IT technology into animated games is effective for
attracting learners attention to the problem solving
activities (Q10). Moreover, we examined the significance
of these items. The response options were scored with a
Likerts five point scale to examine whether the mean of
each item was significantly different from the median. The
results show that each item obtained above-average scores
not only for satisfaction of the system and material designs,
but also to support our view concerned with problem-
solving activities (See Fig. 10). Strong evidence suggests
that the MBLS received high levels of appreciation as a
problem-solving activity from the teachers.
Variables Description Type
ID Identify sample Numerical
Teaching
experience
1 to 8 years (Mean=4.75 years) Numerical
Learning
performance
Each scenario is scored with
0~1point, and the total scores
for each scenario are 4 points
Numerical
Gender Male (51%); Female (49%) Categorical
Learning
material
Static text-based/ Animation-
based game
Categorical
Game
scenario
The subjects of Scenarios1~4
are observation and logic
analysis, associating ability,
logic inference, observation,
logical inference on the
geometric space, mathematic
computation, associating ability
and observation.
Categorical
10 evaluation
items (see
result session
for the items)
5-point Likert-type scale
questionnaires with response
options ranging from strongly
disagree to strongly agree
(type=1~5)
Categorical
International Journal of Computer Trends and Technology (IJCTT) - volume4 Issue5May 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page 1455


Fig. 10 RESULTS OF AVERAGE SCORES FOR EVALUATION ITEMS
C. Analysis of system performance effects on the different
seniority of teachers

In this study, we used a t-test to examine the questionnaire
items separately, based on the different seniority of the
teachers. The seniority was classified into three levels: senior
(8 years), middle (5 years), and junior (1 year). In addition,
the degree of agreement ranked from strongly agree (1 point)
to strongly disagree (5 points), the lower score means the
participants strongly agree with the view we proposed. The
senior teachers gave 2.3 points of satisfaction on the integral
performance. The teachers in the junior (1.8 points) or middle
(1.7 points) level of seniority have higher user satisfaction
than senior. Table 3 illustrates the score of each questionnaire
item from the senior group. The senior teachers significantly
agree (t=3.0 p-value =0.001 <0.05) that the incorporation of
IT technology into animated games is effective for focusing
the learners attention on the problem-solving activity.
TABLE III
THE RESULTS OF THE SENIOR GROUP
The senior level of seniority

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
Max 3 3 3 2 3 3 2 3 3 2
Mean 2.5 2.5 2.5 2 2.5 2.5 2 2.5 2.5 1.5
Min 2 2 2 2 2 2 2 2 2 1
S.D 0.71 0.71 0.71 0.00 0.71 0.71 0.00 0.71 0.71 0.71

Table 4 shows the score of each questionnaire item for the
middle group, which imply that the teachers in the middle
level of seniority have a significantly high level of satisfaction
with the efficiency of the animated game to provoke a
problem-solving ability (t=3.69 p-value =0.005 <0.05) and
enhance learning efficiency (t=3.69 p-value =0.005 <0.05).
Moreover, they significantly agree that animation-based
material could enhance the ability of problem solving (t=7.07
p-value =0.000 <0.05), and feel that the MBLS is helpful
(t=2.45 p-value =0.042 <0.05) and useful (t=4.24 p-value =
0.001 <0.05). The MBLS received good evaluations as a user
interface (t=3.46 p-value =0.007 <0.05), service (t=7.07 p-
value =0.000 <0.05), and for intention of use (t=7.07 p-value
=0.000 <0.05).
TABLE IV
THE RESULTS OF THE MIDDLE GROUP
The middle level of seniority

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
Max 3 3 3 2 2 2 3 3 3 1
Mean 1.75 1.75 2 1.75 1.75 1.75 2 1.5 1.75 1
Min 1 1 1 1 1 1 1 1 1 1
S.D 0.96 0.96 0.82 0.50 0.50 0.50 1.15 1.00 0.96 0.00

The experimental result indicated significant levels of user
interface satisfaction for the junior teachers (See Table 5).
Notably, they strongly agree (t=3.0 p-value =0.001 <0.05)
that the MBLS has well-defined functions and provides a
precise service for the user.
TABLE V
THE RESULTS OF THE JUNIOR GROUP
The junior level of seniority

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
Max 2 1 4 2 1 3 2 3 3 1
Mean 2 1 2.5 1.5 1 2 2 2 2.5 1
Min 2 1 1 1 1 1 2 1 2 1
S.D 0 0.00 2.12 0.71 0.00 1.41 0.00 1.41 0.71 0.00

This supports the claims of the integral performance
analysis and separate evaluation; the MBLS can enhance both
learning and teaching efficiency by providing a multi-material
learning system. Successful personalized learning should be
adaptive to the learners need, and the result of the expert
validation test implied that the MBLS could achieve this goal
of adaptive learning.

V. CONCLUSION
In this study, we developed the MBLS and examined the
system with an expert validation test. The MBLS implements
an animation-based game as dynamic material. The teachers
involved in the process assured the reliability and correctness
of the material content. Our experiment showed that the
MBLS can successfully employ multi-materials in a problem-
solving learning system and the learning activity can be
adapted for personalized learning. The main contributions of
the MBLS are: (1) the evaluation of the teaching experience
related to user satisfaction and collection of related
suggestions on the MBLS from experts. (2) This study also
determined that the animated material is useful for assisting
with teaching.(3) The materials and scenarios have been
integrated based on the problem solving theory. (4) The
International Journal of Computer Trends and Technology (IJCTT) - volume4 Issue5May 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page 1456

contact function of the MBLS uses applied knowledge to
interact with the user (learner/teacher). The system
personalizes the learning activity to improve the efficiency of
learning, and enhances the convenience of human interaction
with the computer. Moreover, based on web techniques, the
MBLS includes the integration of cross-domain knowledge
and the use of real-time information sharing. The findings of
this study are that the personalized material and interaction
mechanism of the system could help learners concentrate on
the problem-solving learning system online. The MBLS
provides immediate feedback to the user, which could assist in
optimizing the learning effect. Future studies should increase
the sample size to investigate more the learning character, and
design the algorithm to adapt the system to the individuals
traits and further personalize the problem-solving activities.
REFERENCES
[1] Y. Lee, Developing an efficient computational method that estimates
the ability of students in a web-based learning environment,
Computers & Education, vol. 58, no.1, pp. 579-589, 2012.
[2] M. Wang, H. J ia, V. Sugumaran, W. Ran, and J . Liao, A web-Based
learning system for software test professionals, IEEE Transactions on
Education, vol. 54, no.2, pp.263-272, 2012.
[3] C. Chen and C. Chen, Mobile formative assessment tool based on data
mining techniques for supporting web-based learning, Computers &
Education, vol. 52, no.1, pp.256-273, 2009.
[4] C. Chen and L. Duh, Personalized web-based tutoring systembased
on fuzzy item response theory, Journal of Expert Systems with
Applications, vol. 34, no.4, pp.2298-2315, 2008.
[5] C. F. Lin, Y. C. Yeh, Y. H Hung, and R. I Chang, Data mining for
providing a personalized learning path in creativity: an application of
decision trees, Computers & Education, In press, 2013.
[6] C. M. Chen, and T. H Lee., Emotion recognition and communication
for reducing second-language speaking anxiety in a web-based one-to-
one synchronous learning environment, British Journal of
Educational Technology, vol. 42, no. 3, pp. 417440, 2011.
[7] E. R. Um, H. Song, and J . Plass, The effect of positive emotions on
multimedia learning, World Conference on Educational Multimedia,
Hypermedia and Telecommunications, Vancouver, Canada , 2007.
[8] G. J . Hwang, P. H Wu, and C. C. Chen, An online game approach for
improving students' learning performance in web-based problem-
solving activities. , vol. 59, no. 4 Computers & Education, 2012.
[9] J . Kirriemuir, and A. McFarlane, Literature review in games and
learning, Bristol, UK: Futurelab, 2004.
[10] A. Amory, Play games to learn: Pre-service teacher development,
The Proceedings of World Conference on Educational Multimedia,
Hypermedia and Telecommunications, pp.2119-2118, 2011.
[11] R. E. Mayer, The Cambridge handbook of multimedia learning,
Cambridge: Cambridge University Press, 2005.
[12] T. N. Hffler, H. Prechtl, and C. Nerdel, The influence of visual
cognitive style when learning from instructional animations and static
pictures, Learning and Individual Differences, vol. 20, no.5, pp.479-
483, 2010.
[13] D. Lewalter, Cognitive strategies for learning from static and dynamic
visuals, Learning and Instruction, vol. 13, no.2, pp.177-189, 2003.
[14] G. Salomon, Interaction of media, cognition and learning, San
Francisco: Jossey Bass, 1979.
[15] R. J . Sternberg, The nature of creativity: Contemporary psychological
perspectives, CUP Archive, 1988.
[16] W. H. Delone, The DeLone and McLean model of information
systems success: a ten-year update, Journal of management
information systems, vol. 19, no. 4, pp. 9-30, 2003.