INTELLIGENT TUTORING SYSTEM USING MULTI-AGENT

RULE-BASED SYSTEM

Tema: Integrasi Dalam Pendidikan

Ahmad Rizwan Romli

Faculty of Information Technology & Quantitative Sciences,

Universiti Teknologi MARA, 40450, Shah Alam, Selangor Darul Ehsan, Malaysia
ahmad_rizwan21@yahoo.com

Norzaidah Md Noh
Faculty of Information Technology & Quantitative Sciences,
Universiti Teknologi MARA, 40450, Shah Alam, Selangor Darul Ehsan, Malaysia
norzaidah@tmsk.uitm.edu.my

Abstract

Intelligent Tutoring System (ITS) is a multi-agent expert system developed specially to give a one-to-one
tutoring to students which simulates the student-teacher learning environment. This system provides
training and tutoring to students by giving notes, examples, exercise, hints and corrections just like how
teachers do in teaching their students. The objective of the system is to teach and tutor the student to
improve the student’s understanding and ability in mathematic similar to the teacher-student one-to-one
tutoring. Student who uses the system will be given notes, examples and exercise. The tutoring method is
different for different type of student. Student who is good in mathematic and will be given notes and
exercise which is up to their level while weak student will be trained until they reached level for the good
student. The system will train student until they are good and passed all the exercise successfully. Once a
student passed all the exercise in the system, the training will end and the student will be classified as
advance in the subject.

Keywords:
Intelligent agent, expert system, artificial intelligent, intelligent system, intelligent tutoring system

1.0 Introduction
Students’ understanding and perception on a certain subject at school depends on how much they are
tutored directly by the teachers. Students especially from the primary school, needs special attention from
the teachers where they need the one-to-one direct tutorial to make them learn faster and understand
better. The more the students given direct tutorial, the better they will understand and the faster they learn
new topics. Involving students in tutorial is an important process. Students need to participate in the
tutorial to understand what they are learning. Tutoring students without students’ participation only could
let the students remember for a short time. In order to store something in their long term memory,
students need to participate in their activity.

The problem with the current educational method is that students are lacking one-to-one tutorial at school.
Only several of the students afford to get personal teacher to come to their house after school to revise
what they have learned in school. Teachers at school often find difficult to satisfy all the students need
because of the limited time and work congestion. When in class, only students who asked or show their
misunderstanding are given attention by the teacher for one-to-one tutorial. The others who stay at the
back listening without ever really manages to understand what they are learning will miss the one-to-one
tutorial. As a result these students are still hanging without answers and miss the boat to proceed to the
next chapter.

In search for the alternative solution for this problem, several tutoring approaches were developed
including the Computer Based Training (CBT) and courseware. This computer software was designed to
provide students with the alternative way of one-to-one tutorial and at the same time improving their
understanding and perception. This solution at first was seen as success as more of this software was
developed to provide education tutorial to the students. Students also accept this method easily as they
prefer to interact with computer which they find it interesting and fun. However the problem for this method
is that the system’s tutoring approach is prefix and that all user’s will get the tutoring approach from the
system. This tutoring approach however is not the optimal solution and does not reflects the real tutoring
between students and teachers at school. The approach does not adapt to students’ capability and that it
assumes all students are the same. There are needs for a system that can really simulate the one-to-one
tutoring between students and teachers and adapts to the students’ capability.
An ITS is a system that not only provide tutoring materials to students, it also adapts to the students
capability. Every student has different capabilities and has different type of understanding. Advance
students, are less dependent to tutoring materials and able to learn new things faster than the weaker
ones. This various type of students’ capability allow the needs of ITS to be initialized.

Therefore, in order to address the issues mentioned above, we aim to:

1. determine the suitability of using rule base intelligent agent for web based intelligent tutoring system.
2. design and develop the intelligent tutoring system.

The project is aim to produce a system that can tutor twelve years old students on subject mathematic
focusing on the topic fractions. The system is able to provide students with notes, examples and
exercises. Students as user of the system will experience one-to-one tutoring and the system will evaluate
their exercises, gives corrections and asses their performance through out the course. The scope of the
system is that the system was design for students who have basic computing skills or those who already
have experience on using computers before. The system also requires the students to give full attention
while on course in order to get efficient results. Any distractions or delay while on course will affect the
result of the system.

2.0 Intelligent Tutoring System

The ITS is a system that provide tutoring approaches to students based on their capabilities. It is designed
using the AI components to first study the students’ capabilities and then come out with tutoring materials
that matches with the students. The system is equipped with expert system where the expert knowledge
came from school teachers who have experience in tutoring students on the particular subject. The expert
system is use to support the intelligent agent to provide the students with matching tutoring materials.

Basically the function of the system is to provide students with one-to-one tutoring approach with tutoring
materials that match with the students’ capabilities. The system is equipped with tutoring materials like
notes, examples and exercises. This materials is use to tutor the students but the students will not get to
see all the materials as they will be shown materials that are only necessary to them. This means that
advance students who are good in the subject and have better understanding will not be shown materials
for the beginners. For example, if a student who uses the system is already good in the subject and
considered to be advance student, the student will be given more difficult exercise rather than basic
exercise for the beginners. Beginner student meanwhile will be given basic exercise, notes and examples
and will be train until the student reaches the level for advance students.

ITS is a multi-agents system where there are 3 different agent working in the background interacting each.
The agents have their specific roles and they provide information for each other. All user activities with the
system will be capture by the agents via the user interface. User interacts with the system by giving inputs
and these inputs are process by the agents to provide feedback to the user. The agents working in the
system are student profile, tutor agent and evaluate agent. The agents in the background are separated to
the user via the user interface. Both user and agents can control what appears on the user interface and
this means that the system and the user are actually communicating each other.

Intelligent Tutoring System

User Interface

Student Evaluation Tutor

Profile Agent Agent

Figure 1: System Design Framework

3.0 Intelligent Agent

The terms "agent" and "intelligent agent" are ambiguous and have been used in two different, but related
senses, which are often confused. Intelligent agent can be viewed from two different angles which are in
computer science and in artificial intelligence. In computer science, intelligent agent is software that
assists users and will act on their behalf, in performing non-repetitive computer-related tasks based on
pre-programming rules and the term ‘intelligent’ means its ability to learn and adapt. Meanwhile in artificial
intelligence, an intelligent agent is used for intelligent actors, which observe and act upon an environment,
to distinguish them from intelligent thinkers isolated from the world. An agent in this sense of the word is
an entity that is capable of perception and action.

There are many definitions and perceptions of intelligent agent but in as much as the definition, intelligent
agents is usually a software-based computer system that contains the properties of autonomy, social
ability, reactivity and proactiveness (Marakas, 2003). The word autonomy means agents operate without
the intervention of human or other agents and have some kind of control over their actions and internal
states. (Jennigns & Wooldrige, 1998). Social ability also known as cooperation refers to ability to interact
with other agents via some communication language (Lin, 2005). In reactivity, agents perceive their
environment and respond in a timely fashion to changes that occur in it (Marakas, 2003). Proactiveness
means agents do not simpy act in response to their environment; they are able to exhibit goal-directed
behavior by taking the initiative (Marakas, 2003). An agent topology was designed base on the properties
of agents which is collaborative agents, collaborative learning agents, interface agents and smart agents
(Lin, 2005).

Intelligent agent can be in the form of single agent system or multiple agent system. Jennings and
Wooldrige, 1998, in their article said that the multi-agent system is where a system is designed and
implemented as several interacting agent. The multi-agent system is to be more complex system than the
single agent system but there are cases that single agent system to be more appropriate.

The system is a multi-agents system where the agents interacts each other as a whole unit. All agents are
important and they receive inputs and provide outputs to each other. Each agent has their specific role
and their task is to provide output for the other agents to process. The intelligent agents were constructed
to process information received from user’s interactivity with the system interface.
3.1 Student Profile

Student agent is an agent for the user. It will capture the information relating to the user from the first time
the user registers until the user finish the using the system. First time user who use the system will have
to register their personal detail into the system. The agent will then use the data and pass the information
to the other agents. During registration, the user need to key in their full name, date of birth, school,
username and password into the system. The agent will pre-determine the new user as a level 1 student
which means beginner. This level is however is not fixed as the user levels may varied depending on their
progress. There three levels of students identified in the system which are level 1 (beginner), level 2
(intermediate) and level 3 (advance). The user may be in any level and their level depends on their
progress in the system.

3.2 Tutor Agent

Tutor agent is the agent that determines the tutoring approach for the user. Each user has different tutor
approach use on them. This agent’s task is to come out with sets of questions which are suitable for the
user based on the user’s level. The agent will use information captured from the student agent to
determine the approach to use to tutor the user. The questions in the question set are not fixed for the
whole process as they will change depending on the user’s progress.

For new user who has registered, they will get fifteen randomly selected questions suitable for level 1
user. The question however are pre-determined questions and does not reflects the user’s exact level.
Once the user starts to answer the questions, the agent will study the user level. After first five questions
answered by the user, the agent will be able to have glimpse view on the user’s level. The agent will then
change the balance ten questions to set of questions that are suitable for the user’s level based on the
first evaluation. This will continue for the next five questions until the user finish all fifteen questions. After
the user complete the first fifteen questions, the agent then can predict thoroughly the exact level for the
user. The user will then again receive another set of fifteen questions that matching the level determined
by the agent.

The next round for the user tutoring process will follow the user level and the agent will provide the user
with more challenging sets of questions in order to increase the user level and capability. The agent will
stop training the user when the user successfully complete all course of questions and reaches the
highest level in the system.

The rules for determining the student tutoring approach are described below:
 Finish 5 questions

Yes
Complete all
within 100 Set user level to level 3
seconds

No

Yes
Complete all
in between
100 to 200 Set user level to level 2
seconds

No

Set user level to level 1 Get remaining questions

based on level set

After another 5 questions

Figure 2: First 5 questions in first round

To ensure student’s consistency, the system will check if the user scores minimum 90% for all 3 levels
before the user can finish the course. If not, the user will have to another round of 15 questions for the
questions level which the user did not manage to score minimum 90%.

3.3 Evaluate Agent

Evaluate agent job is to evaluate the user progress throughout the tutoring process. Every time the user
answers the questions, the evaluating agent captures the data and evaluates the user’s performance. The
agent will look into student’s weakness and strength and provide information for the tutor agent to
determine the set of questions to give to the user. Evaluating agent tasks also are to correct the user’s
mistakes and make sure that the user knows where their mistakes are. The agent makes sure that the
user answers their questions correctly before proceeding to the next question. Besides that the agent also
record the time taken for the user to answer each question and sum up all the time taken to complete all
the questions.

The agent will provide information like total time taken and user’s strength and weakness to the tutor
agent for further process. User who failed to answer a question correctly will have to do correction and the
agent will provide them with hints or notes for the user reference. All user activities in answering the
questions will be recorded by the agent to provide useful information to the tutor agent.

For every questions the user answer, the user will have to go through the evaluation process. Evaluation
process is as below:
 User answers question

Move to next Marking

question Correct

Wrong

Do corrections

Figure 3: Evaluating process

Time taken is counted by the total time to pass 1 question.

For example if the user correctly in the first attempt in 20 seconds, the time taken that will be recorded in
the student model is 20.
If the user answers wrong in the first attempt and correct in second attempt, the total time taken is time
taken in first attempt plus time taken in the second attempt.
st
1 attempt : 25 seconds
nd
2 attempt (correction) : 20 seconds
Total time taken : 45 seconds

Agent will then calculate the percentage of correctness for each level of questions based on the student
model.
If the user scores at least 90% for each question level, the user is considered as advance student.
If not the student need more practices on the questions for the specific level.

Correctness percentage is counted by:

Total correct answers for specific level divide by total of questions answered for specific level multiply by
100.
Example:
Question level 3 answered: 25
Correctly answered: 23
23/25 * 100 = 92%

3.4 Expert System

Expert system is the system which simulates the human expert. In the case of this project, school
teachers have been identified by the researcher as the expert domain. The expert system consists of
knowledge from the school teachers in tutoring students and evaluating their capabilities. The knowledge
gained from the system are then stored in knowledge bank and rules were made to support the
knowledge. The expert systems were used in the agent to help the agent to process the data gathered
from the user activities.
In the tutor agent, the expert system supports the agent in terms of what approach to use in tutoring
different type of user level. The expert system diagnoses the student’s progress and determines the level
of the user. It will then provide information to the tutor agent to construct the questions to be given to the
user for further tutoring.

Knowledge for expert system were obtain from two school teachers that have experience in teaching
standard six school students.

Start

Database
Registration Student Profile

Read notes

Tutor Agent
Do exercise

Correction Fail
Evaluate

Pass

Evaluation Agent
Level adjustment

No
Completed all
exercises

Yes

Finish

Figure 6: Flow chart of the ITS

Figure 6 shows the process flow chart of the system. The system consists of 3 agents working together.
Once registered into the system, user has the options to read notes and examples. This activity however
is not recorded by the agent and the user a free to navigate through all the contents. When the user
finishes reading the notes and examples, the user will then go through the exercise for tutorial process.
Activity in this process will be capture by the agent and user evaluation will depend how well the user
perform. The process will end after the user completed all three levels of questions within the time range
given with minimum 90% score in each level. This process will iterate for several times depending on the
user’s capability. Once the process is completed, the result will be showing the summary of the user’s
performance.
4.0 Analysis
The system result is the output from the system after going through the process of tutoring the user. User
rd rd
who uses the system will finish the course only after they reach the 3 level and completed the 3 level
questions with 90% correctness in each level. After the user finished the course, the system will provide
output that summarizes the progress of the user while using the system. The output then can be use to
study the pattern of the user who use the system.

In order to get the accurate result, the system needs to be tested by real user. The researcher made two
type of testing for the system. First is by using simulation to test for the system accuracy and second
using the real standard six students to be the user of the system. This allows the researcher to make
comparisons between the output gained from the simulation and real students. From the comparisons, the
researcher can analyze whether the system has the system accuracy in tutoring and evaluating students.
High accuracy is important to prove that the system is ready to be use in the education sector.

Further analysis can be described from the table below:

Table 1: Result from real student

Total Progress
User No of
Level Time Level 1 Level 2 Level 3
ID rounds
(minutes) Q C % Q C % Q C %
11 3 40 10 10 100 35 35 100 30 30 100 5
12 3 39 11 10 90.1 37 35 94.6 31 30 96.7 5
13 3 26 5 5 100 20 18 90 35 33 91.4 4
17 3 65 30 29 96.7 45 41 91.1 45 41 91.1 8
18 3 35 10 9 90 30 28 93.3 35 32 91.4 5
19 3 41 15 14 93.3 30 28 93.3 30 27 90 5
21 3 44 30 30 100 15 14 93.3 45 42 93.3 6
22 3 75 30 27 90 30 28 93.3 45 41 91.1 7
23 3 50 30 27 90 30 29 96.7 30 28 93.3 6
25 3 60 35 35 100 40 39 97.5 30 29 96.7 7
26 3 43 20 19 95 25 23 92 30 27 90 5
27 3 49 25 23 92 20 18 90 30 28 93.3 5
30 3 48 16 15 93.75 30 27 90 45 41 91.1 6
31 3 53 20 18 90 40 36 90 33 30 90.1 6
32 3 49 20 18 90 33 30 90.1 49 45 91.8 6
33 3 58 50 45 90 31 30 96.8 33 30 90.1 7
34 3 42 15 15 100 32 30 93.75 33 30 90.1 5
35 3 40 16 15 93.75 33 30 90.1 33 30 90.1 5
36 3 27 25 25 100 21 20 95.2 16 15 93.75 4
37 3 41 30 30 100 20 18 90 33 30 90.1 5
38 3 80 31 30 96.8 50 45 90 47 45 95.7 8
40 3 43 20 18 90 33 30 90.1 32 30 93.75 5
41 3 40 31 30 96.9 20 18 90 32 30 93.75 5
42 3 55 15 15 100 48 45 93.75 30 30 100 6
43 3 50 21 20 95.2 27 25 92.6 48 45 93.75 6
45 3 52 30 30 100 31 30 96.7 32 30 93.75 6

Legend:
Q: Question answered C: Correct answers %: Percentage

Table above shows the result of real students. All students are randomly picked from different school and
area but from the same domain which is 12 years old standard six students. The student were given short
brief about how the system works and given pre-test to get use with the system. When the testing starts,
the students were required to give full concentration to the system and not allowed to do other things until
they finish. This is to ensure that the accuracy of the result. Result from the test done by the students
showed that all of them are able to finish the course within certain time period. The table shows the User
ID, user level, total time taken to finish the course, progress of the students in each level and numbers of
rounds required completing the course.

Results from the test showed that students who are good in mathematic are able to finish the course
within 4 to 6 rounds in approximately 40 minutes. The students were actually learning and improving their
ability throughout the sessions. Some of them who made mistakes in the first few rounds were able to see
their weaknesses and improved for the later questions. There also students who manage to solve lower
level questions in short time but face difficulty as they move to harder questions. Although, these students
proved that they are able to maintain their consistency in answering questions. Consistency is the key to
the system where the students need to reduce their mistakes while answering as fast as they could.
Students who are a little bit slow requires up to 8 rounds of exercises to complete the course. Although
the students find it hard to complete the course, the longer the students use the system, the more the
students improve their understanding. As they learned from their mistakes, they also able to improve their
consistency thus reduce their mistakes. More questions given to the students help them to relate the
previous questions they get with the new one. They also made improvement in terms of answering
technique where the students learned to answer questions correctly in shorter time. The results also
showed that students who finish the course are able to maintain their consistency in answering the
questions. The requirement of 90% correctness in each level requires the students not to afford to make
too many mistakes thus improving their consistency in answering correct answers.

Results from the test proved that the system is capable to provide one-to-one tutoring to students and
adapt to the different level of students. Students are given exercised base on their level and they are
tutored until they are able to do more difficult questions. Generally, students who finish the course made
improvement in terms of understanding, perception, consistency and time management.

Result from the test can be justified further if more tests on real students are made. More results from real
student can help to see more pattern of how the students response using the system. So far the results
shown already prove that the system met its objective but with more real student data, more information
can be gather.

References
Abushar, S., & Hirata, N. (N.D.). Filtering with Intelligent Software Agents. Retrieved October 1, 2007 from
http://www.engin.umd.umich.edu/CIS/course.des/cis479/projects/FISA.html
Ali, S. S., & McRoy, S. (2000). What is an Intelligent Tutoring System? Retrieved September 1, 2007 from
http://www.cs.niu.edu/~freedman/papers/link2000.pdf
Beal, C. R. (N.D.) Intelligent tutoring for math achievement testing: A controlled evaluation. Retrieved
September 1, 2007 from http://wayangoutpost.net/paper/WayangEvaluation.pdf
Beal, C. R., & Lee, H. (N.D.) Creating a pedagogical model that uses student self reports of motivation
and mood to adapt ITS instruction. Retrieved September 1, 2007 from
http://k12.usc.edu/papers/Beal&LeeCRC.pdf
Fuchs J., Heller I., Topilsky M., & Inbar M. (1999). CaDet, a computer-based clinical decision support
system for early cancer detection. Cancer Detection Prev. PubMed. Beeri Community Health
Center, Tel Aviv, Israel. Pages 78-87.
th
Gauthier G., Frasson, C., VanLehn, Kurt., M. (Eds). (2000). Intelligent Tutoring Systems: 5 International
Conference; proceedings / ITS 2000, Montreal, Canada, June 19-23, 2000. Springer-Verlag. New
York, United States.
Hafner, K. (2004). Software Tutors Offer Help and Customized Hints. New York Times. New York, USA.
Hayes-Roth, B., & Larsson, J. E., (1998). Guardian: An Intelligent Autonomous Agent for Medical
Monitoring and Diagnosis. IEEE Intelligent Systems. IEEE Educational Activities Department. New
Jersey. USA. Pages 58-64
Jennings, N. R., & Wooldridge, M. (1998). Application of Intelligent Agents. Queen Mary & Westfield
College, University of London. Retrieved October 1, 2007 from
http://agents.umbc.edu/introduction/jennings98.pdf
Jeremic, Z., Devedzic, V. (2004). Design Pattern ITS: Student Model Implementation. In Fourth IEEE
International Conference on Advance Learning Technologies (ICALT ’04). Retrieved September 1,
2007 from
http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/proceedings/&toc=comp/proc
eedings/icalt/2004/2181/00/2181toc.xml&DOI=10.1109/ICALT.2004.1357693
Lajoie, P. S., & Vivet, M. (2002). Artificial Intelligence in Education : Open Learning Environments . New
Computational Technologies to Support Learning, Exloration and Collaboration. IOS Press. The
Netherlands
Kiong, S.W., Rahman, A. L. A., Zaiyadi, M. F., & Aziz, A. A. (2005). Expert System in Real World
Applications. Generation 5. Retrieved October 5, 2007 from
http://www.generation5.org/content/2005/Expert_System.asp
Ljungberg, M., & Lucas, A. (1992). The OASIS Air Management System. In Proceedings of the Second
Pacific Rim International Conference on Artificial Intelligence (PRICAI '92). Seoul, Korea.
Retrieved October 1, 2007 from http://citeseer.ist.psu.edu/ljungberg92oasis.html
Limoanco, T., & Sison, R. (2002). Use of Learner Agents as Student Modeling System and Learning
Companion. In ICCE Proceedings of the International Conference on Computers in Education.
IEEE Computer Society. Washington, DC, USA. Page 1331
Lin, F.O. (2005). Designing Distributed Learning Enviroments with Intelligent Software Agents. Information
Science Publishing. United States. pages 162-180.
st nd
Marakas, G. M. (2003). Decision Support Systems in The 21 Century 2 edition. Prentice Hall. New
Jersey, USA. Pages 530-570.
Mazza, R. (2003). Using Open Student Models in Distance Learning Environments to Help Teachers
Provide Adaptive Tutoring. In SCOMS- Studies in Communication Sciences, Special Issue New
Media in Education. Retrieved September 1, 2007 from
http://www.inf.unisi.ch/assistants/mazza/Web_area/Pubblicazioni/SCOMS2003/scoms_feb2003v2
.pdf
nd
Negnevitsky, M. (2005). Artificial Intelligence: A Guide to Intelligent System 2 edition. Addison-Wesley.
England. pages 25-54.
Ong, J., & Ramachandran, S. (N.D.) Intelligent Tutoring Systems: Using AI to Improve Training
Performance and ROI. Retrieved September 1, 2007 from
http://www.stottlerhenke.com/papers/ITS_using_AI_to_improve_training_performance_and_ROI.p
df
Russel, S., & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Retrieved September 1, 2007
from http://www.cs.berkeley.edu/~russell/aima1e/chapter02.pdf
Thaw, T. H., & Phon-Amnuiasuk, S. (N.D.) Intelligent Tutoring System for Mathematical Problems:
Explanation Generations for Integration Problem (EGIP). Retrieved September 1, 2007 from
http://pesona.mmu.edu.my/~somnuk/papers/EGIP.pdf