Académique Documents
Professionnel Documents
Culture Documents
ABSTRACT
One of the key aspects in Outcome-Based Education (OBE) is the assessment of the learning outcomes. The
College of Engineering in UNITEN has adopted OBE for all its bachelor degree programmes since the past 2
years. At the early stage of OBE implementation, the specific learning outcomes or also known as Course
Outcomes (CO) for each course were drawn up based on the Programme Outcome (PO) and other requirements.
COs are the attributes, that the student is expected to have or obtained at the time he or she completing the
course. Each course under the bachelor programme has its own set of COs. At the end of each course, the COs
need to be assessed and evaluated to check whether it has been attained or not. A method to evaluate the
achievement or attainment of the COs has been developed. This paper describes the background of the method,
how the method is used, and the results produced. The method utilises data obtained from students marks in
exams, test, project and other formal assessments. A computerised system has been developed based on this
method to expedite the analysis process. The findings are then used for continuous quality improvement.
KEYWORDS
Attainment, OBE, Learning outcomes, Continuous quality improvement, Assessment
INTRODUCTION
Over the past two years, the College of Engineering at the Universiti Tenaga Nasional
(UNITEN) has adopted OBE approach in its teaching and learning process. OBE
implementation is nowadays compulsory in attaining accreditation from Engineering
Accreditation Council (EAC), Malaysia [1]. EAC has adopted the OBE principles from
American Accreditation Board of Engineering and Technology (ABET), where the learning
outcomes for each course need to be measured and used for continual quality improvement
[2]. OBE focuses on outcomes that are identified and measured. These outcomes attributes,
normally in form of knowledge, skill or attitude, which prepare the graduates for their
professional practice [3]. The outcomes are normally looked at 3 different levels, at course
level (Course Outcomes), at programme level (Programme Outcomes), and at professional
level (Programme Objectives).
This paper describes a method that is used to analyse or evaluate the attainment of specific
learning outcomes which are the Course Outcomes (CO) for a course. CO is the attribute that
the students are expected to have after completing the course. The evaluation of whether CO
is attained is essential in determining the student achieving in a particular course. The result
of CO attainment will also be used to evaluate the attainment of Programme Outcomes (PO).
The outcome of analysis will be used to improve the teaching and learning experience in the
particular course.
1
BACKGROUND
All courses under the bachelor degree programme would have their own course outcomes or
also commonly known as CO. These COs are produced based on the requirement of the
programme outcomes (PO). Each CO will be mapped to PO (the CO-PO) matrix. The PO
will be then mapped to the programme educational objectives (PEO). Figure 1 shows an
example of relationship between CO, PO and PEO.
Figure 2: Block diagram showing the relationship between assessment, delivery and the
outcome in OBE
2
In general, assessment method is a huge topic by itself and can be divided into two major
groups formative assessment and summative assessment [5, 6]. It is often difficult to place
a certain assessment type into either formative or summative. Formative assessment is said to
be part of the delivery or instructional process, where it is used to gather information and
adjust the teaching and learning in real-time [6]. Summative assessment is more common
approach, where the students are given a standardized test or examination at a certain period
of time, such as mid-semester test or final exam, which is also known as formal assessment.
This type of assessment is used to test what the student knows at that particular point of time
and whether he or she has met the course goal or the learning outcome.
In this paper, focus is given to using formal assessment data to evaluate the attainment of
course outcomes. Formal assessment generally refers to the standardised assessment that is
used in evaluating the student for their grade. At present, the common method used in
assessing the student in the department includes final exam (which is normally the highest
percentage), mid semester test, quizzes, assignments, mini projects and lab reports (for labbased courses). The data used for evaluating CO attainment is obtained from the students
mark from these assessments.
CO-ATTAINMENT METHOD
Generally, the approach in evaluating the attainment of CO is using existing data from
students marks, for example from the test results, final exam, quizzes and lab report. These
assessments are referred as formal assessment. The method is also sometimes known as
course embedded measurement. This method is chosen because of the information is readily
available and it is common for most courses. In general, assessment methods used are
grouped into 6 categories: (1) Final exam (2) Tests (3) Quizzes (4) Assignments (5) Project
(6) Special. Each of these categories contributes a certain portion of the marks into some of
the COs. The first 5 categories are self-explanatory, while Special is used when a certain
assessment could not be placed in any other categories, such as oral presentation.
Assessment-CO matrix is produced for each individual course based on these 6 assessment
categories. This matrix shows the weightage distribution of the percentage of marks
distribution for each specified CO. Table 1 shows a general form of assessment-CO matrix.
The number in the matrix shows the amount, in term of percentage, contributes for each CO.
Table 1: An example of Assessment-CO matrix
Assessment Type
Final Exam
Test
Quizzes
Assignments
Project
Special
In the example above, the Final Exam contributes to three outcomes (CO2, CO3 and CO4)
with different percentages. Other types of assessment contribute differently to the outcomes
as the number or percentage shown in the matrix.
Each CO will have its on mark based on the percentage given. The CO-mark is calculated
based on the percentage of marks distribution for each assessments group. Using the above
example:
MarksCO1 = 0% of Final exam (FE) + 30% of Tests (T) + 30% of Quizzes (Q)
+ 10% of Assignments (A) + 0% of Project (P) + 0% of Special (S)
Since it is quite common that each assessment group has different overall percentage, there is
a need to factor in the different weight into the calculation. Table 2 shows an example of
overall percentage distribution.
Table 2: Example of overall percentage distribution
Assessment Final
exam
Overall
50%
percentage
Test Quizzes
15%
5%
Assignment Project
10%
20%
Special
-
TOTAL
100%
Based on the overall percentage distribution, the CO-mark needs to be re-calculated so that
the score is normalised accordingly. Using example in Table 2, the new mark for CO1will be:
MarksCO1 = (0.3T x 0.15) + (0.3Q x 0.05) + (0.1A x 0.1)
Similarly, for CO2:
MarksCO2 = (0.2FE x 0.5) + (0.3T x 0.15) + (0.3Q x 0.05) + (0.2A x 0.1)
Where
FE is the students Final Exam mark
T is the students Test mark
Q is the students Quizzes mark
A is the students Assignment mark
The students marks are tabulated according to the assessment groups. An example of the
mark is shown in Table 3. These marks are then used to calculate the CO-marks.
Table 3: Example of student marks according to each assessment group
Student 1
Student 2
Student 3
Student 4
Student 5
Student 6
Final
exam
77.5
84
67.5
68.5
92
74.5
Test(s)
Quizzes
Assignments
Project/Report
Special*
60.5
92.75
79.75
56.25
72
69
4.83333
9.66667
7.16667
5
9.83333
7.66667
9.75
10
9.875
9.75
10
9.875
12.5
13
12.5
14.5
13.5
12.5
After the calculation, the new CO-mark for each outcome for each student is tabulated in a
new table, as shown in Table 4. In this table, the CO marks for individual student according
to each CO is presented. The user can set a threshold value for CO attainment criteria, for
example 70%. Using the data in Table 4 as an example, it is observed that Student1 has
attained the entire COs except CO1. From this data, the attainment for each CO for this
course can be analysed by counting the number of students attaining that particular CO, or by
calculating the average score.
Table 4: Example of CO-mark result
Student
Student 1
Student 2
Student 3
Student 4
Student 5
Student 6
CO9 CO10
NR 83.33
NR 86.67
NR 83.33
NR 96.67
NR
90
NR 83.33
CO1 CO2 CO3 CO4 CO5 CO6 CO7 CO8 CO9 CO10
NO YES YES YES YES YES YES YES YES YES
48% 70% 74% 83% 97% 97% 97% NR NR 97%
CONCLUSIONS
A method to find CO attainment has been developed and implemented on a computerised
system using Microsoft Excel software. This direct measurement method provides a
structured way to analyse the attainment outcomes for each course. The students marks from
formal assessment such as final exam, test and project are used to calculate the CO-mark for
that particular course. From this result, the attainment of each course outcome for the course
can be further reviewed and analysed. Action plan to improve any weakness can be identified
and implemented in the following semester.
The system has been found to be very helpful in the analysis process. Using this system, the
lecturer could key in the relevant data and obtain the CO attainment for his or her course. The
analysis and result will be automatically produced. This system is continuously being
improved and helpful features are being added from time to time.
REFERENCES
1.
2.
3.
4.
Rozeha A Rashid, Razimah Abdullah, Azami Zaharim, Hamzah Ahmad Ghulman and
Mohd Saidfudin Masodi, Engineering Students Performance Evaluation of Generric
Skills Measurement: ESPEGS Model, 5th WSEAS / IASME International Conference
on Engineering Education (EE'08), July 22-24, 2008
5.
Active Learning for Critical Thinkers, University of Texas at Arlington. Retrieved May
19, 2009, from http://activelearning.uta.edu/FacStaff/formsum.htm
6.