Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia

DEVELOPMENT OF AN AUTOMATED ASSESSMENT

FOR C PROGRAMMING EXERCISES
USING PSEUDOCODES COMPARISON TECHNIQUE

Khirulnizam Abd Rahman1, Md Jan Nordin2,

Che Wan Shamsul Bahri Che Wan Ahmad3, Juzlinda Mohd Ghazali4
1,3,4
Selangor International Islamic University College, Bangi, Selangor, Malaysia
2
Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia
khirulnizam@kuis.edu.my1, jan@ftsm.ukm.my2,
cwshamsul@ kuis.edu.my 3, juzlinda@ kuis.edu.my 4

ABSTRACT Programming cannot be learnt without doing a lot of

Doing programming exercises is an important
activity in learning any programming language. In
order to acquire the skill to program, the more
programming exercises are done, the better.
However, this only means more workload for the
instructor to check and grade all the exercises that
are submitted. The workload can be reduced if the
instructor is provided with a tool that can
automatically assess and grade the exercises.
Automated programming assessment is a method to
evaluate and grade students programming exercises,
without the hassle of doing it manually. Thus, this
application is being developed to assess students C
programming exercises. The assessment is based on
the pseudocodes, generated from the students
programming exercise, as compared to that of the
instructor’s. The result is in the form of similarity
percentage. From this research, a computer
programming instructor could assess the student’s
exercises more efficiently.
Keywords: programming automated assessment,
pseudocode generator, pseudocode comparison,
static analysis, web-based application.
1. Introduction
Programming is a subject where the students need to
deal with the syntax and limitation of a programming
language and at the same time trying to mold their
knowledge to develop a process that could solve their
problems. It is a multi-layered hierarchy of skills,
which need to be activated at the same time [2]. In
order to gain the knowledge and to become a skillful
programmer, students need to spend a lot of time to
practice and develop programs on their own.
practice [11].
In order to acquire the skill to program, the more
programming exercises are done, the better.
However, this only means more workload for the
instructor to check and grade all the exercises that are
submitted. The workload can be reduced if the
instructor is provided with a tool that can
automatically assess and grades the exercises. That is
why instructors need a system to manage all the
exercises submitted, and grades them all
automatically. This is where automated programming
assessment becomes handy.
The objective of this software being developed is to
assist the programming instructors in checking and
grading their student’s programming exercises. It
could save times and provide assessment without
bias.
2. Background: automated programming
assessment using static analysis approach
Automated programming assessment is a method to
evaluate and grade students programming exercises,
without the hassle of doing it manually.
This field has been explored since 1960 by
Hollingsworth [5]. In his research, he developed
automated programming assessment to mark
assembly language codes on punched cards. Since
then, many automated programming assessment
system were developed to support other languages
such as C, C++, FORTRAN, Pascal, Java and many
more [1].

119
Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia
For whatever languages the systems could support,
there are two identified approaches in order to test or
mark the programming assignments. The approaches
that normally implemented are static or dynamic, or
sometimes both [13].
Static analysis involves the examination of program
source code without execution. The program source
code structure and syntax are inspected so as to
highlight static errors and produce statistical
information for the programmer [3]. This approach is
normally used for activities intended to pick up other
type of errors or potential error conditions, such as
infinite loops, unreachable statements, conflicting
conditions, improperly nested loops and unused
variables [13].
Dynamic testing involves executing the program by
using test data. The main aim of dynamic testing is to
uncover execution errors in a program. Usually the
output produced from the execution will be analyzed
to determine the program correctness [13].
Many researches have been done on automated
programming assessment. For example Ceilidh [13],
ASAP [4] and BOSS [7] are systems that have been
developed for many years. They are developed by
University of Nottingham, Kinston University and
University of Warwick respectively. In Malaysia
there are few studies in this area such as JAP [9] and
WAGS [12]. However, most of the researches were
focusing on dynamic analysis approach except for
WAGS.
WAGS [12] is an automated grading system that rely
only on static analysis approach. This is a product of
Universiti Teknologi PETRONAS, Malaysia. This
system could handle various programming languages
such as Visual Basic, C and Java. It focuses on
checking the source code of the submitted
assignments. The system will compare assignments
submitted by the students with the solutions provided
by the instructor.
There are one problem that has been identified with
WAGS; that is the variable declarations. The variable
declarations should be extracted and isolated from the
main source code. These variable declarations should
be analyzed using different analysis procedure. If that
is the case, the comparison precision could be
increased.
120
The authors chose to develop a system based on the
pseudocode comparison, which is a static analysis
approach. The motivation behind this is to provide
flexibility in student’s answers. In programming
problem solving, students may solve an assignment
using different statements or different algorithms.
Hence, this will not restrict the student’s creativity.
The pseudocode is a general format of the source
code. By converting the source code into
pseudocodes can provide easier upgradeability. This
is because the pseudocodes comparison module could
be used to assess source codes from other languages.
What we need to do is to developed different
pseudocodes converter to cater other language.
3. The application
There are three modules involved in the process; the
first is lexical analysis, followed by pseudocodes
generation, and then pseudocodes comparison.
Student’s
source code
Lexical analysis
Transformed into
pseudocode
Compare pseudocode
with the solution
models provided
The mark
Figure 1: The main modules in the application.
3.1 Tokenization
This is the process to translate the source code into
smaller chunk of symbols called token. Token is the
smallest portion of source code indivisible anymore.
Examples of tokens are variables, keywords,
arithmetic and logical symbols, lateral values and
Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia

separators [6]. Please refer appendix A for the The pseudocodes translation:
complete specification of the lexical syntax in START
Backus-Naur form. The source code will be declare i as integer
tokenized and as a result, a sequential list of tokens i=1
declare salary as float
will be produced.
declare tax as float
DO
display the value of i
3.2 Pseudocode generation display "Key in the salary\n"
get user data for variable
Next is the process of transforming the program into ,salary
pseudocodes. The list produced by the tokenization IF (salary < 1000)
calculate tax = salary*0.04
process will be used as the input of this process. This
ELSE
conversion into a general form will simplify the calculate tax = salary*0.08
comparison procedure, which is the last module. ENDIF
display the value of salary
A pseudocode standard suggested by Robertson, [8] display the value of tax
is chosen since there is no formal standard for i=+1
pseudocode. Translation is done by converting the WHILE i < 5
source code statements into the pseudocodes display "TQ"
END
language specification. For example if the generator
find int in as the first word in a statement, then the
generator will presume the statement is a variable
declaration of integer data type. Appendix B shows a 3.3 Comparing pseudocodes
table of the statements and the translation to the
pseudocodes. The student’s pseudocode is then compared to the
instructor’s pseudocodes. Assessment is done in two
This is an example of a source code, translated into separate categories. The first is the comparison of the
the pseudocode. number and types of variables, and the second is the
algorithm similarities between the two pseudocodes.
The code example: To handle various acceptable solutions with different
#include<stdio.h> algorithms, the instructor may provide more than one
#include<conio.h> solution model. The students answer will be
main(){ compared with all the solutions provided. The highest
int i=1; similarity percentage will be chosen for the mark.
float salary,tax;
The first step in this process is to segregate the part of
do{
variable declarations. Since the variable declarations
printf("Staff no : %d\n",i);
printf("Key in the salary\n"); could be anywhere in the student’s answer,
scanf("%f",&salary); separating them could increase the pseudocode
similarity level with the pseudocodes from the
if(salary<=1000){ solution models. Pseudocodes without the variable
tax=salary*0.04; declaration will then compared to the solution models
} provided. To handle various acceptable answers with
else{ different algorithms, the instructor may provide more
tax=salary*0.08;
}
than one solution models. The student’s solution will
printf("The salary is: RM.2%f\n", be compared with all the solution models provided.
salary); The highest similarity percentage will be chosen for
printf("Your tax is : the mark.
RM.2%f\n",tax);
i++; The comparison approach used in this application is a
}while (i<=5); non-structural similarity comparison as suggested by
Norshuhaini [2006]. The algorithm used for the
printf("TQ");
getch();
comparison is a string similarity algorithm that
} produces the similarity percentage between two
strings.

121
Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia

Student’s
pseudocode

Pseudocode Pseudocode Similarity

comparison scheme 1 percentage 1

Pseudocode Similarity
scheme 2 percentage 2

Pseudocode Similarity
scheme n percentage n

The highest
percentage

Figure 2: The algorithm of the pseudocodes comparison.

4. Scopes and limitations Web environment is chosen to facilitate users who

This application is initially being developed for basic
C programming exercise. Hence, the statements
catered only for variable declarations, assignment
statements, mathematical expressions, Boolean
expressions, compound statements, selections
(if…else, switch…case) and repetitions (do…while,
do and for).
The application only capable of checking source code
without syntax error. Users should be aware of this
limitation and make sure their source codes are error-
free.
5. Tools
This application is developed using HTML as the
front-end (user interface), PHP is the middle-ware
and Apache as the web server. MySQL is planned to
be used as the database server. However the
management system has not been developed yet.
access the system from outside campus.
6. Conclusions and future works
Generally, this paper provides report on the initial
research to develop another automated assessment
system through comparing the program’s
pseudocode. Although the static analysis approach is
chosen, but the study is focusing more on comparing
the pseudocode, which is quite new.
The authors are still doing the experiment on how to
compare the pseudocodes. There are several
algorithms to consider. During the writing of this
report, the algorithm implemented is by calculating
the similarity between pseudocodes character by
character. This is not the most efficient way on how
to do the comparison. Thus, a new algorithm should
be developed and implemented in the system to
enhance the precision of the comparison. Since the
application is still in development stage, more works
must be done.

122
Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia

6. References [8] Robertson, L. A., Simple Program Design: A

[1] Ala-Mutka K., A Survey of Automated
Assessment Approaches for Programming
Assignments. Journal of Computer Science
Education, Vol 15,pp 83-102, June 2005.
[2] Bloom B. S., et al., Taxonomy of Educational
Objectives: The Classification of Educational
Goals, Handbook I: Cognitive Domain New
York: Longmans, Green, 1956.
[3] Coleman M., Pratt S. , Software Engineering
for Students, Chartwell-Bratt Ltd, 1986.
[4] Douce C., et al., A Technical Perspective on
ASAP : Automated System for Assessment of
Programming, retrieved online from
http://dircweb.king.ac.uk/papers/Douce_C.200
5_316221/ASAP%20CAA%20PaperFINAL.p
df on January 2007.
[5] Hollingsworth J., Automatic Graders For
Programming Classes. Communication of
ACM Vol 3, Issue 10, 1960, pg 528-529.
[6] Holub A. I. , Compiler Design in C, Prentice
Hall, USA, 1990.
[7] Joy M & Luck M., The BOSS System for On-
line Submission and Assessment, retrieved
online from
http://www.ulster.ac.uk/cticomp/joy.html on
January 2007.
Step-by-Step Approach, Boston: Thomson,
Course Technology, 2004.
[9] Romli R., et al., Automatic Correctness
Assessment Program for Java Programming
Assignment. Proceedings of M2USIC 2004,
Malaysia, 2004.
[10] Shaffer S. C., Ludwig : An Online
Programming Tutoring and Assessment
System, ACM SIGCSE Bulletin, Vol 37, Issue
2, Jun 2005. USA:NY,2005.
[11] Truong N., ELP – A Web Environment For
Learning To Program, ACM SIGCSE
Bulletin, Vol 37, Issue 3, September 2005,
USA:NY, 2005.
[12] Zamin N., et al., WAGS: A Web-Based
Automated Grading System For Programming
Assignments From Users’ Perspectives.
Proceedings of International Conference on
Science & Technology: Application in
Industry & Education, Malaysia, 2006.
[13] Zin M. A. & Foxley E . Automatic Program
Assessment System. Retrieved online from
http://www.cs.nott.ac.uk/~ceilidh/papers/ASQ
A.html on January 2007.

123
Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia

Appendix A : The complete specification of the lexical syntax in Backus-Naur form.

InputElement: Whitespace | Comment | Token

Whitespace: \b|\t|\r|\n|\f|\r\n

Comment:
LineComment|BlockComment
LineComment: // any string ended by \r or \n or \r\n
BlockComment: /* any string */

Token: Identifier | Keyword | Literal | Operator | Separator

Identifier: Letter(Letter | Digit)*

Literal: Number | Character | String
Number: Integer | Real
Integer: -?Digit+
Real: -?Digit+(\.Digit+)?
Character: ‘any character’
String: “any string ended with‘”’

Letter: [A,…,Z,a,…,z]
Digit: [0,…,9]

Keyword: asm | auto | break | case | char | const

| continue | default | do | double | else
| enum | extern | float | for | goto | if
| int | long | register | return | short
| signed | sizeof | static | struct
| switch | typedef | union | unsigned
| void | volatile | while | main | printf | scanf

Operator: ArithmetikOperator | LogicalOperator | RelationalOperator

| AssignmentOperator |DecrementOperator
| IncrementOperator | DataTypeRepsOperator

ArithmetikOperator: ArithopMult|ArithopDiv|ArithopMod
|ArithopAdd|ArithopSub
ArithopMult: *
ArithopDiv: /
ArithopMod: %
ArithopAdd: +
ArithopSub: -

LogicalOperator: LogicopNot|LogicopAnd|LogicopOr
LogicopNot: !
LogicopAnd: &&
LogicopOr: ||

RelationalOperator: RelopEequal| RelopNotEqual| RelopLessthan

|RelopGreaterthan |Relopltorequal | Relopgtorequal
RelopEqual: ==
RelopNotEqual: !=
RelopLessthan: <

124
Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia

RelopGreaterthan: >
Relopltorequal: <=
Relopgtorequal: >=

AssignmentOperator: Assop|AssopMult|AssopDiv|AssopMod|AssopAdd| AssopSub

| DecrementOperator| IncrementOperator
Assop: =
AssopMult: *=
AssopDiv: /=
AssopMod: %=
AssopAdd: +=
AssopSub: -=
DecrementOperator: --
IncrementOperator: ++

Separator: Sepcoma|SepEOS|OpenStmtBlock|CloseStmtBlock
|OpenArrBound|CloseArrBound|OpenParenthesis
|CloseParenthesis
Sepcoma: ,
SepEOS: ;
OpenStmtBlock: {
CloseStmtBlock: }
OpenArrBound: [
CloseArrBound: ]
OpenParenthesis: (
CloseParenthesis: )

DataTypeRepsOperator: DTROInteger|DTRODecimal|DTROChar| DTROString

DTROInteger: %d
DTRODecimal: %f
DTROChar: %c
DTROString: %s

125
Conference on Information Technology Research and Applications 2007 (CITRA 2007), 4th – 5th April 2007, Selangor, Malaysia

Appendix B: The summary of source code translated to pseudocodes.

Statement type Example Translation

Variable declaration int num1; DECLARE num1 as integer.
Multiple variable int num1, num2, num3; DECLARE num1 as integer.
declaration DECLARE num2 as integer.
DECLARE num3 as integer.
Declaration with int num1=3; DECLARE num1 as integer.
initilization Set num1 with 3
printf()
scanf()
If statement if(conditions){ IF conditions
… …
}else if(conditions){ ELSEIF conditions
… …
}else{ ELSE
… …
}

Switch…case switch(…){ IF conditions

statement case 1: …
… ELSEIF conditions
case 2: …
… ELSE
case n: …
…
default:
…
}
Do…while do { DO
statement … …
}while (conditions); WHILE conditions
While statement while(conditions){ WHILE conditions
… …
}
For statement for(statemet1;statement2; statement1
statement3){ DO
… …
} statement3
WHILE statement2
Comparison == Equals
operators != Not equals
<, >, <=, >= *Maintained
Logical operators && AND
|| OR
! NOT
Mathematical *All operators are
expression maintained
Compound A++; A=A+1
statements B--; B=B+1

126