Académique Documents
Professionnel Documents
Culture Documents
1, 45–59 45
© 2018 Vilnius University
DOI: 10.15388/infedu.2018.03
1. Introduction
New software engineers and casual developers are needed in many different areas. How-
ever, students face many difficulties while learning the logic of computer programming,
frequently failing in university courses.
*
Corresponding author: ednacanedo@unb.br; tel.: +55-61-98114-0478
46 E.D. Canedo, G.A. Santos, L.L. Leite
2. Related Work
There are several studies to understand the difficulties faced by computer programming
learners. It is known that in the academia there is an intense effort by researchers and
professors to try to comprehend the reason behind the difficulty in learning program-
ming concepts. This difficulty of comprehension is an obstacle for an ever higher num-
ber of students, as presented by (Jenkins, 2002) and (Lahtinen et al., 2005). Not only
that, there is an effort from the leaders of Teaching Institutions and the Government to
provide better learning conditions throughout the undergraduate course. Moreover, there
has been much debate among computer science professors as to which programming
An Assessment of the Teaching-Learning Methodologies Used in the ... 47
ing the infrastructure of Universities, Federal Institutes, and Federal and Local Public
Schools, improving the technology and equipment of the classrooms and laboratories,
along with several programs to encourage teacher qualification. Even with the amount of
programs and research, there are still problems in learning programming.
At the UnB, there’s an agreement between professors and researchers involved with the
Intro to Programming subject that learning to program is not a trivial activity, since it
introduces a series of cognitive requirements to the daily life of the student, going further
than technical requirements.
In their majority, these cognitive requirements incorporate a need for the student to
change their way of thinking and acting in their academic life to a different reality than
the one they got used to during high school.
If we consider that they’re met with this different way of thinking in the first semester
of their course, changes need to be incorporated/absorbed in a short period of time, since
the subject lasts for a semester. The subject load is of 60 hours of class that usually begin
in March and finish in the end of June, or begin in August and finish in the beginning of
December of each year.
The cognitive requirements pointed out by (Wilson and Shrock, 2001) and (Wieden-
beck, 2005) are:
1) The resolution of problems is a competency that involves cognitive processes
such as creativity and rationality, through a set of mental meta-skills that some-
times go unnoticed, and which are supported by other skills, such as reading and
interpreting the description of a problem.
2) The full understanding of the requirements of a programming paradigm is not a
trivial activity, and entails an inherent degree of difficulty. This understanding
involves abstraction and problem-solving skills.
A majority of professors of programming subjects try to make the student understand
that to program is, first and foremost, an exercise of basic reasoning skills (reading, writ-
ing, and calculating) and mental skills (comparing, describing, interpreting, classifying,
and analyzing) that develop over constant practice and exercise. It is necessary to put
in a considerable amount of time to pass a programming subject, especially in the first
semesters of an undergraduate course.
Teaching evaluation is described as the process whereby the quality of teaching is as-
sessed. This measure of assessment can be conducted using formative and/or summa-
tive approaches. In higher education, formative assessments of teaching are focused on
providing instructors/professors with information that can help them to improve their
teaching, as presented by (Pitterson et al., 2016).
An Assessment of the Teaching-Learning Methodologies Used in the ... 49
Several programming subjects are offered in the UnB courses. The first contact for the
student comes with the Intro to Computer Science (ICC) or Basic Computing (CB)
subject, which will be called Algorithms and Computer Programming (APC) starting
2015.2, even though it will maintain the syllabus and workload.
The subject program for the Technology/Engineering courses offered by UnB has
a set of differentiated subjects, but all programs offer the students plenty of subjects
of programming languages, besides the first ones (ICC, CB and APC). It is critical that
the student has a good understanding and comprehension throughout this first contact
with programming subjects. As such, understanding the teaching/learning process of
the first programming subject can help professors and students in the task of develop-
ing a more effective learning environment, this minimizing the difficulties found by
students in such subjects. This is the main contribution hoped to be achieved with this
study.
Fig. 1 presents an overview for the enrolled, passing, and failing students in the Intro
to Computer Science subject in Engineering courses at UnB. In the second semester of
2008 (2008/2), we had 251 enrolled students. In the first semester of 2013 (2013/1), we
had 408 students enrolled in the subject.
As can be seen in Fig. 1, in the second semester of 2008, the students a failure rate of
25% (2008/2). It can be seen that this rate went up with each passing term. In 2010/2, for
example, the failure rate was at 52%. In 2012/2, the failure rate was of 58%. In the term
in which the ICC subject was offered, 2013/1, this rate was at 54%.
Starting from 2013/2, the Intro to Computer Science was renamed to Basic Comput-
ing in all five Engineering Courses, namely: Aerospace Engineering, Automotive Engi-
neering, Energy Engineering, Electronic Engineering, and Software Engineering.
Fig. 2 presents the number of failing and passing students in the second term of 2008
(2008/2) in the Computer Science, Computing, Electrical Engineering, Communication
Network Engineering and Mechatronics Engineering courses. The Computing course
had the highest student failure rate at 43%. The course with the lowest failure rate this
semester was Mechatronics Engineering at 4%.
Fig. 1. Registration, Failing and Passing Students in the Introduction to Computer Science.
50 E.D. Canedo, G.A. Santos, L.L. Leite
Fig. 2. Failing and Passing Students in the Computer Science, Computing, Electrical Engi-
neering, Communication Network Engineering and Mechatronics Engineering Courses.
Fig. 3 presents an overview of the second semester of 2013 (2013/2) in all Engineering
and Technology courses at UnB. In the Fig. 3, it can be observed that the Automotive En-
gineering course had the highest failure rate at 63%, followed by the Electronic Engineer-
ing course at 53%. The Computer science course had a failure rate at 46% and the Mecha-
tronics Engineering course had a failure rate at 44%. The Computer Engineering course
had a failure rate at 32%. The course with the lowest failure rate was Communication
Networks Engineering at 20%, followed by the Software Engineering course at 19%.
Fig. 4 presents the scenario obtained in the first semester of 2017 in Engineering and
Technology courses at UnB. In the Figure, it is possible to identify that the Automo-
tive Engineering course once again had the highest failure rate, 36%, followed by the
Aerospace Engineering Course at 27%. The courses with the lowest failure rate were
Computer Science, at 13%, and Software Engineering, at 14%.
The Brazilian framework established by the federal government is the National Exam
for Student Performance (ENADE), a component of the National System for the Evalu-
ation of Higher Education (SINAES) which, besides cognitive contents, also evaluates
the infrastructure and opens up space with some questions about the teaching-learning
process (Garrido et al., 2017). Since it is a tool that deals with the big picture, in order to
deal with the courses of various natures, it brings information with little specificity for
the evaluated Institutions. Even though there is a series of questionnaires available for
the evaluation of teaching, these tools prioritize the acquisition of global data on teach-
ing in these institutions, not focusing in the satisfaction of the student in relation to the
general and specific aspects of the subjects offered throughout their education.
The students satisfaction with a given subject, when reviewed in literature, regards
the subject as a whole, exploring aspects such as the infrastructure, the professor, teach-
An Assessment of the Teaching-Learning Methodologies Used in the ... 51
Fig. 3. Failing and passing students in the Technology and Engineering courses in 2013/2.
Fig. 4. Failing and Passing students in Engineering and Technology Courses in 2017/1.
52 E.D. Canedo, G.A. Santos, L.L. Leite
4. Analysis of Results
The evaluation tool came to be thanks to the interest by some UnB professors, in 2017,
in knowing the level of satisfaction of UnB’s students in relation to the introductory pro-
gramming subject and in the potential of information that could be raised in a student-
focused subject evaluation.
A group of professors of UnB’s Software Engineering and Computer Science course
developed an internal tool to evaluate these subjects. The main advantage of creating
ones own questionnaires is that they can correspond to the college’s traits and necessi-
ties. After many literature reviews and group discussions, the first version of the tool was
tested and then made available for all students of engineering and technology courses.
The tool is a questionnaire (https://canedo.typeform.com/to/K2OrV5) that seeks to
assess the satisfaction of the student in relation to the different domains of the subject.
This includes an overview of the subject and various specific aspects of it, in a quantita-
tive way with open space for a qualitative expression. Thereby, as they evaluate the sub-
ject, the student actively participates in identifying the problem related to the high level
of failure and high dropout rates and, especially, in the discussion of possible improve-
ments. This allows the student to acquire ways of analyzing their action in a more criti-
cal manner, making them take a role of responsibility in the teaching-learning process.
(Moreno et al., 2007). To the university, it allows for the diagnosis of the status of the
discipline in the Engineering and Technology courses at the UnB, detecting its potential
weaknesses and strengths, problems, and successes.
At the launch of the questionnaire, there was considerable resistance by the faculty.
Many felt threatened, as they believed it would focus on investigating and punishing
them for their teaching roles; they believed the students wouldn’t be competent enough
to evaluate their teaching methodology, and so on.
4.1. Results
It is known that the student passing rate for a certain subject does not necessarily mean
that the teaching methodology or computational resources employed by the professor
are successful. Therefore, how to evaluate the success and benefits achieved in student
learning thanks to a certain teaching methodology?
As a way to evaluate the results achieved with the methodology adopted by the pro-
fessors of ICC, CB, and APC, some research questions were defined to compose an
investigative and evaluative questionnaire. This questionnaire was made available to all
students who had already taken CB, ICC or APC classes.
In total, 637 students answered the questionnaire. The obtained results were as
follows:
Fig. 6. Algorithms and Computer Programming and Basic Computing Subject taken by students.
Fig. 7. Students’ Answers to the Question: Did You Consider the Subject Challenging,
in Regards to Learning and Passing Rates?
Fig. 9. Opinion of the students in regards to the methodology of the professor assigned
to the subjects.
An Assessment of the Teaching-Learning Methodologies Used in the ... 55
Fig. 11. The Infrastructure (Laboratories) Provided by the UnB a Good Environment
to Take the Subjects.
Fig. 12. The Support by the Assistants is Enough to Learn the Subject.
Fig. 13. The teaching practice adopted by the professors is adequate to the level
of difficulty of the subject.
5. Conclusion
Analyzing the information from the database of UnB’s academic system, it is possible to
see a high rate of failure in the ICC, CB and APC subjects.
The questionnaire allowed us to understand a bit more of this situation thanks to the
students answers.
The students considered the UnB infrastructure a negative factor that needs to be
remedied. There are many broken machines in the laboratories, which calls for an invest-
ment for better lab equipment. It is also necessary to rethink the number of students in
each class. A reduced number of students would allow for a closer and more personal
teaching practice by the professor and better, more efficient understanding by the stu-
dent. Besides, the material made available by the professors needs to be improved, add-
ing more practice exercises and their respective solutions.
Within the currently adopted practices, we can identify the ones that are well re-
ceived by students, such as:
1) The inclusion of assistants: 71% of the students claim they look for the assistants
to look for help with the subject.
2) Perception of the importance of the subject: 85% say that the Intro to program-
ming subject is very important for their course.
3) Related to the use of a methodology: 42% of the students assure they noticed that
the teacher used some teaching methodology or practice.
4) Leadership: 73% of the students believe the professor of the subject possessed
leadership in relation to the class.
5) Project: 65% of the students feel comfortable working in a research project under
the professor with whom they took the Intro to Programming subject.
6) Language: 59% of the students believe that the programming language used in
the subjected facilitated their learning process and understanding of the subject.
58 E.D. Canedo, G.A. Santos, L.L. Leite
Acknowledgements
The authors of this paper would like to thank all the students of the University of Brasília
(UnB) who answered our survey and sent us reports and improvement proposals for the
research.
References
Al-Jishi, E., Khalek, N.A., Hamdy, H.M. et al. (2009). Students’ perceptions of the efectiveness of a profes-
sional skills program in preparation for clerkship training. Education for Health, 22(2), 57.
Bennedsen, J., Caspersen, M.E. (2007). Failure rates in introductory programming. ACM SIGCSE Bulletin,
39(2), 32–36.
Bosse, Y., Gerosa, M.A. (2017). Why is programming so difficult to learn?: Patterns of Difficulties Related to
Programming Learning Mid-Stage. ACM SIGSOFT Software Engineering Notes, 41(6), 1–6.
Cohen, P. A. (1981). Student ratings of instruction and student achievement: A meta-analysis of multisection
validity studies. Review of Educational Research, 51(3), 281–309.
Edgcomb, A., Vahid, F., Lysecky, R., Lysecky, S. (2017). Getting students to earnestly do reading, studying,
and homework in an introductory programming class. Proceedings of the 2017 ACM SIGCSE Technical
Symposium on Computer Science Education, (), 171–176.
Ehlert, A., Schulte, C. (2010). Comparison of OOP first and OOP later: first results regarding the role of com-
fort level. Proceedings of the Fifteenth Annual Conference on Innovation and Technology in Computer
Science Education, (), 108–112.
Galbraith, C.S., Merrill, G.B. (2012). Faculty research productivity and standardized student learning out-
comes in a university teaching environment: A Bayesian analysis of relationships. Studies in Higher Edu-
cation, 37(4), 469–480.
Garrido, S., Oliveira, K.M., Rezende, F.A., Funghetto, S.S., Griboski, C.M. (2017). A expansão da educação
superior no Brasil, a indução da qualidade a partir do SINAES e as novas perspectivas para a educação a
distância. Cadernos de Pesquisa: Pensamento Educacional, 10(25), 19–35.
Hoffbeck, J.P., Dillon, H.E., Albright, R.J., Lu, W., Doughty, T.A. (2016). Teaching programming in the con-
text of solving engineering problems. Frontiers in Education Conference (FIE), 2016 IEEE, (), 1–7.
An Assessment of the Teaching-Learning Methodologies Used in the ... 59
Jenkins, T. (2002). On the difficulty of learning to program. Proceedings of the 3rd Annual Conference of the
LTSN Centre for Information and Computer Sciences, 4, 53–58.
Kahneman, D., Fredrickson, B.L., Schreiber, C.A., Redelmeier, D.A. (1993). When more pain is preferred to
less: Adding a better end. Psychological Science, 4(6), 401–405.
Lahtinen, E., Ala-Mutka, K., Järvinen, H. (2005). A study of the difficulties of novice programmers. ACM
SIGCSE Bulletin, 37(3), 14–18.
Martins, S.W., Mendes, A.J., Figueiredo, A.D. (2010). Comunidades de Investigação em Programação: Uma
Estratégia de Apoio ao Aprendizado Inicial de Programação. IEEE-RITA, 5(1), 39–46.
Moreno, L., Gonzalez, C., Castilla, I., Gonzalez, E., Sigut, J. (2007). Applying a constructivist and collabora-
tive methodological approach in engineering education. Computers & Education, 49(3), 891–915.
Murphy, E., Crick, T., Davenport, J.H. (2017). An Analysis of Introductory Programming Courses at UK Uni-
versities. Journal Art, Science, and Engineering of Programming – AOSA, 1(2), article 18.
Pitterson, N. P., Brown, S., Villanueva, K. A., Sitomer, A. (2016). Investigating current approaches to assessing
teaching evaluation in engineering departments. In: Frontiers in Education Conference (FIE), 2016 IEEE.
1–7.
Watson, C., Li, F.W. (2014). Failure rates in introductory programming revisited. In: Proceedings of the 2014
Conference on Innovation & Technology in Computer Science Education. 39–44.
Wiedenbeck, S. (2005). Factors affecting the success of non-majors in learning to program. In: Proceedings of
the First International Workshop on Computing Education Research. 13–24.
Wilson, B.C., Shrock, S. (2001). Contributing to success in an introductory computer science course: a study
of twelve factors. ACM SIGCSE Bulletin, 33(1), 184–188.
E.D. Canedo holds a PhD degree in Electrical Engineering from University of Brasília
(UnB), Brazil in 2012. Master by Federal University of Campina Grande, (UFCG) in
Software Systems (2002). I’m graduated in Systems Analysis by University Salgado de
Oliveira; Goiás (1999). I’m Professor of the Computer Science Course at University of
Brasília – (UnB) since 2010. Her research interests include Informatics on Education,
Software Engineering, Cloud Computing and Software Systems.
G.A. Santos is a full time Professor at University of Brasilia (UnB), Brazil, since 2010.
He was a Professor at the Catholic University of Brasilia during 2003–2010. He obtained
his master’s in computer science in 2001 and currently is a PhD student at Electrical En-
gineering (UnB). His research interests include Informatics on Education, Collaborative
Learning, and Educational Robotics.
L.L. Leite holds a PhD in Computer Science and has interest in the following research
areas: Human-Computer Interaction and Computer Science in Education. She develops
projects related to the use of technologies to support teaching and learning, aiming to
decrease the dropout rates in Computer Science courses. Her projects includes topics
such as Computational Thinking, Teacher Training, and Algorithms and Programming
Education.