Poor Quality of Student Learning: What does the data show?

Public debate in India bemoans the lot of government schools in the country. The implicit assumption is
that all’s well (or at least almost well) with private schools. With the help of Wipro’s community
initiatives group, our company tried to verify this assumption. Between February and April 2006, we
administered specially prepared tests to about 32,000 students of classes 4, 6 and 8 of 142 English-
medium schools in Kolkata, Mumbai, Delhi, Chennai and Bangalore. These schools were those
perceived by the public to be the leading schools of their cities (and were identified through an
independent survey).

An analysis of the results of the study suggests that even in these ‘top’ schools, students are not learning
well and with real understanding – their learning is more superficial and based on recall of bookish facts.
Probably more disturbing was that these children - from our top schools - scored lower than the children
of 43 countries in an international achievement test. Details of the study and the findings are available at
http://www.ei-india.com.

The first part of the study was a survey of about 200 people from different walks of life in each metro to
identify the ‘best’ schools as per popular perception. Based on their responses, a list of 50 top schools
was drawn for each city. These schools were then invited to participate in the study. Students of classes
4, 6 and 8 of each school were tested for their learning achievement with a special test tailored to their
age and ability. The test tried to measure how well students of these classes understand the key concepts
in English, Mathematics and Science. Apart from the multiple-choice questions, students were also
required to write a small paragraph or essay, which would help study their writing competencies.

A secondary study was also conducted to understand the progression of learning achievement across the
classes: a common test was administered to students of classes 4, 6 and 8, to gather insights on the
retention and development of knowledge, as students move to higher classes. Additionally, about 25%
of the questions in this paper were taken from an international assessment study (the Trends in
International Maths and Science Studies - TIMSS, http://www.timss.org) for which performance data of
students from all over the world is available. An expert panel of educationists and principals guided the
survey.

The findings suggest that schools are laying disproportionate emphasis on rote and procedural learning
and not surprisingly, students tend to be ‘strong’ in those. To a certain extent this is good - for example
it builds habits of rigour and hard work. But when it starts replacing original thinking and creativity,
over-reliance on rote can be extremely counter-productive. This is happening, and we need to be
alarmed.

Thus it seems that students are learning mechanically. They are able to answer questions based on recall
or standard procedures quite well, but their performance on questions testing understanding or
application is far below acceptable levels.

Similarly, students fare poorly in questions requiring practical, real-life skills. Some examples of these
skills are measurement and estimation, correct use of day-to-day expressions in language, and
application of principles of science in everyday life. For example, students are able to say that iron nails
conduct electricity as this is taught in textbooks, but they are not able to deduce that if one of three
batteries of a torch were replaced by an iron nail, the torch would still light up (though less brightly).

The student performance suggests that they are unable to tackle questions that appear to be a little
different from what they typically find in textbooks or in the class. Their ability to apply what they have
learnt to new, unfamiliar problems - so important in today’s world - tends to be low.

Comparative Analysis
A comparative analysis of the performance of the 5 metros again threw up a surprise. It was found that
the performance of the cities fell into two categories, with Kolkata, Mumbai and Delhi clearly
outperforming Bangalore and Chennai. It was also found that schools affiliated to the CISCE (ICSE)
board out-performed the CBSE board which in turn out-performed the state boards.

Boys outperformed girls by a margin that was statistically significant in Mathematics (in all classes).
The gap widens even more among better-performing students and in the case of difficult questions. We
believe that these differences are not because boys are inherently better in Mathematics than girls, but
due to social messages encouraging boys to do better in Mathematics and probably discouraging girls.

Many of these findings were corroborated through the secondary study in which learning levels across
classes were compared. While learning clearly improved from class 4 to 6 to 8, a number of students
seem to be understanding class 3 and 4 concepts only around class 6 or later.

Educational Initiatives and Wipro have released all the data including the question papers and detailed
analysis in the public domain. (They are available at http://www.ei-india.com/whats-wrong-with-our-
teaching). The main reason for this is to allow the issue of quality of learning to be more widely debated
on a foundation of hard data, rather than subjective “opinions”. In the coming year, we are planning to
expand this study to more cities in the coming year, and also enhance the study in other ways.

It is clearly in our power collectively to improve the quality of learning in our schools, but that will
happen only if we choose to make that commitment to the next generation, by way of focusing on real
learning. The current focus on valuing high scores in the board exams or fancy facilities in our schools is
unlikely to take us far, as far as real learning is concerned. Our tests should be such that they measure
real learning.

Yet the reaction to surveys like this remains restricted to trying to apportion blame. Who is responsible
for this state of affairs? We shall look into this in the next part of this article.
 Poor Quality of Student Learning: Who’s to Blame?

According to a study conducted in top private English medium schools by us jointly with Wipro,
students, even in these ‘top’ schools, are not learning with understanding. Also they score lower than the
average of students from 43 countries in a reputed international test. Details of the study and the
findings are available at http://www.ei-india.com.

Results showing poor learning in government schools are not new, but this study is probably the first of
its kind involving research based large scale assessment of top private schools. So who is to blame for
this poor quality of student learning?

The first tendency is to point accusing fingers towards teachers and schools, as it is they, after all, who
are engaged in the task of actually teaching children.

After teachers, government and government policy is what we love to blame. If only government would
privatise schooling, or run English-medium schools, or pay teachers better, different lines of reasoning
go..., our problems would disappear!

Though each of the above has some element of truth in them, the complete truth, as always, is far more
complex. And it is that the education system concerns us all. As boards, schools, teachers, parents,
students, administrators, businesses or simply tax-payers, we are a part of it. With the exception of
students, all of us are to blame for this mess.

The boards of education – the CBSE, the ICSE and the State Boards – are responsible for setting and
correcting test papers and announcing results. If there has been a dilution of standards, if questions asked
are too mechanical, if students score above 90% and still count themselves as failures – the boards are
responsible for this.

Many of us, as parents, may see ourselves as pressurized and the victims of an uncaring system, but it is
parents who have perpetrated and strengthened this system. Parents goad their wards to score more and
more - showing scant care for real learning. Between 60% and 80% of students in private schools attend
tuitions, and this would not be possible without parental approval.

Tuition classes have virtually become a parallel system of schooling, but a system where nothing matters
more than guide-books, marks and exams. The business of coaching at various levels is one of the most
profitable, and online tuitions are now a growing, if ignominious, export.

Educational experts and researchers are often guilty of not trying enough to get their word out. To take
one example, how many of us know that educational research is almost unanimous in the
recommendation that primary education should be in the mother tongue, not a language unfamiliar to the
child (like English, the medium of almost each and every of our elite, coveted schools)?

Of course, schools and teachers have not exactly distinguished themselves when it comes to ensuring
that real learning is happening in schools. They see themselves as powerless, caught between the boards
on the one side, parents on the other and tuition classes on the third, and do not perceive themselves as
having the power to transform society.
The rest of us – members of the corporate world, or simply other citizens – are not free from blame
either. We pay our taxes and 2% education cesses, but we condone a lot of what goes on in the name of
education, including huge salary differentials in education compared to corporates, because we are too
busy (and we are not the experts, are we?) to make a difference. If there is one thing our educational
system lacks the most, it is a connection with real life, and can we as the practitioners absolve ourselves
of all blame for this?

Maybe an acceptance that it is we, collectively, who have got ourselves into the current quagmire, will
help us realize that only we can get ourselves out of it. It is not as if valiant attempts have not been made
or are not being made to change things, but we have not allowed those efforts to reach a critical mass.
We are part of a system – a large, complex system – and such systems have a dangerously effective way
of neutralizing efforts to change the status quo. Yet such systems can be changed, and history does show
notable examples. This realization, the acknowledgement that “we’re all in it together” and a willingness
to challenge old assumptions and stereotypes with confidence and hope, rather than trying to fix blame,
may be the place to start.

In the concluding part of this series, we shall examine a set of concrete steps that can be taken to make a
dent in the existing system and try to create one which will help students learn better.
 Poor Student Learning: What can be done?

That there are serious gaps in student learning – both in government schools and in top private schools -
is a matter of grave concern. In the earlier part of this article, we concluded that all of us – teachers,
schools, educationists, parents, corporates, education boards, tuition teachers – are collectively
responsible for this, and little will be achieved by pointing fingers at each other. Can something be done
to improve the system? Can the solution take all these various groups along, instead of pitting one
against the other?

The overall solutions to address the current malaise are fairly well-known and often proposed: Change
the paradigm of teaching and learning – for example, increase group and activity work, train teachers in
participative techniques, reduce dictation of notes, etc. Improve textbooks so that they may cover less
content but more meaningfully, possibly in greater depth. Eliminate textbooks till class 3, having only
teacher guides at that stage. Change the nature of the board exam questions, so that understanding - not
the power to memorise - is tested… etc.

These suggestions are not new. However, attempting to make such large changes is a risky endeavour
with significant chances of failure-
• First of all, implementing these changes across such a large system is a complex and massive effort.
• Secondly, there is the challenge of taking a large number of people along. Different people and
groups have their own perspectives and priorities, and this may prove a difficult task. The negative
repercussions or fall-outs of such changes can be high at a practical level- especially in the
transitional stages. Protests by parents and students to drastic changes in board exams can be
imagined.
• Thirdly, how does one monitor and agree that progress is being made? This involves “measuring the
immeasurables” like student learning, teacher effectiveness, and school efficacy – a discipline in
which we are still struggling and have had very limited success. Without an agreed upon
“barometer” for these key factors, it would be impossible to drive consensus or even have
meaningful debate.

In the language of “systems thinking”– when an attempt is made to change a large system by tweaking a
few of its parts, the system reacts with steps that in effect compensate for the changes, so that the net
change in the system is minimal! Thus guides get produced even for new textbooks aimed at improving
practical learning. Parents in Maharashtra protested and forced a plan to test children in class 4 to be
withdrawn, possibly without understanding that this was a measure of accountability primarily for
primary school teachers, not a desire to add another exam to already over-burdened students.

What is the alternative then? Can we make smaller changes in a way we “incentivise” stakeholders and
educate them, thus inducing slow but irreversible changes in behaviour that can bring about the desired
results? Here then are some innocuous-looking ideas based on 3 simple philosophies:

1. Compulsion will not work, but building on people’s desire to contribute, do good and better
themselves may. For example, banning tuitions is unlikely to work. But steps should be taken that
lead to the elimination of tuition. Similarly, do not force teachers to, say, enhance their skills, but
provide opportunities that can be voluntary tapped, and the motivated ones will automatically take
the opportunity and the some others will follow later.
2. Government should do less and enable more. It should have expert teams and even those teams
should not do the stuff. That should be left to private players. The expert team should form
guidelines, rate and ratify stuff, reward excellence, etc.

3. Doing all the right things is obviously the ideal, but even if some of the ‘easier’ things can be done
or corrected, a gradual momentum towards improvement can be built up.

On the face of it, these are small ideas that try to tweak a small part of the system. They are relatively
easy to implement – there is very little compulsion in any of them – and they can be started in small
ways. And yet in their own way, these measures can together help completely change the current system
– no less!

1. Boards (starting with the CBSE) should award percentile scores in addition to marks. Over 3 years,
the format and the official board results should start emphasising the percentiles, not the marks.
2. An application section should be added to board exams. The application section should have 10%
weightage in the first year, but this weightage should increase by 10% each year till it reaches 60%.
3. Two public education campaigns - maybe not very different from the current ‘Incredible India’
campaign on promoting tourism and respecting heritage - should be launched. One of them should
emphasise the role of the teacher and encourage the best people to join the profession. The other,
targeted at parents, should emphasise that real learning is more than marks and exams.
4. The state textbook boards should be converted into Textbook Approval Committees. Staffed by
professionals, one central committee should provide syllabus guidelines for textbook makers, and the
state committed should approve and provide a rating to every textbook submitted. Printed textbooks
must carry this rating and schools can choose from the different textbooks available.
5. On the lines of the steps initiated by the NCERT, all states should put up the current textbooks
online for free access. The new universal standard called ‘Unicode’ should be used for regional
languages.
6. Various steps should be initiated so that a clear message is sent out that excellence is valued. It is
interesting how this is either not the case or not communicated effectively today. Some of these steps
are:
a. A nation-wide voluntary scholarship test around class 6. Gradually, such competitions can be
organised around different non-academic skills as well.
b. Competitions for teachers - around best teaching aids, lesson plans, etc. The web should be used
extensively to both promote transparency and aid sharing.
7. Various steps need to be taken to actively promote professional development for teachers. The most
important of these would be voluntary tests for teachers - tests that allow a teacher to know where
she stands. Government should encourage private initiatives to develop these and ensure quality.
Though the certification would not (and should not) have any statutory value, the quality they
represent will soon earn value. For any new initiative, a ready pool of the more motivated, more
capable teachers will be readily available.
8. Addressing teacher grievances: Teachers have a number of valid problems and one way to address
them is to allow them to be brought out into the open. Technology allows us to do this today - a
section of a national website for teachers should allow them to enter their grievances in their own
language. An independent group will remove trivial complaints and also maintain a summary (state-
 wise, later district-wise probably) of how many grievances have been published and how many have
been resolved.
9. Last, but as important as anything else, encouraging systematic research on a number of topics
which are of critical importance if the educational system as a whole has to improve. Some of these
are the mother tongue versus English question; identifying a measurable set of ‘Markers of
Excellence’ of a good school, and even compiling ’50 Good Ideas than Work’ or something like that.

The above are just a few ideas, presented briefly. The point is to initiate a larger debate and take this
forward. For more details or to join the debate, please write to sridhar@ei-india.com

Sridhar Rajagopalan is the managing director of Educational Initiatives a company working to enhance
student learning through initiatives like its ASSET test.