ASER 2018: Behind the headlines

Thanks to more than a decade of Annual Status of Education Reports (ASER), the main headlines from the surveys are widely known – the learning levels of Indian school going children are worryingly low. In this article, Rukmini Banerji takes a closer look at the recently released ASER 2018 data to see what else it can tell us, and discusses actionable steps to bring forward children who are falling behind.

Thanks to more than a decade of ASER (Annual Status of Education Report) reports, the main headlines from the surveys are widely known.¹ Even those who are not education experts or researchers can tell you that after five years of schooling, only half of all children in India can read at grade 2 level. And that the results for basic arithmetic are even more worrying.

In the early years of ASER, there was disbelief. Whether in meetings in the Planning Commission or in discussions at village level, people would say "how is it that children cannot read, after all they are going to school!" Sceptics would question the sampling. Critics would reject the tools. Others would be doubtful about how volunteers could pull off such a massive exercise. But year after year, like clockwork, the report would become available in mid-January. The results were consistently saying that learning needs attention. The relentless hard work of thousands of people involved in the effort began to pay off. More studies of children's learning began to appear. All of this influenced and contributed to local as well as national debates on education.²

Acknowledging and accepting a problem is certainly an important first step. It is now well recognised that learning levels are low and that they are not changing much as years go by. In fact, for a few years, we even saw distinct declining patterns. What is also known is that although children continue to add years of schooling to their portfolio, for many, learning trajectories remain relatively flat. As Pritchett (2017) puts it, “if a learning profile is flat, schooling only measures ‘time served’ and not ‘skills gained’.” (RISE (Research on Improving Systems of Education) 2017)

The next step beyond acknowledging, recognising, and accepting is understanding which in turn requires going behind the headlines. The World Development Report 2018 argues that when issues of learning are taken seriously, and learning becomes a high priority, then progress can be made towards solving the learning crisis. The three fronts on which the report recommends action are assessing learning outcomes; acting on the evidence to make schools work for all learners; and aligning all actors to make the whole system work for learning.

Taking a closer look at ASER data

Now that everyone accepts that learning outcomes are worryingly low, let us take a closer look at ASER data to see what else it can tell us. For the purposes of this discussion, let us focus on grade 3. After spending two years in the formal education system, children are ‘settled’ in school. Grade 3 is also the earliest grade at which the national achievement test is administered.³ It is also relatively straightforward to align what children are expected to do by the end of grade 2 or beginning of grade 3 with several of the ASER tasks. In the ASER process, the ‘highest’ level task, at least in reading, is to ask a child to read a text at grade 2 level of difficulty. In arithmetic, children are asked to recognise numbers, do a numerical two-digit subtraction problem with borrowing, and finally solve a numerical division problem (for example, divide a three-digit number by a one-digit number). The ASER tests are progressive, so each child is marked at the highest level that she can comfortably reach. In most states, by the time children enter grade 3, they are expected to be reading a simple text fluently and confidently doing arithmetic operations like addition or subtraction with numbers at least up to 100. Hence, if a child can read text at grade 2 level of difficulty and correctly solve numerical subtraction problems, then we can say that the child is at ‘grade level’ for grade 3.

According to ASER 2018, the all India figure for the percentage of all children in grade 3 who are able to read at grade 2 level is 27.2. The corresponding number for the proportion of children who can at least do subtraction is 28.1. It is obvious that these figures are low; in grade 3, only a quarter of all children are ‘ready’ for the grade in which they currently are. In addition, year after year, ASER data has been pointing to the wide spread of learning levels within the same grade. Table 1 shows the distribution of learning levels for a national sample of grade 3 children (all India rural) in 2018.

Table 1. Percentage of all children in grade 3 (rural) who:

Reading level	Cannot recognise letters yet	Can recognise letters but cannot read words	Can read words but cannot read sentences	Can read text at grade 1 level but not higher	Can read grade 2 level text	Total
Grade 3	12.1	22.6	20.8	17.3	27.2	100
Arithmetic level	Cannot recognise numbers till 9 yet	Can recognise numbers till 9 but not higher	Can recognise numbers till 99 but cannot subtract	Can do 2-digit by 2-digit subtraction but not division	Can do 3-digit by 1-digit division or higher	Total
Grade 3	7.6	26.9	37.5	19.6	8.5	100

Source: ASER 2018

All these children are in the same grade and in the same age group but their ability to read or do arithmetic varies widely. Data indicate that in a grade 3 class, we may have some children who are at grade 2 level, some at grade 1 level and some who are like preschoolers in terms of their literacy and numeracy levels. This variation has been referred to as one of the “most critical constraints in the structure of the Indian education system today” (Muralidharan 2018).

Table 1 suggests that only about a quarter of all children in grade 3 in rural India can read fluently. If you cannot read, you cannot be expected to do a pen-and-paper test. The data in Table 1 clearly shows that the vast majority of children cannot read, which means that they cannot follow written instructions. The first implication of looking closely at the data is that assessment methods for grade 3 cannot only have the usual written tests. ASER uses tools that are used one-on-one with each child. If we want to understand whether a child can read, there is no way to figure this out, other than asking her to read and then listening to her.

To go one step deeper, let us look at the spread of learning levels in two states – Himachal Pradesh (HP) and Uttar Pradesh (UP) (Table 2). Both are states where improvement in learning levels is visible in ASER data between 2016 and 2018.

Table 2. Percentage of grade-3 government school children reading at different levels: Himachal Pradesh & Uttar Pradesh

	Cannot recognise letters yet	Can recognise letters but cannot read words	Can read words but cannot read sentences	Can read text at grade 1 level but not higher	Can read grade 2 level text	Total
Himachal Pradesh
2016	2.4	16.6	12.3	23.6	45	100
2018	2.4	10.6	15.5	24.1	47.4	100
Uttar Pradesh
2016	28.2	40.3	15.3	8.9	7.2	100
2018	24.5	36.7	16.8	9.7	12.3	100

Source: ASER 2016-2018.

Even with a cursory look, several patterns are clearly visible. First is the variation between the two states. If there were to be a league table of reading for grade 3, HP would be at the top end of this list while Uttar Pradesh would be towards the bottom. In HP, almost half of all children can read at grade 2 level and another quarter are close behind. The picture from UP is exactly opposite. In 2018, in UP, 12% children are at grade level and another roughly 10% are close behind. That leaves three quarters of the children who are at least two grade levels behind. Compared to what is expected in grade 3, this suggests that 75% to 80% of children in UP who have reached grade 3 are still at pre-school level of literacy and numeracy. (As children move up in the school system, the dispersion gets wider. By grade 5, there are children who are at grade level as well as children who are still struggling with numbers or letters – so at least five grade levels behind!).

Imagine the challenge that teachers face in teaching in such contexts. Not only is there vast variation in the levels of children in the class, but distance between the expectations of the curriculum and where children currently are is also massive. The usual teaching-learning approach used in most Indian classrooms is to teach from the grade level textbook and focus on “teaching to the top of the class” (Banerjee and Duflo 2012). Further, “the curriculum targets only the very top of the distribution and leaves most students behind; the immense variation within a classroom makes the delivery of any effective instruction very hard; and, consequently, most students are far from grade-appropriate standards even after completing the full course of elementary education.” (Muralidharan 2018)

Teaching at the right level

In a recent article, Raghuram Rajan and Abhijit Banerjee lay out eight things that India needs to do in 2019. For education, they say “The Right to Education Act focuses on input requirements for schools that have little bearing on learning outcomes, which have deteriorated alarmingly. Learning must be our central focus, with all schools, public and private, responsible for delivering a minimum level of basic skills to every child. Bringing those falling behind up to par through remedial teaching will be critical.”

Effective ways to bring forward children who are falling behind are available. Pratham's ‘Teaching at the Right Level’ interventions have been rigorously evaluated by MIT's (Massachusetts Institute of Technology) Abdul Latif Jameel Poverty Action Lab (J-PAL) over the past two decades. This research has shown that the approach has led to some of the largest and most cost-effective learning gains of any primary education programme evaluated. The most recent study in UP showed that the overall large learning increases in a classroom were particularly driven by the children who most needed help – those who began at the lowest levels of literacy and numeracy. Thus, even the very low learning situation and highly skewed distributions seen in UP can be reversed in a matter of days with the right focus and effort. As the WDR 2018 suggests, making learning high priority and aligning the system to ensure learning for all can reap good results. Based on these experiences, several state governments across the country are putting time aside during the school day to be used specifically for building foundational skills, and children are being grouped by level rather than grade for instruction. Promising results from the ‘teaching-at-the-right-level’ approach are becoming visible in large scale implementation by states. More work of this kind will lead to a national belief that the situation seen for grade 3 in the ASER data can be improved, across states and in other grades, without too much additional cost.

But what led to this learning crisis to begin with? There are many contributing factors. Poorly educated parents and the lack of learning support at home is certainly a contributor. Inadequate school readiness, rote learning methods of teaching, paucity of appropriately trained teachers, and no system of identifying or helping children who are not making adequate progress in the early grades – all can be listed as problems.

However, a key underlying feature is what has been termed the “negative consequences of over-ambitious curriculum” (Beatty and Pritchett 2012). For example, in the grade 3 textbook in UP, there is a section where a young child goes with her father to a shop to buy a mobile phone. Her father has Rs. 3975. They see several mobile phones – one for Rs.3260, another for Rs. 3460, yet another for Rs. 3874 and a last one for Rs. 4077. The child and her father have to take a decision on which phone they can buy and how much money they would have left over after buying a mobile phone. Remember this is a situation in which 60% children in the state cannot as yet recognise numbers till 100, and only 11% children can actually do operations involving subtraction.

Actionable points

In conclusion, once the headlines of this year's ASER have been absorbed, anyone reading the ASER 2018 report and analysing the implications of the evidence for policy and practice, must leave with at least these three action points in mind:

(i) Appropriate assessment: Pen-and-paper assessments do not make sense for most children in grade 3 in India. Understanding their current level of reading or arithmetic will need other methods like working with them one-on-one with oral, interactive tasks.

(ii) ‘Catch up’ action: It is needed urgently and on large scale. If most children can acquire basic foundational skills like reading and arithmetic by the end of grade 2, then a huge national problem of later learning gaps can be solved. Existing research and practice show that effective programmes can be implemented to solve the learning crisis early. But this requires moving away, at least for part of the school day or school year, from the current curriculum and textbook content to focus on foundations. To ensure that every child has the opportunity to ‘catch up’ requires a significant realigning of all elements of the education system. This ‘catch up’ will involve millions of children and hence how to get this done must be the highest priority for policymakers, planners, and practitioners.

(iii)Immediate and thorough re-visioning for the early grades: This extends to rethinking both ‘what’ and ‘how’. What are the goals? What should a child entering grade 3 be able to do? How can curriculum in the first two years support teachers and schools to enable children to reach these goals? How should it be reflected in textbooks and other content? How should the teaching practice and assessment methods be changed? It is not simply a question of ‘lightening’ the load but more of reconceptualising what is needed and at what pace. Today's textbooks expect a far higher level of literacy and numeracy ability than today's children bring to the classroom in grade 1, 2, or 3. It is essential and urgent to realign academic expectations with the system's ability to deliver, with teachers' capability to support, and children's capacity to acquire, assimilate, and progress.

Concluding remarks

All available data shows that India is close to achieving ‘schooling for all’. Now is the time to make ‘learning for all’ a national priority. We need to move beyond this year's ASER headlines into meaningful action. Ensuring that every child has the opportunity to acquire foundational skills in primary school will need substantial changes in the ways that the system currently works. We need to rework what we are doing, why we are doing it, and how we do it, from the policy level to the classroom level.

As a country, we have acknowledged that we have a crisis of learning on hand. Now it is time to understand the contours of the problem and take decisions accordingly, so that year on year there is progress. The first step to lift up the learning trajectory of children is to ensure foundational skills. To enable millions of children to learn how to read, to comprehend, and to calculate we need a massive ‘catch up’ effort. This ‘catch up’ needs a ‘push forward’ and not a ‘hold back’. We need to believe that the real right to education is not only in terms of years of schooling but ‘value added’ in terms of learning; first foundational skills, then higher level capabilities and knowledge, and finally to being able to cope with a dynamic and changing wide world beyond.

This article first appeared in Annual Status of Education Report (Rural) 2018.

Notes:

Oza and Bethell (2013) state that “there are still many lessons that can be learnt from the reporting formats used by, for example, Pratham/ASER and Educational Initiatives. Notwithstanding any technical limitations, these agencies consistently produce reports which are attractive and eminently readable. ASER, in particular, has been extremely successful in extracting from its studies "headline findings which catch the attention of the media and, hence, generate a great deal of press coverage” (p 46).
See Oza and Bethell (2013) reference to ASER being influential in policy formulation by both the central and state governments, “The grassroots approach utilised has been significant in bringing attention to learning outcomes in India” (p 22).
NAS – national achievement test is a nationwide assessment carried out periodically by the central government. A sample of children in grades 3, 5 and 8 are assessed.