Home-based learning through low-cost mobile teaching: Findings from Odisha

Evaluating the effectiveness of a low-cost Home-Based Learning programme implemented during the Covid-19 school closures in Odisha, this article finds that it improved the mathematics and language learning outcomes of children in under-resourced communities. With a rise in hybrid teaching and learning, the findings suggest that such technology-based learning interventions have the ability to provide a safety net in the event of sudden school closures and cushion against a decline in learning levels.

The Covid–19 pandemic led to partial and full school closures that affected almost one billion school children globally. According to the UNESCO (2021) and UNICEF (2021) press releases, an additional 100 million children were projected to fall below the minimum proficiency levels in reading. While many students around the world relied on virtual learning online, for those with no internet access, education came to a standstill.

In India specifically, the pandemic resulted in the closure of 1.5 million schools, and the nationwide lockdowns in 2020 impacted 247 million children enrolled in elementary and secondary school (UNICEF, 2020). With over six million children in India out of school even before the advent of Covid-19, this issue became one of grave importance since school closures would further widen the gap between those with the resources to remain engaged in learning and those lacking them. In a recent paper (Ojha and Yadav 2023), we examine the effects of a phone call and SMS-based intervention called Home-Based Learning (HBL), with the idea that prioritising educational recovery is crucial to avoid a generational catastrophe, especially for the poor in rural areas.

This paper joins other recent research testing the effect of low-cost phone-based learning, bringing clear evidence to this important topic (including Angrist et al. 2020, Crawfurd et al. 2021, Radhakrishnan et al. 2021, Rodriguez-Segura 2022, Ahluwalia et al. 2023).

The state of schooling in Odisha during Covid-19

Odisha was first hit by Covid-19 in March 2020 with little respite thereafter. Odisha was one of the top ten affected states in the country, clocking a test positivity rate of more than 10% during the month of July, with community transmission setting in (Agragami 2020). As a preventive measure, the state government shut down all schools and colleges, first until June, which was further extended over the course of the year until December 2020.

Initiatives like the E-Vidyalaya app and Madhu app were launched, with a focus on senior secondary classes, but such measures reached only 2.2 million of the 6 million students in the state. As part of the state government's Odisha Shiksha Sanjog programme, students at both private and public schools received learning materials over WhatsApp, but almost 50% of them could not access these resources due to a lack of smartphones, limited internet access and connectivity issues (ASER, 2020).

The lockdown lifted for a brief period in early 2021 – however student engagement continued to fall, with fewer than 10% of students attending Radio Pathshala classes, and only 22.5% attending the virtual classes broadcasted by Doordarshan under the Shiksha Darpan initiative. On the directive of the state government, in March 2021, students of Standard 1 through 8 were promoted without the requirement of any examinations for a second consecutive year. This was followed by a second Covid wave, which brought in a second round of school closures.

The intervention

In light of the school closures, the HBL programme was intended to mitigate learning loss by targetting students who could not access study material and were out of school. The intervention targeted students at government schools in in three districts in Odisha. Specifically, the programme provided remote instructions teaching basic number recognition and arithmetic operations, and the native language, Odia, through phone calls and simple text messages along with automated voice messages sent to children aged 6-10 years. The intervention was implemented by ThinkZone, a social enterprise that worked with instructors and in consultation with school teachers to deliver the programme, with support from Mo School, a Government of Odisha initiative under School and Mass Education Department.

The three districts of Odisha in which the interventions were conducted were chosen such that they represented similar levels of development. Within each district, some clusters were randomly assigned to the treatment group and some to the control group. Phone numbers for students were obtained from the government schools in each cluster.

The HBL programme was implemented for all government school students from Standard 1 to 5 in the treated clusters. The voice calls and text messages were sent to a parent's phone, so that they could help engage their children in the activity-based learning content. Access to the content was provided free of cost, and the activities were based on the learning outcomes (in terms of language and arithmetic skills for primary grades) specified by the state government, jointly with the National Council of Educational Research and Training (NCERT). Baseline and endline tests were designed based on the ASER testing tools, that are simple, quick to administer and considered largely reliable in the context of India (Vagh 2012).

Key findings

Utilising a difference-in-differences strategy¹, our study found that exposure to HBL resulted in improvements in student outcomes, measured in form of endline test scores in Maths and Odia. The improvement in scores is consistent across different specifications, alternative estimation techniques, and sub-samples, thus providing robust evidence that the intervention resulted in better learning outcomes in the treatment group, relative to students in the control group who were not a part of the programme, during the time when schools were shut. Specifically, the intervention led to an improvement in basic number recognition and arithmetic operations by 4.69 percentage points, and language learning scores of children by 5.52 percentage points. Additionally, the treatment had a statistically significant effect on both girls and boys in the treated cohort, although the effects are larger in magnitude for girls for both Maths and Odia. The paper also finds greater effectiveness of the treatment for the weakest students.

Limitations and concluding remarks

The improvements in student learning levels ought to be appreciated in the broader context of pandemic-induced school shutdowns and movement restrictions, which were among the longest and most severe in the world in India. The results speak to the extent to which such low-cost interventions having the capability to supplement education in a post-pandemic world, wherein hybrid formats of teaching and learning are expected to become commonplace.

Like any recovery after a crisis, there is no single way to ensure that students return to school and are able to catch up with their peers with access to better resources. As such, teacher-assigned homework and parental involvement have been traditionally considered two important sources of learning. The HBL programme tries to integrate both of these aspects in their online classes, with the aim to keep students engaged in learning. Allowing an opportunity to supplement morning school with an evening revision through a programme like HBL may also reduce dependency on costly private tuition.

Our results are in line with recent evidence by Rodriguez-Segura (2022), Kumar et al. (2022) and Ahluwalia et al. (2023) that establish the reliability and validity of phone-based assessments and recommend effectively integrating them in teaching-learning pedagogies. While the programme cannot (and should not) be seen as a substitute for regular schools, governments may consider investing in providing such safety nets, especially for students with limited access to a smartphone or a school and wish to remain engaged with learning in the face of sudden shocks.

Notes:

1. A difference-in-differences strategy is used to compare the evolution of outcomes over time in similar groups, where one was impacted by an event or policy – in this case, the HBL programme – while the other was not.