Ideas to improve the performance of India’s school system, many of which involve the use of information and communication technology, have been proposed and implemented with varying degrees of success for a long time. In this note, Vivek, Bhattacharjee, Mani, and Kumar report on the lessons learned from the launch of a mobile application by the Government of Jharkhand, to replace the pen-and-paper-based data collection by education administrators in public schools.
Ideas to improve the performance of school systems, many of which involve the use of information and communication technology (ICT), have been proposed and implemented with varying degrees of success for a long time. Technology tools, at the bare minimum, can help ensure availability of high-quality information, and make data management more efficient. This, in turn, could lead to better, results-based financing (RBF) for project implementation by facilitating improved monitoring and evaluation. However, failed technology interventions happen to outnumber success stories for most education problems across geographies. The wisdom on successful technology integration professes a precise articulation of the problem in the specific operating context in close consultation with stakeholders, identifying the ‘right technology’ and implementation strategy to meet the challenges in the operating context, and regular monitoring and evaluation for continuously improving design and implementation. In Covid times, when public and private systems in developing countries are transitioning quickly to adopt technology-based solutions, it is even more imperative to exercise caution in designing and deploying effective ICT.
Mobile app for public education administrators in Jharkhand
Block Resource Persons (BRPs) and Cluster Resource Persons (CRPs) are a cadre of officials who serve as the ‘middle-level management layer’ in India’s lower primary (grades 1-5), and upper-primary (grades 6-8) public schools. They constitute the interface between teachers and district- or state-level functionaries, and are responsible for regular on-site monitoring of schools, providing academic mentorship to teachers, and mobilising local community members to participate in improving school management. Their monitoring function includes collection and analysis of data on school-level attributes, teacher performance, and student achievement. These administrators are often engaged in administrative activities, and much of their time is spent on collecting data from schools, leaving little time for academic mentoring and support of teachers. The Government of Jharkhand needed to cut down the time spent on manual data-gathering, as well as generate better analytics for decision support. To achieve this, it developed a mobile application, and offered it on pre-loaded handheld devices (‘tablets’) to BRPs/CRPs, so that they could replace pen-and-paper-based data collection, collation, and reporting.
The technology solution appears to have helped. The BRPs/CRPs whom we consulted for the preparation of this note indicated that the tablets helped them with faster and easier filling of data in schools. They now had to carry only a handheld device instead of reams of paper records, and worry less about safely carrying and depositing paper forms in offices. Copying and transferring of data from one form to another as well as data aggregation tasks became much easier. Use of these tablets resulted in quick wins among district and sub-district officials as well. Some of the BRPs/CRPs who had not been regular became regular with their school visits. These qualitative perceptions were further reinforced by official meetings at the district level where the findings from the data were discussed, and show-cause notices were issued to diverging teachers and BRPs/CRPs, signaling that the data were being reviewed and used.
Key factors of success
Acquaintance with, and trust in, mobile technologies helped in the adoption of the tablet: All the BRPs/CRPs consulted were acquainted with feature phones or smartphones. From their use of basic functionalities on feature phones and of smartphone apps, such as WhatsApp, YouTube, and Google, they had the digital competency to browse mobile apps and search information. Those using touchscreen smartphones were also familiar with common interfaces of apps. It helped that the tablets used Android, an operating system common in low-cost touchscreen phones.
Classroom training followed by on-site assistance and personalised guidance played a critical role in ensuring uptake and continued use of the app: While the BRPs/CRPs were used to simple communication and searching apps, they were not familiar with using mobile technologies for collecting data. In this context, on-site assistance from the government was reported as helpful in solving unique issues faced by the BRPs/CRPs, and in ensuring continued use of the app and tablet for data collection.
Lessons for implementation
Plan for poor network connectivity: It is useful to have the feature of both online and offline data collection since most of the areas in which schools are located, especially rural areas, have poor network connectivity. To build trust, it was important that BRPs/CRPs were always able to collect data irrespective of network conditions, without the risk of data loss. However, there were instances of data entries being stamped with incorrect time and date of submission, and/or app losing data after getting stuck mid-use, based on our consultation with the BRPs/CRPs. While the veracity of this information and frequency of such issues is not clear, it was evident that some BRPs/CRPs resorted to manual pen-and-paper-based recording of key data points, in addition to feeding them on the tablet. This could potentially result in doubling of efforts and exert additional burden on an already overworked cadre.
Ensure adequate testing cycle for new tools: Some BRPs/CRPs claimed that they were incorrectly identified for disciplinary action because of incorrect date and time-stamping. This indicates that the testing cycle for the software might not have been sufficiently robust. Technologies, especially when there are indications of emerging successes, lead to a tempting urgency to scale up. However, expanding the user base – without extensive testing, and systems and protocols for adapting and improving – may lead to poorly performing technology solutions. This may even cause long-term damage by disappointing the end users, who would be reluctant to try out technologies in the future.
Engage end users early in the design and implementation process: Several suggestions about the design, user interface, and key features of the app emerged during authors’ consultation with the BRPs/CRPs. These include a seemingly obvious feature to view and use the data being filled out on the devices, so that the BRPs/CRPs could take quicker action based on the data they were collecting. Another feature suggested was to allow for user feedback through the app. Such enhancements, perhaps missed due to late user engagement, can only be made available in the next upgrade cycle of the app.
Consider the operating environment, and invest in systems and processes: Technology, with its seemingly precise and clean quantitative data and improved efficiencies, creates a perception of effectiveness that encourages decisions to be based on the data collected. However, the collection of inaccurate data could move the system from data-less decision-making to inaccurate data-driven decision-making, which can worsen the performance of the system. In this particular case, it was important to identify the motivations that could ensure that BRPs/CRPs will report accurate data, without fear of malicious sanctions from within the system, or reprisals from the teachers.
Integrating technology into educational systems is an intricate, multifaceted effort that involves a series of deliberate decisions, plans, and measures. The journey from concept to implementation starts with a vision, and identifying, precisely, what problem ICT will solve. The work entails understanding the context – identifying, understanding, and adequately preparing the stakeholders, identifying existing infrastructure and associated costs and benefits, and accounting for adequate funding for set-up and maintenance. Once implementation begins, a cycle of monitoring, evaluation, continuous re-evaluation, and improvement must begin. Amidst all this, it is crucial to understand who the end users are, and ensure that their perspectives are integrated into the design, testing, monitoring, and improvement of the solution. Shortcomings in any of the above processes will result in poor uptake and usage of ICT.