On October 2, the government will host the UN Secretary General and other international delegates at the Mahatma Gandhi International Sanitation Convention, to celebrate India’s progress on ending open defecation. In this post, Srivastav and Hathi contend that while the recently released National Annual Rural Sanitation Survey data seem to show that open defecation is being eliminated from rural India, this conclusion is unfortunately premature.
India’s Swachh Bharat Mission (SBM) began in 2014 with the ambitious goal of bringing widespread open defecation (OD) down from close to 70% of rural Indians to zero by 2019. The National Rural Sanitation Survey (NARSS), conducted by the Ministry of Drinking Water and Sanitation (MoDWS) and funded by the World Bank, found that as of March 2018, 77% of rural households had access to toilets, and that 93.4% of individuals in those households use their latrines regularly. To demonstrate the accuracy of these figures, the MoDWS often compares these numbers to a second national survey conducted in 2017 by the Quality Council of India, a society created by the Government of India, which also found similar access levels (63%) and usage levels above 90% (Livemint, 2017, Kattakayam 2017). Ministry officials have pointed out that both survey’s figures are similar to the figures in the MoDWS’ Management Information System (MIS) (Joshi 2018).
In a country that has struggled for decades to address poor sanitation, the director of SBM recently stated that “sanitation coverage in rural India has more than doubled” between the start of SBM in 2014 and today (Joshi 2018). Similarly, the Secretary of the MoDWS said that India is 11 years ahead of schedule to fulfill its share of meeting the UN’s (United Nations) Sustainable Development Goal on sanitation. Ordinarily, such success would be cause for celebration. Unfortunately, there is no up-to-date, independent measurement of OD in rural India against which to measure these reports.
Comparing OD-free declaration with NFHS-4 data
National Family Health Survey (NFHS)-4 is India’s fourth round of Demographic and Health Survey (DHS), a high-quality, internationally comparable survey project that collects vital demographic data around the world. NFHS-4, which collected data in 2015 and 2016, found that over half of rural households defecate in the open. These results contrast with those of the NARSS, conducted just months later.
The MoDWS has publicly dismissed the quality of NFHS-4, saying that its sanitation numbers are inaccurate (Joshi 2018)1. But several indicators demonstrate that the NFHS is broadly sound. First, as one would expect of truly objective data, and as is true across datasets and other health indicators in India, the NFHS-4 shows considerable heterogeneity in OD across states. Additionally, this heterogeneity is correlated with causes and consequences that are plausibly related to OD. For example, a large body of literature has demonstrated that open defecation is correlated with other health indicators like child height, a correlation that is seen using NFHS-4 data (Coffey and Spears 2018a), as well as DHS surveys across countries (Spears 2018).
The government has tried to explain the difference between the NARSS and NFHS-4 findings by claiming that the NFHS does not capture SBM’s most recent activity. While this is possible, a recent study by Coffey and Spears (2018b) found that over the 23-month course of NFHS-4 data collection, progress in reducing OD took place at a much slower pace than would be required to meet SBM’s 2019 deadline. The study also found that district-level OD outcomes were highly correlated with latrine coverage statistics from the 2011 Census. It is possible that the SBM became much more productive after the end of NFHS data collection. Plans for NFHS-5 are underway, and should provide reliable and up-to-date data on OD.
NFHS-4 allows verification of ODF status for districts that were declared ODF before or shortly after they were surveyed. Table 1 shows the percent of rural OD measured by NFHS-4 in districts that were declared ODF by the government between three months before and three months after when they were surveyed. During this period, 18 districts were declared ODF. Seven of these were in Kerala, which already had 95% latrine coverage in the 2011 Census. Outside of Kerala, however, NFHS-4 found high percentages of households without latrines in places that were, or were about to become ODF. For example, Ajmer district, Rajasthan, was declared ODF in July 2016, but NFHS-4 data collectors were still surveying the district that month, and found approximately 39% OD at the household level. NFHS-4 data collection was completed in Narmada district, Gujarat in April 2016, where 64% OD was found, and yet the district was declared ODF just three months later. It is noteworthy that although Himachal Pradesh was declared the second ODF state in the country in 2016, seven out of the state’s 12 districts are listed in the table below. An even more recent study found 49% OD in rural Rajasthan in October 2017 (Performance Monitoring and Accountability 2020, 2017), but the state was declared ODF just six months later in April 2018.
Table 1. Districts declared ODF between three months before the end of NFHS data collection and three months after
|State||District||Month of final NFHS-4 data collection (2016)||ODF month (2016)||Difference in months||Fraction of households reported as practicing OD by NFHS-4|
|(ODF declaration vs. NFHS presence)|
|Himachal Pradesh||Lahaul and Spiti||August||July||-1||12%|
ODF declaration in practice
Why did the NFHS find such high percentages of OD in districts that were being declared ODF by the government? Understanding the process of ODF declaration helps answer this question. In most places, SBM committed to subsidising a latrine for households that were listed as not having one in a 2011-12 government survey. In practice, local officials declared villages ODF if latrines had been built for all of the households in the 2011-12 list2. This method is problematic for three main reasons. First, ODF status is declared based on latrine ownership, rather than latrine use, which is what matters for improving health. Second, sarpanches 3, and block and district officials have been under enormous pressure to get latrines built rapidly. Faced with unrealistic targets and timelines, many have a strong incentive to report latrine construction even if some latrines are incomplete or non-existent. Finally, the 2011-12 survey excluded many households, including all new households that formed between 2011-12 and the present. According to a recent study by the Accountability Initiativein Udaipur, Rajasthan (Deshpande and Kapur 2018), “GP and block level officers reported that the baseline numbers were not reliable.”
It makes sense, then, that districts in which many households do not use latrines could nevertheless be declared ODF through such a process. But how did a large sample survey like the NARSS, implemented by a research firm, show such high rates of latrine ownership and use?
National Rural Sanitation Survey
NARSS appears to follow many of the best practices in survey research. For example, MoDWS made efforts to include sanitation and survey research experts in the process of designing the survey (including both authors). Experts were asked their advice on how the survey sample should be drawn, and how specific survey questions should be worded. These are important aspects of survey design that must be done well if a survey is to produce informative numbers.
Experts, ourselves included, were also invited to attend the NARSS’ surveyor training sessions. What happened in the training helps explain the survey’s results. The specific goal of reporting low OD and high latrine use, particularly in villages that the government had already declared ODF, was clearly communicated by the Ministry representatives to the survey company, and by the survey company to the surveyors. Although good data depends on getting the technical details right, what we saw during the training demonstrated how data can be shaped according to objectives other than wanting to learn the truth.
A government representative told the survey company’s head trainer that NARSS survey data should match the MIS: if it did not, the company would have to go back to villages in which they found OD that did not match the MIS. From the perspective of the survey company, this would be an extra expense, thereby disincentivising the survey company to report OD if it were found. As described above, from ground-level reports and from the experiences of previous sanitation campaigns, we know that MIS reporting is based solely on construction, not use, and is often inflated because local officials face intense pressure to meet targets.
When the surveyors were introduced to the survey’s purpose, structure, and questions, the trainer opened by suggesting that he hoped the survey would bring back evidence of a successful programme, rather than by emphasising objectivity and detail-oriented recording of facts. From a research perspective, it is inappropriate to tell surveyors what they should find. Subsequently, the trainer went on to instruct the surveyors to mark ‘no OD’ even if they saw feces on the ground: “Let’s say that when you go the place where people defecate in the open, you see some [faeces] on the ground. It could be that some kid did this today, but it is not there every day. It might also be that someone was passing that way and had to [relieve himself], so he [defecated] there. That’s not the fault of anyone from this village. If you mark ‘yes,’ then you’ve tarnished the reputation of the entire village. In such a situation, we should code ‘no OD.’” Similarly, surveyors were instructed not to accept respondents’ reports of not owning a latrine without intense follow-up questioning.
Biasing information and instructions were also programmed into the survey software on the surveyors’ tablets. For instance, surveyors were told before they arrived in a village whether it had been declared ODF, despite advice against providing surveyors with this information from experts at several organisations. Additionally, if a surveyor marked a household as not owning a latrine, the software prompted: “Ask again properly, and check whether this household has a latrine or not.” No such double checks or suggestions of initial impropriety were made when the surveyor marked that a respondent owned a latrine.
By the time the surveyors were on their way to a village to practice the survey, they had internalised what was expected of them. As the train pulled out of a station near Patna, one surveyor looked out the window and saw a man going to defecate in the open. He said: “Look – we are going to do some work about OD and look at how this guy is going out.” A fellow surveyor responded: “If one man is doing it, that doesn’t mean anything. If the whole village or the whole society is doing it, only then do we consider it. If only a few people are doing it, then it doesn’t matter.”
Even though many experts tried to help craft a NARSS that would produce accurate data on sanitation progress, the various forms of pressure to find latrine ownership and use meant that honest and objective data collection would be difficult, if not impossible. The pressure to report favourable outcomes can be seen in the conspicuous uniformity of reported latrine use among latrine owners across states: latrine use among owners is over 90% in Andhra Pradesh, Madhya Pradesh, Uttar Pradesh, Bihar, and Maharashtra. We hope that a truly independent survey, free from bureaucratic influence, can provide an updated estimate of rural OD soon.
From examining the available data, and from field visits in several states, we believe that SBM has accelerated the decline of OD, possibly significantly, which is essential for improving health. Although not the focus of this post, we and others have also found that it coerced and threatened many people in the process. Finally, SBM did not fully eliminate OD, and it is using the tools of statistics to claim that it achieved much more than it likely did. What matters now is what happens next. OD in rural India is not over, visiting dignitaries notwithstanding. The question after October 2 is: how can sanitation policymakers set aside these statistics to focus on sustaining the gains that were achieved, reversing any harms that occurred along the way, and counting and reaching the people who still defecate in the open?
- If anything, the NFHS-4 data likely underestimates the extent of OD that was present in rural India in 2015-16 because it asked about sanitation behaviour at the household level rather than at the individual level, so it does not account for the fact that some people defecate in the open even if others in their household use a latrine. A 2015-16 study in Dharmpuri district in Tamil Nadu found that more than half of adults reported practicing OD despite having a latrine in their home, and that the “mere presence of toilets in households does not deter people from defecating in the open” (Yoganath and Bhatnagar 2018).
- ODF declaration is distinct from ODF verification. Declaration is done by local officials; verifications are to be done by either teams from other villages, blocks or districts, or third parties (MoDWS, 2015). In practice, the verification process has been much slower than the declaration process.
- Sarpanch is the elected head of a Gram Panchayat.
- Census of India (2011), ‘Availability and type of latrine facility: 2001-2011’, Data sheet.
- Coffey, Diane and Dean Spears (2018a), “Child height in India: Facts and interpretations from the NFHS-4, 2015-16”, Economic & Political Weekly, LIII(9): 10-13.
- Coffey, Diane and Dean Spears (2018b), “Open defecation in rural India: Levels and trends in the NFHS-4”, Economic & Political Weekly, LIII(9): 10-13.
- Deshpande, D and A Kapur (2018), ‘Unpacking the processes of achieving open defecation free status. A case of Udaipur, Rajasthan’, Research Report, Accountability Initiative, August 2018.
- Joshi, Yugal (2018), “The real status of rural sanitation”, Economic & Political Weekly, LIII(28): 100-101.
- Kattakayam, J (2017), ‘Swachh Bharat is largest behavioural change programme … 250 million people no longer practice open defecation’, Times of India, 29 September 2017.
- Livemint (2017), ‘63% of rural households have access to toilets: survey’, 9 August 2017.
- Ministry of Drinking Water and Sanitation (2015), ‘Guidelines for ODF Verification’, Swachh Bharat Mission (Gramin) Division, Government of India, 3 September 2015.
- Performance Monitoring and Accountability 2020 (2017), Indicators for Water, Sanitation, and Hygiene.
- Spears, Dean (2018), “Exposure to open defecation can account for the Indian enigma of child height”, Journal of Development Economics, in press.
- The Hindu (2018), ‘Swachh Bharat progress key to UN SDG on open defecation’, 7 September 2018.
- Yoganath, Nallathambi and Tarun Bhatnagar (2018), “Prevalence of open defecation among households with toilets and associated factors in rural south India: an analytical cross-sectional study”, Transactions of the Royal Society of Tropical Medicine and Hygiene, 112(7): 349–360.