Economist-activist Jean Drèze has argued that economists are no better equipped to comment on development policy design than other social science researchers and other
Jean Drèze’s thought-provoking piece on the complex relationship between evidence, policy, and politics has been widely shared in policy research circles recently. One of the main theses in this article is that development economists (more precisely, quantitative social scientists in development – we will use ‘development economists’ as a general umbrella term to cover all of these, à la Drèze) are no better equipped to comment on development policy design than researchers from other social science disciplines and also other stakeholders. However, development economists continue to hold sway among bureaucrats and policymakers in government as well as big donor organisations in the international development space.
We explore the roots of development economists’ centrality in social policy design. We locate it in the coming together of two key factors: (a) the nature of the evidence they generate, and (b) the top-heavy policymaking paradigm in India.
Policymaking, as Drèze argues, requires more than just evidence. The apparent ubiquity of the phrase ‘evidence-based policy’ must not distract us from the recentness of the phenomenon. Much of policymaking continues to be in the old mould and is not evidence-based. Political preferences, personal ideology and beliefs, anecdotes, etc. continue to play key roles in determining policy.
However, focusing squarely on evidence alone, we make two broad points: one, that a more multidisciplinary approach, even under current constraints, can prove beneficial. Two, evidence-generation can also be a by-product of increased citizen-State interactions, a neglected aspect of policy design in India.
General vs. particular
If phenomena can be broadly classified as the ‘particular’ and the ‘general’, modern development economics has equipped itself with the tools for dealing with the latter.
Consider the following hypothetical policy problem: understanding the impact of a cash transfer scheme being implemented in Bihar at scale. What a well-designed study in economics allows us to state, often with a great degree of certainty, is the impact of cash transfers for the whole of Bihar on a range of quantifiable outcomes. Technically, this is simply the average effect – referred to as the average treatment effect (ATE) in the literature. This is, needless to say, extremely important from a policymaking perspective. Some studies, depending on sample sizes and design, may be able to tell us more: for example, the impact on households residing in Sitamarhi district, or the impact on women. Fewer still, can tell us even more, without compromising on precision – for instance, the impact on women in Sitamarhi. In practice, effects can differ across dimensions of caste, religion, class, and institutional setups, to name only a few. Furthermore, even the outcomes measured may be limited to those broad (albeit, again, important) outcomes amenable to precise capture in quantitative metrics – consumption expenditure, nutritional intake, or transaction costs. More amorphous outcomes like impact on local politics, caste alignments, or even the local economy may be harder to grasp. Thus, in the way it is practised, modern development economics is not equipped with the tools required to give us a very particular sense of the effect of a policy: identifying distributional consequences over complex outcomes requires a degree of nuance that is largely beyond the grasp of the average effect1.
Indian social sciences beyond economics have a long tradition of grasping complex social realities in small geographies. M.N. Srinivas pioneered ‘village studies’ and also the uniquely Indian discipline of social anthropology. Many of Srinivas’ key insights – the concepts of ‘vote banks’ or (the more contested) ‘sanskritisation’ – have surprising generalisability. These came from a deep engagement with a handful of villages, but now constitute accepted wisdom. Thus, while this line of research and work is equipped foremost to deal with the ‘particular, that is, to characterise phenomena occurring over smaller areas, it can, on occasion, attempt to theorise and shed light on issues occurring over wider geographies.
Centralised policymaking privileges ‘general’ over ‘particular’
In light of the above, what is the reason for the primacy accorded to quantitative data in informing (social) policies? It is our belief that the centralised policymaking regime in India prioritises the ‘general’ over the ‘particular’.
Independent India has had a history of centralised planning. The early planning processes adopted in India owed in no small measure to P.C. Mahalanobis, a statistician by training. While many aspects of social policy in India’s federal set-up are meant to be designed and implemented by state governments, much of it – including more recent social protection initiatives such as the Mahatma Gandhi National Rural Employment Guarantee Act (MNREGA) and the National Food Security Act (NFSA) – remains, for the most part, centrally conceived and designed. State-level development bureaucracies – and most certainly everyone below – are largely reduced to mere implementing agencies, focused on following rules set by agents above (Mangla 2015).
In a centralised policymaking set-up, it is natural that the ‘general’ will be privileged over the ‘particular’ for two reasons. One, policies are often designed for extremely large populations and this necessitates erring on the side of one-size-fits-all policy design. Two, the distance of the policymaker from the larger target population allows him/her the unfortunate luxury of being isolated from the consequences of the heterogeneity in average effects of the policy. Put differently, the voices of those adversely affected by the policy find it harder to reach this distant policymaker.
There is nothing natural about this centralised policymaking setup. Besides historical and political economy2 factors, it is also a consequence of a lack of resources and State capability at the local level (Dasgupta and Kapur 2017) to ideate, pilot, iterate, dynamically adapt, and generate evidence. This poses a tremendous burden on policymakers at higher levels, for the breadth of their policy activities is immense. Given the pressures most policymakers work under, the reduction of evidence to ‘numbers’ proves extremely seductive. It is much easier (and considerably quicker) to weigh policy alternatives if you can just put them on a simple balance. This is what other non-quantitative types of evidence do not allow for very easily.
The role of multidisciplinary evidence
This status quo, however, is not an ideal state of affairs. Taking Drèze’s point further, in the circumscribed set of cases where the ‘bridge between evidence and policy' is functional, evidence-gathering requires a marriage of sorts – bringing together qualitatively and quantitatively trained researchers, and civil society members. There are at least three ways in which qualitative researchers (this could also potentially include local civil society members) can significantly contribute to policy design.
First, in understanding distributional consequences: as we argue above, qualitative research brings nuance and texture to both the discussion of
Second, mechanisms of impact: if quantitative evidence can give us a big-picture understanding of what works, then
Anthropologist Jefferey Witsoe’s careful work (2013) in Bihar serves as an example here. His primary objective is to study the impact of the policies – or lack thereof – of the controversial former Chief Minister, Lalu Yadav. On the face of it, the numbers tell a bleak story: growth stalled – and briefly reversed – in Bihar, while it soared in the rest of the country, propelled by the recently unleashed forces of liberalisation. Bureaucracies were severely understaffed; their morale suffered a significant blow. Crime thrived, particularly in urban areas – this period in the state is often referred to by sorry epithets such as ‘jungle raj’ and ‘goonda raj’.
Witsoe’s work allows us to peel layers underneath these broad facts and numbers. Using testimonies gathered from marginalised groups (backwards, Dalits, and Muslims) over years of fieldwork in a few villages in Bihar, he argues that Lalu Yadav’s political methods had complex distributional consequences. In particular, it significantly improved the relative well-being of the poor and the vulnerable. Moreover, he argues, this improvement was not merely in terms of income, but it refers to something more amorphous, for the nearly 15-
To understand the mechanisms that led to this impact, Witsoe uses data gleaned from months spent in a Block Office – the lowest nodal bureaucratic outpost prior to 2001 in Bihar. Lalu Yadav systematically weakened the upper-caste dominated bureaucratic State and, consequently, directly strengthened local politicians. Democracy conflicted with development, evocatively emphasised in another party slogan of the times: “
Witsoe’s work is both insightful and atypical, given its sweeping nature. We are not arguing here that all qualitative work can unearth structural issues in the manner that
Third, qualitative work can help unpack anomalies in the data. Anomalies – or outliers – form classic examples of the ‘particular’ informing the ‘general’. Drawing an example from the discipline of neuroscience,
An analogous hypothetical example in social policy can be constructed: imagine that the policy of cash transfer works well on average, but fails spectacularly (predicted quantitative impact, although noisy, is highly negative) in three blocks in Samastipur district. A qualitative researcher with deep knowledge of the context could help shed light on the reasons for doing so: it could be something unsystematic or unchangeable – dumb luck, or a recent change in personnel, or the nature of the caste composition of the area. It could also be something fixable, like corruption. Perhaps, the local bureaucrats in those three blocks have ‘cracked’ the new cash transfer system, indulging in practices that reap immense benefits to themselves at the cost of the poor. These officers can be outliers, but there is a chance that the incident could also serve as a sign of things to come, as their peers learn from them: in either case (though especially in the latter scenario), this would call for tinkering with policy design or major policy change.
Citizens’ voice as evidence
We have discussed what may be considered rigorous forms of evidence – both quantitative and qualitative – and their value and place in policymaking. We propose that citizens’ experience and participation are forms of ‘evidence’ as well. For instance, the ‘particular’ experiences of some citizens with respect to mandatory Aadhaar4 authentication for accessing benefits such as the public distribution system (PDS) rations in some states are not in any way less valuable than abstract (and problematically calculated) metrics of ghost beneficiaries.
From a policymaker’s perspective, the issue with this form of evidence is to separate
Grievance redressal mechanisms form one potential channel6 A well-functioning grievance redressal mechanism – where citizens use the platform to consistently air and solve their issues with various arms of the State – can allow for the flow of a rich stream of systematised evidence. Viewed in the aggregate, these could form inputs for policy design. For instance, a disproportionately large number of complaints made in Darbhanga district regarding the PDS could signal something systematically wrong with procurement or distribution of grains. Grievances from female complainants could be distinct from those from male ones. Thus, a well-functioning MIS (management information system) on grievances could curate complaints, and from the policymakers’ perspective, help shape policy tweaks and design.
Social movements, civil society groups or even more traditional NGOs (non-governmental organisations) offer a second channel for curating and amplifying citizens’ voices – especially in flagging issues related to operations7. A recent example of policy change is the withdrawal of the Direct Benefit Transfer (DBT) scheme for food subsidy in Nagri block in Jharkhand after nearly a year of civil society action (organised by the Right to Food Campaign in this case). This was followed by a social audit conducted by the state government’s Rural Development Department. The civil society
In general, social policy is almost never set in stone. Good policy design allows for dynamic changes and, crucially, incorporates listening to
In sum, insofar as we have centralised policymaking and an over-burdened policy apparatus, the ‘general’ will be privileged over the ‘particular’ and development economists will continue to generate the kind of evidence that influences policy. However, this does not preclude the need for much greater State engagement with a wider set of researchers, cutting across disciplines in social sciences. This is particularly true for researching
The authors are grateful to Dipa Sinha, Niharika Singh, and Nandini Gupta for their incisive comments.
- This is not to say that quantitative work in development cannot give us a sense of the particular: Drèze and Khera’s surveys often reveal new – and quite nuanced – trends in access to public services. This is partly because the inferences made are rooted in lived experiences in the field. The nascent sub-field of behavioural development economics also makes very particular claims. Similarly, a carefully grounded structural model can help make the leap from some general claims (reduced form estimates – from a randomised controlled trial (RCT), for instance) to a more nuanced understanding of mechanisms and distributional outcomes.
- Centralised policymaking allows for centralisation of credit (for its success; though not always blame for its failure), contributing in turn to potentially favourable electoral outcomes. To put it simply, decentralised policies are not suited to branding under a party or a political leader’s name and hence the returns to a successfully implemented policy are higher the higher one goes up the political or bureaucratic ladder.
- Indeed, this loose framing of Witsoe’s work as an ‘impact evaluation’ of the Lalu Yadav years is a device we use strictly for expositional purposes. It is not our claim that this was at all the original intention of his work.
- Aadhaar or Unique Identification number (UID) is a 12-digit individual identification number issued by the Unique Identification Authority of India (UIDAI) on behalf of the Government of India. It captures the biometric identity – 10 fingerprints, iris and photograph – of every resident, and is meant to serve as a proof of identity and address anywhere in India.
- We are not claiming that more rigorous forms of evidence are free from bias, only that the signal is clearer and knowledge more systematised.
- (i) Informal grievance redressal mechanisms, (ii) scheme-specific formal grievance redressal mechanisms, and increasingly, (iii) state-wide formal channels like the Bihar Right To Public Grievance Redressal Act (BPGRA) in Bihar do exist but have had variable levels of success. We are envisaging a mature version of the
- A free press plays a role in systematising evidence by conducting fair investigations, mediating between the citizens and the State, and informing public opinion.
- Dasgupta, A and D Kapur (2017), ‘The Political Economy of Bureaucratic Effectiveness: Evidence from Local Rural Development Officials in India’.
- Drèze, J (2018), ‘Evidence, policy, and politics’, Ideas for India, 3 August 2018.
- Duflo, Esther (2017), “Richard T. Ely Lecture: The Economist as Plumber”, American Economic Review, 107(5).
- Mangla, Akshay (2015), “Bureaucratic norms and state capacity in India: implementing primary education in the Himalayan region", Asian Survey, 55(5): 882-908. Available here.
- Ramachandran, VS, S Blakeslee and N Shah (1998), Phantoms in the brain: Probing the mysteries of the human mind, William Morrow, New York.
- in (2018), ‘Jharkhand withdraws direct benefit transfer scheme for food subsidy after problems in implementation’, 8 August 2018.
- Witsoe, J (2013), Democracy against development: Lower-caste politics and political modernity in postcolonial India, University of Chicago Press.