Financial inclusion for the poor: Using RCTs for effective programme design

While the Pradhan Mantri Jan Dhan Yojana – the Indian government's flagship financial inclusion scheme - is impressive in its mission, it does not seem to have achieved meaningful results so far. In this article, Ruchira Bhattamishra, Senior Fellow at the Centre for Development Research, Sustainability and Technical Advancement, makes a case for the use of the Randomised Controlled Trial methodology for impact evaluation to gather credible evidence on the effectiveness of pilot financial programmes/particular components of programmes, before the government invests heavily in scaling them up.

The Pradhan Mantri Jan Dhan Yojana (PMJDY) - Prime Minister Narendra Modi's flagship financial inclusion scheme - while impressive in its mission, seems to be unable to translate the high-level commitment it enjoys into meaningful results so far (Boudot and Mowl 2014). Although it has been more than year since its inception, recent reports indicate that of 196 million accounts opened so far, a third have zero balance. Promoting financial inclusion by mandating that banks offer universal accounts may seem like a throwback to previous schemes that have met with failure. The PMJDY mission statement calls for an expansion of mobile banking through Unstructured Supplementary Service Data (USSD) technology¹ that would, in principle, enable access to banking services through simple texts sent using an ordinary mobile phone. Although many among the poor can afford and already own mobile phones, there has been little progress in developing an appropriate legal and regulatory infrastructure for the meaningful expansion of mobile banking.

However, given some of the new design features of the PMJDY such as direct benefit transfers and 24-7 access to savings in customisable amounts via a special 'RuPay' debit card²; impressive technological advances; and the massive political stock that the Prime Minister has invested in the success of the scheme, many continue to hope that PMJDY will achieve much more than previous financial inclusion schemes. PMJDY's success in achieving its ultimate goals, however, warrants the use of the latest methods, not just in finance and technology but also in monitoring, evaluating and fine-tuning the programme. While opinions (for example, see Mint interview with Pranab Bardhan, and I4I conversation between K.P. Krishnan and Nirvikar Singh) on how to make PMJDY a success are widely discussed and debated, it is unfortunate that the role the method of randomised controlled trials (RCTs) can play in achieving the same is being overlooked.

How RCTs work

By providing credible causal evidence on the effectiveness of pilot financial programmes or financial services on small, relevant population groups before settling on a programme design that is rolled out widely, RCTs can inform whether and which elements of PMJDY are achieving their goals. In the basic RCT setup, before a development programme is initiated, individuals are randomly assigned (for example, through a lottery or coin toss) to one of two groups. Given random assignment, both groups are statistically similar. However, only one group (often called the 'treatment group') receives the programme benefit, while the other (often called the 'control group') does not. Any differences between the treatment group and control group detected after the programme is initiated can be viewed as caused by the programme. If designed correctly and implemented well, an RCT can credibly test the effectiveness of the programme. RCTs can also be designed to test the relative effectiveness of a particular component of a programme by randomly assigning these programme components across different treatment groups.

RCT lessons from other countries

While the use of RCTs in international development continues to be discussed within the research community, their power in shaping public policy in developing countries was first highlighted by Oportunidades (originally called Progresa), the Mexican government's main social welfare programme. An independent evaluation of Progresa, commissioned by the Mexican government, found that the programme substantially improved the welfare of poor families (Skoufias and McClafferty 2001). These findings not only influenced the Mexican government's decision to sustain and scale up Progresa but also convinced other governments to adopt similarly-designed programmes, highlighting the role that RCTs can play in shaping policy and improving programme design (Rawlings and Rubio 2005).

RCTs have also provided credible evidence on programmes that have failed to work as desired and thus can help governments in deciding which programmes to stop or reform. Consider the case of the One Laptop per Child programme (OLPC) in Peru. Under the OLPC programme, the Peruvian government bought 902,000 laptops at a total cost of approximately US$180 million. Although it was seen as obvious that providing laptops to every child in school would improve educational outcomes, an evaluation of OLPC indicated that the programme did not improve student test scores in math and reading (Christia et al. 2012). The lessons from the OLPC programme RCT apply broadly, including for financial inclusion: even if a programme seems unambiguously positive and uses the latest, promising technologies to deliver its benefits, rigorous evaluation is needed in order to inform whether it is effective.

These days, governments, non-governmental organisations (NGOs) and aid agencies increasingly implement RCTs to test innovative policies and programmes. One such example of a recent RCT that evaluated financial sector reforms includes an evaluation in Malawi that showed fingerprinting of loan applicants improved repayment (it is currently being scaled up by the Government of Malawi) (Giné, Goldberg and Yang 2012). Another example is the Government of Andhra Pradesh's evaluation of biometric-authenticated Smartcards for benefit transfers which showed that the intervention decreased corruption and led to gains in efficiency of the Mahatma Gandhi National Rural Employment Guarantee Act (MNREGA) (Muralidharan, Niehaus and Sukhtankar 2014).

Exploit the power of RCTs for effective programme design

Now, plans to promote financial inclusion in the next phase of the PMJDY offer an exciting opportunity for the government to use RCTs as an integral part of the process towards arriving at effective programme designs in an efficient way. For example, the Reserve Bank of India (RBI) has announced that it will establish financial literacy centres to promote financial inclusion. Although it seems reasonable to assume that financial literacy programmes are beneficial, recent evaluations find that the impacts of financial literacy on financial behaviour and outcomes are, at best, mixed. For example, an evaluation from India found that financial literacy training for female workers in the informal economy did not make it more likely that they would save (Field, Jayachandran and Pande 2010). On the other hand, another evaluation, also from India, found that a five-week financial education programme made it more likely that participants viewed savings and productive loans positively as well as increased participants' awareness of basic financial products, such as savings accounts (Carpena et al. 2011). So, the evidence on financial literacy programmes, at least the way they are currently designed, is inconclusive, calling out for more evaluations of future efforts so as to clearly align design with objective.

In particular, for populations that are illiterate, the design of financial literacy programmes is important. Consider the recent announcement of the expansion of formal bank accounts in the state of Odisha – home to a large tribal population with very low levels of literacy and socioeconomic development in general (Bhattamishra 2012). Having bank accounts and receiving conventional financial literacy training may result in no or limited benefit to such populations.

Given this base of evidence, new RCTs can be designed to learn about the conditions and pathways that matter for the effectiveness of financial literacy programmes before the government invests heavily in such programmes. And more generally, RCTs can provide credible evidence on the effectiveness of a wide range of design and implementation considerations under the PMJDY scheme in India. Given the Indian government's resolve to promote and achieve meaningful financial inclusion, exploiting the power of RCTs may well be their key to success.

Notes:

USSD technology enables customers to enter a simple code on any handset on any mobile network to launch a basic mobile banking menu, thereby making banking a reality for those not able to easily access brick-and-mortar banks or afford smartphones. It is currently the most cost-effective communication technology providing mobile financial services to low-income customers.
Advantages of the RuPay debit card include increased flexibility in terms of acceptance in all ATMs and other transaction portals throughout India, faster processing of transactions, and lower transaction fees compared to debit cards issued by private banks.