While a lot of funding goes towards community-driven development projects, rigorous evidence on their socioeconomic impact is limited. This column evaluates the impact of JEEViKA – a rural livelihoods project in Bihar that seeks to empower marginalised women by organising them into self-help groups. It also highlights the importance of understanding how these initiatives work, and the challenges involved in evaluating their impact.

Recent poverty reduction strategies rely on a community-driven approach. By mobilising the poor into institutional platforms, policymakers hope that the poor can articulate their demands better. In response, iterative, sustainable interventions can be designed that are ‘of’ the poor, ‘by’ the poor, and ‘for’ the poor. Indeed, various states in India have initiated such projects in the last decade. In Bihar, the Bihar Rural Livelihoods Project known as JEEViKA) became operational in 2007 with funding from the state government and the World Bank. JEEViKA’s mandate is to mobilise women from impoverished households into community-based organisations such as self-help groups (SHGs); once such organisations are made sustainable and federated into Village Organisations (VO), JEEViKA progressively rolls out different baskets of interventions such as microcredit, livelihood opportunities, and convergence with other government schemes, on a demand basis. It is expected that the mobilisation of socioeconomically marginalised woman into women’s groups would enable them to reach higher levels of empowerment, both with respect to the household and the community. The multi-billion dollar National Rural Livelihoods Mission (NRLM), funded by the Government of India and the World Bank, adopted a similar approach to poverty reduction and became operational in 2012. Over the next decade, NRLM aims to reach out to 600,000 villages across India for the formation of women organisations and subsequent poverty reduction by a variety of interventions.

In spite of the substantial funding of such community-driven development (CDD) projects, rigorous evidence to understand their impact on socioeconomic inequality is rare. However, that has not been a stumbling block in the replication of these models in a variety of sociopolitical contexts. In this column, I present results from an impact evaluation of JEEViKA. The design of the study is not necessarily gold standard. However, given the paucity of evidence on the success of CDD projects, especially in the context of Bihar, this exercise would hopefully shed some light on the efficiency of this particular strategy to deliver benefits to marginalised households.

How JEEViKA works

The basic building block of the project is to promote socioeconomic inclusion of rural impoverished households by mobilising female members from such families into SHGs. In an average SHG, 10-15 women meet regularly to participate in savings, borrowings and repayments. Additionally, it provides a small platform for these women to come together and discuss their day-to-day lives. Each member saves about Rs. 5 to Rs. 10 per week. The members start inter-loaning among the group by drawing on the aggregate savings parked at the SHG, and repayments are ensured by peer pressure. After such practices are sustained for around 5-6 months, the project provides the SHG with a one-time grant of Rs. 50,000. The SHG disburses this fund as loans to members on a demand-priority basis, whereby each member puts in her demand, but the SHG members jointly decide the prioritisation of demands. Once the initial repayments come in and the corpus grows back, the SHG disburses loans to members who were not included in the ‘first priority’ list. Going forward, JEEViKA facilitates these SHGs to get linked to banks and leverage additional funds from formal credit institutions. The cost of such credit is usually 24% on an annual basis, compared to 60-120% annual interest rates if the money was borrowed from moneylenders, shopkeepers or other informal sources. Once 10-15 SHGs form in a village, they are federated into a VO. The VO drives the next level of interventions in livelihoods (provision of funds, inputs, training of farmers from beneficiary households etc.) social development (procurement of grains at wholesale rates, specific loans to cover health emergencies, etc.), and entitlements (better access to the Public Distribution System (PDS), Mahatma Gandhi National Rural Employment Guarantee Act (MNREGA), pensions, etc.). The VO also has a social mandate to identify issues at the village level and liaise with the project’s staff to provide practical solutions.

Evaluating the impact of JEEViKA

JEEViKA started operations on a pilot basis in 2007 in six blocks (in six districts, namely, Gaya, Khagaria, Madhubani, Muzaffarpur, Nalanda, and Purnia). In 2008, the project expanded to 12 more blocks in these districts. 200 villages which were entered by JEEViKA in 2008 were identified as the ‘treatment’ group. JEEViKA had started expansion in 24 ‘phase 2’ blocks (again, in the same six districts) in 2010. Villages in these 24 blocks where JEEViKA had not entered as of late 2010 were identified as possible ‘control’ villages. Census 2001 data was then used to select the final 200 villages for the control group, which were closely matched to the 200 treatment villages in the incidence of Scheduled Caste/Scheduled Tribe (SC/ST) population, among other parameters. A questionnaire capturing various socioeconomic parameters (example, debt roster, asset roster, women’s mobility and decision-making powers, etc.) was fielded to 10 randomly selected households from each village in early 2011. This questionnaire had a retrospective module to mimic a baseline scenario at the beginning of 2008, for appropriate parameters. Finally, suitable econometric methods were used to match sampled households from the 200 treatment villages with those in 200 control villages to estimate the impact of JEEViKA over a three-year period (early 2008 to early 2011).

Looking at all loans that were outstanding during early 2011, I find:

• Compared to an outstanding amount of Rs. 14,500 on an average high cost loan (where high cost is defined by a monthly interest rate of 4% or greater) for a control household, the corresponding burden for a treatment household was Rs. 5,000. 
• Additionally, for every Rs. 100 borrowed, the treatment household borrowed around Rs. 7 more for debt reduction or productive purposes, as compared to control households.

A higher percentage of treatment households owned mobile phones, watches, cows and bullocks; however, I did not find any evidence of higher ownership of land, buffaloes or other livestock, and jewellery or other assets. There was a slight reduction in open defecation and a slight improvement in food security conditions among treatment households.

When we consider indicators of women’s empowerment, I find:

• 9% women in treatment areas went to panchayat meetings, 5.5 percentage points higher than control areas.
• 79% women in treatment areas provide an opinion about the education of their children, 10 percentage points higher than control areas.
• 83% women in treatment areas would or have acted to prevent domestic violence, 9 percentage points higher than control areas.

Leveraging the cheap credit provided (or facilitated) by JEEViKA, beneficiary households were able to reduce their debt burden; an encouraging sign was that the cheap credit was not used to fund consumption uses. Instead, beneficiary households used the funds for productive investments, as evidenced by a more diverse asset base. I find no evidence of diversification of livelihood activities undertaken by household members. Due to the retrospective nature of the questionnaire, questions on income or consumption values were not included, both for 2011 and 2008. Finally, the mobilisation of socioeconomically marginalised women into organisations improved their capability to take control of their lives.

Limitations of the evaluation design

The retrospective questionnaire imposes severe limitations on the analysis, due to the chance of errors in recalling the past. Since outcome levels in the past are an important part of any ‘before-after’ analysis, retrospective surveys should be sparingly used. If used, only questions which may be recalled with greater accuracy should be asked. In general, it is worth considering whether popular evaluation strategies, including Randomised Controlled Trials (RCT), are appropriate for assessing the impact of such projects. For example, in these contexts, the entering of a village by the project does not imply delivery of any benefit (apart from very small inter-loans); substantial exposure happens only 6-8 months down the line when SHGs get capitalised and a VO comes into existence. Studies need to account for such durations while designing the evaluation horizon.

The basket of interventions offered by such projects requires a very comprehensive questionnaire to capture the possible causal impacts. Respondent and/or interviewer fatigue may seriously compromise the quality of responses gathered via such questionnaires. An alternative may be to collect data only on outcomes and not processes (example, collecting data on income/consumption, and not debt, livelihood activities, etc.); this is a risky, because it prevents any understanding of the catalytic effect that different interventions may have on one another in engendering socioeconomic change.

Qualitative surveys which enquire deeply into such pathways are very useful in these cases. Given that the interventions are gradually phased in, such surveys can shed light on the interaction effect of the different interventions. Most importantly, such surveys can provide insights into the communal environment changes that take place in response to these interventions; this understanding is very useful for policymakers and social scientists who seek to replicate such strategies elsewhere.

Unfortunately, qualitative surveys cannot cover a large sample due to various constraints of time and human resources. Thus, combining qualitative and quantitative methods can help disentangle the pathways and effects of such multi-pronged projects, and provide answers to ‘what happened’ and ‘how it happened’. The former is after all essentially a post-mortem analysis; it is the latter that development practitioners should actually be looking at.