Impact is not equal, but it is equitable

Not all lived realities can be captured quantitatively. But some successful evidence-based approaches are working to decolonise impact evaluation.

In discussions on locally led development, the focus tends to be on power dynamics, decision-making and hierarchy, mostly in relation to resource mobilisation. While this is central to the debate, it is also essential to decolonise from the start to the end of the development process, and this is the domain of monitoring and evaluation (M&E).

The current development space’s monitoring and evaluation frameworks, practices and impact evaluations are based on perspectives from donor countries and their ecosystem of privileges. What is currently not considered is the context of the local community where funding is mobilised.

A biased cycle

Quantitative monitoring and evaluation measures that are widely used carry various degrees of implied bias and look at impact through a siloed lens. The knowledge of community experts, which includes an understanding of crucial intersectional nuances, is often not captured by M&E requirements. This has a knock-on effect on budgeting, which then becomes based on an understanding of impact that fails to include the on-the-ground context of communities and grantee or partner organisations. This has resulted in organisations working in extreme conditions and unsustainable approaches to organisational resources.

Short-term impacts

The standard approach to impact evaluation has wide-reaching consequences. It impacts grantee partner culture, human resources and community practice. Grassroots organisations have no choice but to model themselves on donor standards, and this can result in local organisations amplifying marginalities and engaging in short-term impact to ensure their organisation survives, even if these activities do not fit with organisational goals or community needs. When it comes down to the detail, there is a fundamental issue with the way impact processes are designed.

The overall impact process is designed during the resource mobilisation stage, while sub-impact processes are designed by partner organisations during their application process. Since the main impact indicators are pre-designed without the understanding of the local people who the funds will be used for, the indicators of what success will look like tend to be globally generic.

But what does success mean in different contexts? How does the same issue look different in different communities? These are the kinds of questions that should guide the impact design and evaluation process, but rarely do.

Challenging the status quo

The lived realities of impact designers and evaluators from high-income countries are different from the lived realities of the communities being supported. The history and trauma that has led to this power imbalance and been carried forward must be reflected upon and examined.

While this should be fundamental to locally led development, we are yet to see this in practice in impact evaluation design and accountability. If grassroots organisations do not meet the standard metrics of evaluation, this is seen as failure, resulting in much-needed resources being cut off.

Valuing qualitative data

There is a critical need for data-driven decision-making, but data can be qualitative as well as quantitative and process design must explore both types of data. Behind each statistic is a human life with complex, overlapping identities. While an aspect of someone’s identity may be a focus area or a social issue for a donor, it is a lived reality for that person and can’t always be captured quantitatively. There are successful evidence-based approaches that work in monitoring and impact evaluation. Intersectional feminist movements have been using them and are driving sustainable impact.

So, how do we work towards equitable impact?

• Use a participative approach, integrated at the design stage of the grant or resource mobilisation.

• Donors should establish a measurement of impact by co-designing processes with potential or past grantees, experts, and the communities the intended resources are for.

• Impact evaluation must be a two-way street, from bottom up and top down, meeting in the middle.

• From the teams designing the evaluation framework to those facilitating the process, members must be diverse and from the local context.

The above steps should be integrated during the programme design process, where the team uses reflective questions during the donor organisation’s strategy review, such as those we will be using in Bond’s upcoming Action Learning Sets.

Bond’s Action Learning Sets

Using the Bond guide: Becoming locally led as an anti-racist practice as a basis for discussion, Bond will be leading Action Learning Sets (ALS) to provide a space for Bond members to explore how they are shifting mindsets and practices to become more locally led within their organisations.

Bond’s Action Learning Sets will serve as a guiding charter to re-think the current resource mobilisation process and come with a user-friendly and practical outline. As such, they act as a catalyst for locally led development. The ALS will be a space where members can share the challenges that they are grappling with, achievements and progress that they are making, learnings from shifting practice and importantly, they will provide a support network to collaboratively navigate our functional and organisational next steps.

The first two ALS will start in the autumn and will focus on Programmes and Partnerships and Funding and Relationships with funding partners. If you are interested in finding out more, join our Practice for locally led development working group.

A decolonising culture

Culture eats strategy for breakfast. To have a decolonised strategy of what impact is can help to induce a decolonising organisational culture. This is also made possible through the recruitment process, which must be inclusive and accessible.

There is a critical need for a global framework for decolonising impact evaluation. Current schools of thought on M&E must include learnings from locally led development. If not, the M&E space will continue to be colonial.