TNRC Introductory Overview Monitoring, evaluation, and learning for anti-corruption projects: What conservation practitioners need to know

Anti-corruption programming is rarely linear, and sustainable change can require many years to emerge. Political contexts can change; important actors can cycle in and out of positions of authority. Also, anti-corruption programming is relatively new to the conservation and natural resource management (NRM) sector. This makes the role of monitoring, evaluation, and learning (MEL) for developing an evidence base critically important.

Conservationists and NRM practitioners looking to address corruption that affects the objectives of their work will need to develop comprehensive MEL plans that employ a range of evaluative concepts and methods, as well as ethical procedures common to research activities. Fortunately, there is a wealth of information available on both time-tested and experimental approaches. This introductory overview provides a summary of approaches that program designers, program managers, and MEL specialists should consider when planning MEL. This overview does not address project design considerations. For thoughts on theories for how anti-corruption change happens, please see this U4 paper and watch the Targeting Natural Resource Corruption (TNRC) Knowledge Hub for forthcoming model results chains.

Anti-corruption interventions should start with developing a fully articulated theory of change (ToC) supported by evidence about the corruption problem, including the corruption drivers. But the ToC is a living document. Throughout the process, teams should document questions and assumptions and continually reflect to make informed decisions. As well, the best anti-corruption ToCs are created through equitable processes that bring a variety of perspectives to the discussion. For more on this see the information provided below on participatory processes.

Start with a definition, but don’t be afraid to refine it

Definitions are always important for strong MEL, but for anti-corruption work, deciding how the project will define the term “corruption” is highly consequential. While “abuse of entrusted power for private gain” is the most common definition used, the term “corruption” is used by different people in different contexts in different ways. The choice of definition, such as focusing on individual corrupt behaviors or institutional pressures to be corrupt, heavily influences the direction a project will take and therefore what gets monitored and evaluated (Williams 2021).

The definition will also play a role in any context analysis the project conducts, which will be vital to establishing baselines and understanding contextual factors like the political dynamics of different actors. Important risks, entrenched interests, and potential obstacles are just a few of the important aspects of the program’s context that the MEL plan will ideally monitor. Doing so helps ensure program leadership has the information it needs to make informed decisions in response to unexpected changes. In fact, the definition itself may change. For example, corruption may not ultimately be the main problem; capacity limits or some “bad” behavior may be the more direct cause of the problem. Or what was originally identified as a result of individual bad actors may turn out to be a broader system of incentives that lead to corrupt behavior.

Complexity requires adaptation, and adaptation requires detail and evidence

Any project needs a strong ToC, especially when targeting “complex” issues that are difficult to fully understand or for which the solution is uncertain, or both. For these “wicked” problems, the strength of a MEL plan and, therefore, the ability to adaptively manage the project, depends heavily on the amount of detail in a ToC and the evidence that supports it.

In terms of detail, a ToC should, at a minimum, include a results chain that lays out expected results, predicts how the results will influence each other, and describes the assumptions along that chain. Interrogating confidence in the assumptions (and how critical that assumption is to the results chain) will help decide whether piloting less confident aspects of a design would be advisable before fully investing in it. For many projects, knowing years in advance what specific higher-level results will be possible can be difficult. Concepts from systems change, such as “boundaries,” “relationships,” and ”contribution,” can provide broad categories of desirable change. Teams can then flexibly maneuver within those categories as stakeholders act and react to the project activities.

Evidence can come from a variety of sources. Formal or informal assessments, reports issued by local NGOs, academic studies, and coverage from reputable journalists can all be valuable. Information can come from other sectors as well, like financial regulation, law enforcement, international trafficking, human rights, etc.

Political economy analyses (PEAs) are especially valuable for understanding the political dynamics of a complex context. That information can be used to understand and mitigate risk, ground truth feasibility of the ToC, and plan for the “soft work” of thinking and working politically (TWP, see below). TNRC has multiple resources on political ecology, an environmentally-focused form of PEA (Devine 2020, Nash 2020) that illustrate how political dynamics can influence conservation and natural resource management decisions. USAID also has a comprehensive resource page to help in procuring a PEA, in addition to its applied framework.

Most anti-corruption program approaches are similar to those used in other development programming – behavior change, transparency, institutional reform, advocacy, and citizen engagement, to name a few. Likewise, common monitoring methods that practitioners may already be familiar with, like surveys, focus groups, and institutional assessments, can be used to monitor progress in anti-corruption efforts. A hallmark of this field is that corruption can be very difficult to quantify, however, so projects typically will need to use a mix of quantitative measures and qualitative documentation. This UNDP resource has an extensive range of examples for monitoring approaches. It shows that common techniques can often be adapted, provided the following considerations are addressed.

Methods should recognize sensitivities, biases, and risks

Data collection methods that rely on interviews and other inputs from stakeholders should be designed with the recognition that talking about corruption directly may not be easy or even possible in some cases. Community members or other stakeholders may not be willing to discuss corruption due to physical or financial risks, and the least powerful are often at the greatest risk. Others may feel that it is improper to talk about people in a disparaging way, especially to strangers. Individuals may agree to provide information, but they may choose to withhold pieces of information or to tell the story they think corrupt elites, or the interviewer or the project, want told. Individuals collecting the data may themselves be put at risk for bringing attention to these topics. In order to mitigate these risks, actors will use a variety of euphemisms in their conversations and reports,[1] which can complicate building the evidence base for ToCs.

Monitoring and Evaluation Considerations

• Develop questions in a way that stakeholders will recognize the intent and feel safe discussing the issue in question without having to use the word “corruption.” Terms such as “impunity” and “lack of accountability” can be effective alternatives to “corruption” in many contexts. Framing issues in terms of “good governance” or “transparency” or “integrity” can make interviewees feel more at ease. Even more seemingly vague terms, such as “equity,” “quality,” and “consistency,” have been used to talk about processes and behaviors in ways that avoid passing judgment on the motivations of individuals and still allow conversations about desirable improvements.
• Integrate randomization techniques that can increase an interviewee’s sense of safety.
• Always use informed consent processes to ensure participants understand how their information will be used and what risks they face in providing it.
• Ensure that project confidentiality and data protection protocols are up to the task. This is particularly important in this field, and if anti-corruption activities are likely to become a significant part of an organization’s work, investing in organization-wide efforts might be worthwhile.

Corruption is often hidden

Much corruption happens away from the public eye, in private conversations or facilitated by private banking transfers. In other cases, corruption occurs inside institutions that are not accessible to the public. This can make it difficult or impossible to know if the problem is corruption or something else, like organizational inefficiency or capacity limitations.

Monitoring and Evaluation Considerations

• Identify proxy indicators for indirectly measuring or documenting change. Page 2 of this U4-CMI resource provides tips on characteristics of good proxies.
• Document assumptions in the project design, including how the anti-corruption reform is meant to reduce the related biodiversity threat, as recommended by this excellent USAID resource. Then document the limitations of the chosen monitoring methods and ensure the team is prepared for the uncertainty the assumptions and limitations introduce. For example, without sufficient evidence, a project may make the assumption that corruption is limiting institutional performance in a forest ministry. Without access to the institution, monitoring change will rely on proxies, including reduction to forest threats. If they see improvement, teams should rule out all other factors that could have led to it before reporting success. MEL approaches like contribution analysis and process tracing can be helpful. Teams not seeing results will have to ask whether corruption is really the issue or whether they have the right anti-corruption approach.
• Continually review assumptions about what is considered corruption by people involved in the project and about whether the project's anti-corruption approaches remain appropriate for the context you're working in. For example, power imbalances can thwart many citizen-based approaches, so checking assumptions about whether power can or should be challenged to achieve objectives, and the risks involved, should be an ongoing discussion for this type of approach (Aston 2021).
• Supplement MEL by strengthening informal networks (e.g., social media connections) with other actors in the system. This increases the chances of reliable insights reaching the team. Teams can build on their existing approaches to thinking and working politically by accessing these less formal information sources.

Corruption is resilient

Corrupt actors will often find new means of hiding their activities or develop entirely new corrupt behaviors to evade the latest reform. This means evidence of results can take time to appear. But it also means new biodiversity or human well-being threats can emerge that will require adaptation.

Monitoring and Evaluation Considerations

• Apply systems thinking to understand where a project fits in to the broader picture and identify ways to monitor changes in that system that the project may cause. Complexity-aware monitoring, as described in this note from USAID’s Learning Lab, has a number of well-established approaches, such as Outcome Harvesting and Most Significant Change. These provide frameworks for working with stakeholders to develop definitions, data collection plans, and analysis strategies. They can be used in community-based projects and institutional reform.
• It is important to note that qualitative data by its nature is larger in volume than quantitative information. Answers to ten Likert questions on an e-survey are simpler and smaller to store, process, and analyze than answers to ten open-ended questions in an interview. There is also the added cost of traveling to collect the data, including, potentially, the salary and travel costs for interpreters. While it’s not surprising that many projects rely on quantitative data, qualitative is worth the investment for the richness of understanding it affords.
• Consider whether corruption is endemic. In these contexts, extra precautions can be needed to prevent corruption from affecting the project. For example, a “double blind” approach, using multiple external assessors who have no contact with the project or each other, can be useful (Cavanagh 2012).
• Evaluate whether tech tools can assist the project. Many projects working across large swaths of land and ocean already use some tools, such as cameras, to monitor illegal harvesting or poaching. They may also be useful in anti-corruption programs to monitor activity far from project sites. Even though anti-corruption reforms often target behaviors “upstream” from individual sites, such as licensing offices or ranger recruitment, cameras could still be instrumental in establishing whether those reforms accomplished their end goal and reduced poaching, for example. Still, a project will need to bear the limitations of any technology in mind. Poachers may identify new routes through parks to evade observation, requiring additional or moving cameras.
• Be realistic about what changes are desirable, what results are possible, and what can be expected in one, three, or five years. Early conversations can help set targets and align expectations (including the funder’s) accordingly.

Corruption is subjective. So is success in changing it

As mentioned above, what is and isn’t corruption can be difficult to agree on. In some places, community members may have adapted to a corrupt practice. In other places, some may perceive more corruption than others because different groups in society, like women, experience corruption differently (Kramer, Hart, and Simoneau 2020). Opinions may also vary on what will work and what constitutes “success.” In these situations, “fixing” corruption can have unintended consequences or leave whole sections of society out of the solution.

Monitoring and Evaluation Considerations

• Integrate opportunities for stakeholders to contribute to designing the project, assessing project results, and interpreting project data. Authentically participatory processes, like “empowerment evaluations,” distribute decision making power to project stakeholders. This can vastly strengthen the quality of data.
• Make ample use of qualitative methods, including qualitative indicators when appropriate. Systematize the collection of qualitative information related to the extent and nature of change. Focus group discussions, one-on-one interviews, case studies, and cutting-edge methods like photovoice (Masterson, Mahajan, and Tengö 2018) will also enrich understanding of a project’s results immensely.
• Disaggregate MEL data, including qualitative data, by gender at a minimum. Assessments should indicate if additional groups, such as those of low economic status or marginalized ethnicities, are disproportionately affected by the biodiversity issues driven by corrupt practices. If data does not reflect their experiences, the project will be less able to address their needs.

And, of course, context, context, context

Like so many other aspects of development and social change, corruption differs greatly from place to place and agency to agency. The successful anti-corruption reforms that exist have varied widely. Trying to simply replicate a successful anti-corruption reform from one context to another has rarely been successful, but lessons about why a reform worked in one setting can be applied when assessing the conditions for a similar approach elsewhere. Proxy indicators that have been effective for one time and place will undoubtedly need to be adapted and tested for different contexts. Broad quantitative indicators, even if directly relevant, may fail to provide depth in understanding the political, cultural, and social dynamics of the context.

Monitoring and Evaluation Considerations

• While there are no lists of standard, project-level anti-corruption indicators that can be easily copied into a project MEL plan, this guide from the UN Development Program provides examples that can inform context-relevant indicator development. Proxy indicators can be highly effective for getting an approximate understanding of project results where direct measurement or documentation is not possible. Part of using them successfully, however, requires that the ToC adequately rule out other influential factors that might explain movement in that indicator. This is another reason to invest in your ToC from the start.
• The most useful M&E plan will be heavily informed by local knowledge. Methods should consider what terms are acceptable and still meaningful for discussing corruption (impunity? good governance?), as well as what mode of knowledge sharing best aligns with the targeted stakeholders. This ethical protocol from the Indigenous Eval Network for conducting evaluations provides insight into some relevant considerations for working with stakeholders from indigenous groups.
• Context monitoring should pay close attention to political changes. Many practitioners may already have valuable relationships with key actors that help them think and work politically (TWP) and understand or anticipate changes that impact the project. The MEL plan will not specify TWP activities, but it can include periodic activities that offer team members a chance to share what they’re learning through their TWP activities and ensure the information is factored into adaptive decision making.

The complexity inherent in anti-corruption projects should not discourage the team! Adaptive management navigates challenges over the course of a project using data to periodically reassess the effectiveness of the design. On the one hand, the fundamental idea of adaptive management is simple: projects try something, see how it goes, make changes based on what they learned, and try again. On the other hand, doing adaptive management well requires systematic planning from the start. At a minimum, teams should plan (and budget) for:

• means to monitor contextual issues, possibly including rapid feedback monitoring.
• regular meetings to reflect on MEL data, discuss sectoral or contextual trends, and consider changes in response to new developments or new evidence about the validity of program assumptions.
• consistent documentation of decisions and learning that come from these meetings.
• an individual responsible for ensuring these activities happen, even during busy months.

This brief from ODI provides a conservation case study on using MEL practices to support adaptive management.

If the project includes a MEL Specialist, that role will be different from more linear, conventional conservation projects. Rather than just measuring change, calculating indicators, and writing M&E reports, a MEL Specialist supporting an anti-corruption project will ideally work closely with project leadership to bring structure to adaptive decisions.

These and other adaptive practices, when tied to a strong theory of change, can improve the team’s adaptive rigor, staying alert to needs and opportunities for strengthening project activities. It will also generate a valuable record for external evaluators who might otherwise struggle to comprehend the full story of the project’s work.

1. All conservationists and NRM practitioners can start to build their capacity for this work immediately by finding out who is working on corruption in their context and connecting with them. This can include local anti-corruption NGOs, human rights organizations, academics, or international NGOs working on related topics like crime and fraud.
2. Project designers should develop a strong ToC with an articulated definition for corruption rooted in the project context. The ToC should be grounded in evidence and any gaps in evidence should be recognized as assumptions that need to be regularly monitored and questioned.
3. MEL plan designers should utilize concepts from systems change and related complexity-aware monitoring methods to supplement MEL activities in a way that accounts for the uncertainties in anti-corruption initiatives. 
4. Project and MEL plan designers should avoid the temptation to save resources by relying on national or international data sources or copying and pasting indicators from one project to another. Local data collected using contextualized methods will improve design and outcomes. Project decision makers should ensure budgets are sufficient to cover not only potentially extensive data collection but also the expertise to analyze and use that data.

[1] These observations about the range of terminology also apply to other research methods. When doing web or journal searches for research or evidence related to a corruption issue, search terms like “US+environment+corruption” may not yield many results. Broader search terms like “discretion,” “governance,” “rule of law,” and “misconduct,” in addition to “governance” and “transparency,” can improve search results.