TNRC Guide | Researching Social Norms and Behaviors Related to Corruption Affecting Conservation Outcomes - Part 2

This TNRC Guide shares practical knowledge for program designers and implementers to reduce corruption’s impact on conservation. It is Part II of a two-part series.

Corruption behaviors are complex, so research to identify ways to address them can be challenging (Schwickerath, Varraich, and Smith 2017). A hypothetical example illustrates the point: Anti-corruption practitioners interested in reducing bribery at border checkpoints known for high volumes of illegal timber trade need to understand where best to invest resources to achieve meaningful impact. Options could include interventions that reduce social expectations and community tolerance of giving bribes, initiatives that promote changed behavior by appealing to professionalism and codes of conduct among potential bribe takers, or interventions that encourage or embolden potential bribe givers not to give money when requested. Each of these responses to a corruption problem focuses on different actors and seeks to influence different social norms (SN) that might motivate a behavior change (BC).

Not every corruption problem may be right for such SNBC approaches, of course, and alternatives or accompaniments could include more transparency, increased scrutiny and oversight, or the introduction of technology (Mgaza 2022). Identifying whether to use SNBC or these more “structural” amendments, and if SNBC is chosen, then where, how, and with whom to engage, will depend on multiple factors. These might include prevailing practices of bribing enforcement officials, along with contextual factors that might influence the demand for bribes (such as low salaries, few rewards or other incentives for better professional standards, or a lack of recognition or pride for protecting community resources and stopping illegal wildlife trade), and the perception of personal risk among community members interested in combatting corruption.

This Resource Guide introduces some foundational principles and common considerations for research into conservation-focused anti-corruption actions, complementing a companion Guide on baseline and formative research. The Guide is not a manual; authoritative materials like the “Manual on Corruption Surveys” (UNODC 2018) are already available to support anti-corruption research.

Instead, this Guide introduces two “packages” of research that introduce non-specialists to some of the relevant core approaches and methods for assessing and adaptively managing SNBC approaches and understanding whether they have achieved the overall aims, goals, and ambitions.

As corrupt practices are by their nature sensitive and usually illegal, some overarching principles should be considered when conducting related SNBC research. The principles currently available in the official CITES Guidance on Strategies to Reduce Demand for Illegal Wildlife Products might be considered and applied to research on corruption facilitating IWT as well. The aim of such principles would be to mitigate any risks and deliver reliable, robust, quality insights that can inform the decisions of relevant authorities.

Building on Economic and Social Research Council guidance, principles include the following:

1. Research should aim to maximize conservation benefit and minimize personal risks. Those conducting research into sensitive or illegal behaviors should have comprehensive safeguards and risk mitigation strategies in place. Inexperienced researchers could expose themselves and their subjects to risk (Nature 2022).
2. The rights and dignity of individuals and groups should be respected (Nature 2022).
3. Participation should be voluntary and informed. Respondents should be aware how the information they are providing will be used and should participate freely and without coercion.
4. Research should be conducted with integrity and transparency. For example, questions should be framed in a fully neutral way, and should not lead the respondent to answer in a certain way (like unintentionally leading respondents to agree with the researcher’s expectations).
5. Independence of research should be maintained. A clear separation must be maintained between people conducting the research and those who are the subjects of it. Otherwise, conflicts of interest could compromise results. Where conflicts of interest cannot be avoided, they should be disclosed and managed.

For those interested to understand more about these topics or seeking more detailed guidance on how to approach research relevant to corruption, in addition to the UNODC Manual, useful materials include:

• “A Note on Research Methodology for Combatting Corruption”
• “Guide to Using Corruption Measurement and Analysis Tools”
• “Measuring Corruption: Myths and Realities”

The rest of this document introduces the two research “packages” that can be employed to monitor project progress against outputs, outcomes, and the theory of change, and to evaluate impact. The research in these two packages would chronologically follow the three packages introduced in the companion Research Guide on baselines and formative assessments, so these packages are numbered consecutively.

“Package 4,” overleaf, focuses on tracking project progress as implementation proceeds (“monitoring”). “Package 5” focuses on assessing the achievements, success factors, and lessons learned as a result (“evaluation”).

To ensure an adequate veracity of insight and appropriate attribution of any impact, both research packages should be undertaken as far as possible. If there are constraints in terms of time and resources, not all methods shown are compulsory. This is explored further in subsequent sections, meanwhile a summary is per Table 1:

Table 1. Summary of Research Packages (numbering continuing from the first Guide)

As performance measurement is relatively straightforward and likely to already be included in project monitoring, this section focuses on primary social research methods to test the fidelity of the intervention and track progress against outcomes. These methods include a mix of direct and indirect observation- and opinion-based measures. Combining these approaches (i.e., using “mixed methods”), produces insights about the extent to which delivery is achieving the desired impact or where adaptive management is required.

For both performance measurement and outcome tracking, monitoring should occur throughout the project. Some outcome tracking methods are, however, more time-consuming and costly to deploy and may only be feasible during key project milestones like the mid-point or end of a financial year. Milestones would usually be set when implementation commences, and they should be captured in a Monitoring, Evaluation and Learning (MEL) plan. All team members should have access to this plan and use it as a core reference point and project management tool, to ensure efficient, focused, and effective project delivery.

Figure 1. Conservation Measures Partnership Adaptive Management Cycle

Triangulating data emerging from social research through a “mixed methods” approach is established good practice (Anguera et al. 2020). It is especially relevant for research on sensitive or complex topics such as corruption, where it can help improve the quality and integrity of insight.

Choosing which method(s) to deploy and when, however, can be tricky for non-specialists. Decisions depend on factors such as the time, funds, and skills available, and the “confidence levels” required. Research Guide 1 introduced the “pros” and “cons” of different social research methods; Tables 2 and 3 in this document (below) draw some further distinctions between common quantitative and qualitative methods. These distinctions should help inform decision making around which approach to deploy when to meet monitoring needs.

Table 2. Matrix of Methods

Key: Quantitative, Qualitative

Table 3. Merits of Methods

Key

Challenging

Manageable

Desirable

While each project circumstance and team approach will be different, a rule of thumb is to ensure monitoring processes use at least two direct opinion-based methods and one direct observation-based method, for performance measurement and outcome tracking, across the life of the project. Data acquired through monitoring processes should also augment that gathered at the project start (i.e., baseline data and formative insight) and end (e.g., impact evaluation). The use of additional indirect opinion- or observation-based methods will add value to monitoring processes but should first be appraised for the desirability, feasibility, and viability of data acquisition (Choudhary 2019). There is a wealth of further material publicly available on this topic and suitable for practitioners, so those interested to understand more are directed towards the following resources:

• “Monitoring and Review,” Chapter 17 of “Global Anti-Bribery Guidance”
• “Anti-Corruption Measurement and Monitoring”
• “Monitoring, evaluation, and learning for anti-corruption projects: What conservation practitioners need to know”

As stated in Table 1, the purpose of Package 5 is to evaluate all project performance and impact measures, usually defined at the planning stage, based on the project goal, aims, objectives, and outcomes. Similar to Package 4, because performance assessment is assumed to be straightforward (e.g., totaling the people reached and/or outputs produced), this section focuses on impact evaluation.

Approaches to impact evaluation vary greatly according to the project intervention types and SNBC activities, but just as for monitoring, they would generally lean heavily on data acquired through primary direct social research. Some indirect methods can be used if change can be attributed to the project.

The need to attribute change directly to the project, and to what degree of detail, typically governs the choice of research approaches for impact evaluation. For simplicity, two “options” are to compare changes:

1. Over time – comparing key indicator values for differences between baseline (Package 2) and “endline” and attempting to attribute any change to the intervention.
2. Between a treatment and a control group – comparing key indicator values amongst those exposed to the project (the “treatment” group) and those not (the “control” group) to identify differences in targeted outcome. See Figure 2 for a visual illustration.

Figure 2. Selecting treatment and control groups to allow for impact evaluation Figure 2: Selecting treatment and control groups to allow for impact evaluation

Source: Adapted from Haynes et al. (2012)

With either option, everything except the specific treatment should be kept as consistent and comparable as possible. This includes variables like the research methods, question type, timing, framing, and demographics of participants. By keeping all of the other variables the same, any changes over time (in option 1) or between groups (in option 2) in the variable(s) the project sought to influence can be better attributed to the project.

However, the feasibility of achieving this consistency and comparability can be a challenge, and it can vary considerably between the options. This is best illustrated by changes catalyzed by the COVID-19 pandemic: between 2019 and 2022, many studies around the world saw shifts in variables like income level and residential status. This reveals some of the risks associated with impact evaluation, especially evaluation focused on measuring change over time (option 1). The elements that should remain consistent may sometimes be beyond the control of the research team. With option 2 it is easier to accommodate such potential inconsistencies, but the approach must be “baked into” the project design from the start for the integrity of the “control” group to remain intact. More information on these topics is available in Olofsgård (2014).

Another consideration for Package 5 is the need to ensure any claimed changes in attitudes or actions self-reported by the target audience are objectively verified where possible. This is especially challenging for corruption behaviors which by nature tend to be hidden, but research techniques that could be used include observation or investigation. Options for observation techniques are summarized in Table 2. Investigation techniques meanwhile, might be split into those involving desk-based analysis, including techniques to measure “proxies” when the corruption is hidden, and those involving fieldwork. However, investigative fieldwork should only be conducted by appropriately qualified professionals, as this is a high-risk area. Desk-based analyses could use social listening, critical discourse, and big data analysis of social and other types of media; or reviews of publicly accessible information like police and court-case records, hotline reporting, or criminal convictions. Especially when the intervention is limited in size or time, however, identifying causation or contribution to impact from implemented activities is not always possible.

A final point is the need to understand not only what has changed and whether it was a result of the project, but also what specifically caused the changes. Capturing success factors and lessons learned is critical to designing better interventions in the future or scaling projects. A mixed methods approach can be instrumental in qualitatively understanding the drivers of quantitative results.

Impact evaluation is a complex topic, and there has only been space here to cover some basic concepts. Practitioners should gather more information before confirming the type of approaches to be adopted, and useful resources for this include:

• “Effectively Evaluating Anti-Corruption Interventions”
• “How to Monitor and Evaluate Anti-Corruption Agencies”
• “Evaluating Anti-Corruption Initiatives”

Annex 1 provides an example “dipstick” survey process and set of questions relevant to Package 4. Typically, the process would be broken down into three sequential steps such as:

1. Identify the main survey objective and define the key questions
2. Clarify the survey population, sampling method, and sample size
3. Conduct an ethics review, recruit the sample, and deliver the survey

A hypothetical anti-corruption example is provided below:

Step 1: Identify the main objective and define the study questions

Objective: To understand if public knowledge and attitudes are changing around corruption.

Key questions: 

• What is your attitude towards bribery?
• How do you feel about reporting bribery if you witness it?
• What do you think stops people in society from reporting bribery?
• What do you think should be done to stop bribery?
• Have you seen any good examples of bribery being stopped elsewhere?

Step 2: Clarify the survey population, sampling method, and sample size

Survey population: Those living in city X, and those living in and around national park Y.

Sampling method: Representative of relevant socio-economic demographics for these areas.
Consider including a booster sample of those who report being asked to pay
a bribe in the past 12 months (P12M). Snowball sampling may be required.

Sample size: 100 people between 18 and 65 years old, from each area.

Step 3: Conduct an ethics review, recruit the sample, and deliver the survey

The survey will be conducted by experienced and fully qualified enumerators, and as the research involves human subjects, the survey topic guide and overall approach will be reviewed and approved by an Ethical Review Committee (ERC) / Institutional Review Board (IRB).

The enumerators will ensure consent is appropriately informed and that no pressure is applied, nor promises made, to those completing the survey. Respondents will be offered a confidential space in which to complete their responses should they prefer this, using handheld devices such as a tablet or mobile phone. Subsequent storage of such equipment, data recording devices, and the data itself will be tightly managed and involve the use of a fully secure facility. Analysis of survey results will involve cleaning and anonymizing the data before it is accessed/used by others.

The survey will be pre-tested to ensure it takes between 10-15 minutes to complete, and that there is clarity and understanding around the questions. Once this has been completed, any tweaks can be made before enumerators canvass the target geographies to recruit survey participants.

A mid-point check-in and coordination call will be arranged with project managers to check all is on track and to sense-check initial insights emerging.

For more information on how to design and run a ‘dipstick survey’, an excellent example is available in relation to the use of insecticide treated nets by children. Although this is not a corruption related example, the detailed approach is illuminating and could be adapted easily.

Annex 2 introduces a six-step process recommended for NGOs by commercial social listening firm Awario. Awario defines social listening as: “The process of gathering and analyzing online posts both on social media and news websites, blogs, and forums.” A tool like Awario “allows you to collect all the posts that include keywords and keyword combinations you choose and analyze them based on demographic and psychographic categories, such as authors’ gender, language, location, sentiment expressed in a post and so on.”

Awario’s six-step process, adapted to anti-corruption, is:

Step 1: Define the goal

The goal for anti-corruption practitioners would likely relate to project impact evaluation, like monitoring a campaign to build customs officers’ pride at properly confiscating animal products. Other goals could include brand visibility, message amplification, reputation management, and social mobilization.

Step 2: Identify keywords of interest

Keywords are those relevant words and phrases that would appear in the media being targeted. Such words will vary according to the project type and focus. There is no limit to how many words or phrases should be identified, although a shorter, more specific list that avoids generic descriptors will likely lead to more useful and easier to manage analysis. Awario also recommends including misspellings, like “bribes, bribe, brib, birbe…”

Step 3: Choose alert settings

Alerts can be set for various aspects of a keyword search. “Negative keywords” can be excluded; for example, if you are specifically interested in corruption that enables fisheries crime, avoiding terms such as forestry and logging). The language, location, date range, and source can also be calibrated.

Step 4: Review raw data results

Awario’s software and platform will start providing results in real-time, thus generating instant insight into whether the keywords, filters, and alerts chosen are delivering the information required.

Step 5: Review analytical reports

Frames for analysis could include straightforward statistics (e.g., the number of posts using certain phrases or words, and the location of the posters); direct comparison of datasets (e.g., how many news stories about a certain company that mention “bribes” versus mentions of “influence peddling”); or a “deep-dive” into specifics (e.g., certain politically exposed persons or areas where corruption is understood to be rife).

Step 6: Apply insights

Based on the evidence generated and trends identified from the data, identify any quantitative assessments of impact (e.g., fewer mentions of corruption compared to the baseline, more mentions of confiscating rhinoceros horn). These assessments can inform broader learning about success factors to replicate or adaptations to the existing project.

Those interested to understand more on this topic can read “Defining Social Listening: Recognising an Emerging Dimension of Listening.”

Research and writing of this document was led by Gayle Burgess, Behavior Change Programme Leader of TRAFFIC: email: [email protected]. The author is grateful to Elizabeth Hart, Gabriel Sipos, Preston Whitt, Sabri Zain, Claudia Baez-Camargo, and Maija Sirola for their reviews of the document.