1. Introduction
The second of BCN's two main goals is to evaluate the effectiveness of
enterprise-oriented approaches to conservation. To this end, the BCN and
its partners need to be able to document the conditions under which this
strategy is effective.
BCN's Core Hypothesis is "If enterprise-oriented approaches to
community-based conservation are going to be effective, then the
enterprises must: 1) have a direct link to biodiversity, 2) generate
benefits, and 3) involve a community of stakeholders.
More specifically, these three elements of the core hypothesis are:
- Linkage between the enterprises and biodiversity:
The enterprises must
directly depend on the in-situ biological resources of the region. The
BCN thus seeks to develop enterprises that would fail if the biological
resource base upon which they depend was significantly degraded.
- Generation of short and long-term benefits: The enterprises must
generate benefits (economic, social, and/or environmental) for a community
of stakeholders both in the short run and, with a high probability, in the
long run, after BCN funding ends.
- Community/Stakeholder involvement: The enterprises must involve
members of the local community, and often others, who are stakeholders in
the enterprises and biodiversity of the area.
In effect, the hypothesis is that if local communities receive sufficient
benefits from an enterprise that depends on biodiversity, then they will
act to counter internal and external threats to that biodiversity. The
analysis outlined in the following section details how BCN proposes to
test this hypothesis.
2. Analytical Design
The basic unit of analysis that we will use is the project
site.1 Examples of the different BCN-funded project sites are provided in the far
left hand column of Figure 2a and a definition of project site is provided
in Section 3.1.
At the most fundamental level, the BCN Hypothesis involves an examination
of the effects of an enterprise based strategy on biodiversity
conservation. Biodiversity conservation thus becomes the yardstick by
which we judge the relative success or failure of a given project. In
scientific terms, it is our dependent variable. We need to measure this
variable in order to be able to draw any conclusions about the BCN
hypothesis (see Section 3.2 below for details as to how we might do
this).
The success or failure of any given project is a function of many
different biological, social, and economic, and political/institutional
factors. Examples of these different factors are listed in the column
headings of Figure 2a. These factors collectively are, in scientific
terms, our independent variables. The aim of our analysis is to determine
as accurately as possible how important each of these factors is in
affecting our final target condition. To do so, we will undertake four
types of analysis which will be described in the following sections:
- Single Factor Analyses
- Case Study (Site) Analyses
- Multi-Factor Analyses
- Offsetting Factor Analyses
2.1 Single Factor Analyses
A single factor analysis involves examining the relationship between any
one factor and conservation success (in scientific terms, a
cross-sectional bivariate analysis). As shown in Figure 2a, this type of
analysis is typically conducted by examining a cross-section of sites with
respect to the factor in question (the area surrounded by the solid
lines). For example, to understand the impacts of benefit distribution,
we would look at benefit distribution across all project sites and
describe the relationship between degree of benefit distribution and
conservation success. Assuming that benefit distribution does affect
conservation success and that both these factors can be measured
quantitatively, this relationship would look something like the graph in
Figure 2b. Single factor analyses can also be conducted qualitatively by
examining in detail the relationship between a given factor and
conservation success in a thematic study (see below).
Figure 2b. A Hypothetical Analysis of the Impact of Benefit Distribution
on Conservation Success
These Single Factor Analyses of these
variables will provide a baseline understanding of the situation impacting
conservation success at the various BCN projects. As we will see below,
however, they are not by themselves sufficient since we also need to
consider interactive effects between variables.
Table 2-1. Planned Single-Factor Analyses
Details about the data required for each factor are presented in Appendix C.
| Factor |
Linked Enterprises
LE1. Degree of Enterprise Linkage with Biodiversity
LE2. Enterprise Ownership
LE3. Enterprise Management
LE4. Enterprise Profitability and Sustainability
LE5. Enterprise Marketing
Generation of Benefits
GB1. Scale of Cash Benefits (Absolute)
GB2. Scale of Cash Benefits (Relative)
GB3. Variability in Cash Benefits
GB4. Distribution of Cash Benefits
GB5. Non-Cash Benefits
GB6. Timing of Cash Benefits
GB7. Frequency of Cash Benefits
Community of Stakeholders
CS1. Existence and Strength of Stakeholder Group
CS2. Representativeness of Stakeholder Group
CS3. Leadership in Stakeholder Group
CS4. Resource Governance
CS5. Community Policing
CS6. Community Participation in Project and Monitoring
Contextual Factors (See Appendix D)
Economic Factors
Stakeholder-Related Factors
Political, Institutional, and Historical Factors |
2.2 Case Study (Site) Analyses
A case study involves looking at one or two sites to gain an understanding
of how a series of factors interrelate with one another. As shown in
Figure 2a, this type of analysis is typically conducted by examining one
or two sites (the area surrounded by the double lines). For example, we
might study the ATI-Nepal sites in some depth to gain an understanding of
how the different project factors interrelate. Another example would
involve comparing sites in the Harvard and PHF projects as well as other
outside examples to get an understanding of how community-based timber
production enterprises might be related to conservation success.
Table 2-2. Planned Case Study (Site) Analyses
| Sites |
Topic |
| All Sites |
Summary for BCN Final Document |
|
| All Enterprise Types |
Summary for BCN Final Document |
| ATI-Nepal |
Viable community managed enterprise |
| TNC-Solomon Islands |
Community managed marine sanctuary |
| PHF and Harvard LTFE |
Community-based timber production |
|
| Palawan |
CADC implementation and decentralization |
| USP-Fiji |
Bioprospecting in the Pacific |
To some extent, in future BCN reports we will be doing these case study
analyses for all the sites in the BCN portfolio of projects by continuing
the ìStories from the Fieldî section from the 1996 and 1997 Annual Reports
in the final BCN document. If we do an overview book on BCNís
experiences, we may also feature a few case study sites in greater
detail. From a more formal analytical perspective, however, it probably
makes sense to select only key studies for in-depth development.
Selection of these sites should be based on 1) addressing key policy needs
and issues, and 2) developing specific studies that address holes in the
other analyses.
2.3 Multiple Factor Analyses
Whereas a single factor analysis involves examining the relationship
between any one factor and conservation success, a multiple factor
analysis involves looking at all factors together, and thus provides some
consideration of interactive effects (in scientific terms, a
cross-sectional multivariate analysis). As shown in Figure 2a, this type
of analysis is typically conducted by examining all of the data
simultaneously (the entire area surrounded by the dotted lines). It
involves looking at the impact of each factor on the dependent variable,
keeping the others constant. Strictly speaking, this type of data is only
done with quantitative data using multiple regression techniques. In
looking at large numbers of factors with relatively few data points, it is
often difficult to get any clear resolution about the effect of any one
factor (in statistical terms, the test has low "power.") Nonetheless, it
is important to undertake this analysis to see if we can get it to work.
2.4 Offsetting Factors Analyses
In the event that the multiple factor analysis does not have sufficient
power to resolve interactive effects, we can still look at how
combinations of various factors offset one another. As shown by the
dashed lines in Figure 2a, this process involves doing a more qualitative
assessment of how two or more cross-sectional variables fit interact with
one another. In effect, this type of analysis involves looking more
subjectively at the models specified for each single factor in conjunction
with one another. For example, we might predict that resource governance
rights have an interactive effect with strong leadership. In effect, we
are saying that a project can succeed if they have strong governance
rights over their land or if they have strong leadership. These analyses
can be conducted either quantitatively or qualitatively.
3. Data Needs and Analytical Tools
The analytical plan outlined in the preceding section requires collecting
four types of data:
- Definition of Study Sites (Unit of Analysis)
- Measurements of Conservation Success (Dependent "Y" Factors)
- Assessments of Interventions (Independent "X" Factors related to the
project)
- Assessment of Context (Independent "X" Factors unrelated to the
project)
3.1 Definition of Study Sites
The first step in the overall analysis is to determine what the specific
"project sites" are that will be used as the basis for analysis. The
specific parameters that define each study site will have to be outlined
on a case-by-case basis by each BCN program officer in conjunction with
project partners. There are, however, a number of general dimensions that
need to be considered in the definition of a project site:
- Spatial Dimension -- What area are we considering as the project site?
Given that BCN's primary goal is conservation, we will spatially define
the core site as the area of biodiversity habitat that the project is
attempting to conserve. It is generally functionally equivalent to the
area that the stakeholders have the ability to manage or influence (either
positively or negatively). Note that this definition means that human
settlements will be excluded from the definition of the project site
unless they are intertwined with the core biodiversity. Within this
spatial definition, we will also define which specific types of habitats
are present at the site.
- Temporal Dimension -- Over what time period are we considering the
effects of the enterprise? While some of the projects have been operating
for many years prior to receiving funds from BCN, others are only now just
getting underway. To control for these differences, most projects will be
defined as starting at the onset of the BCN implementation grant.
However, we will also record as a factor the length of time that the
enterprise was in operation prior to (positive numbers) or after (negative
numbers) the starting period.
- Enterprise Dimension -- What activities are included under the definition
of enterprise? In most BCN-funded projects, there is a gray area between
"enterprise activities" (setting up production systems, marketing
products, monitoring the impact of harvesting) and "project activities"
(organizing stakeholders, capacity building, monitoring social effects).
We will thus have to carefully define for each site what constitutes the
core enterprise and what constitutes the surrounding project.
- Stakeholder Dimension -- Who is considered in terms of analysis of
participation, benefit distribution, and other social factors? In most
sites, these analyses can be defined in terms of local residents who have
an actual or potential impact on the core biodiversity of the
site.
An initial list of the sites being considered is presented in Appendix
A1. We may decide at some point that we will not be able to collect the
full suite of monitoring information for all of the sites, but may instead
choose a subset of these sites for most analytical efforts. On the other
hand, we may also be able to "expand our sample size" for some analyses by
adding information about other non-BCN funded enterprise-oriented
conservation projects.
For each site, we will have to define the specific site parameters as
outlined above. Appendix A2 provides guidelines for defining these sites.
These definitions will be completed by the Program Officers working in
conjunction with project team members.
3.2 Measurements of Conservation Success
At each of these sites, the most critical variable that needs to be
monitored is the degree of biodiversity conservation (the dependent
"Y-axis" variable). This measurement is the yardstick by which we will
assess success of the projects. For the purposes of this analysis, we are
not interested in absolute levels of biodiversity conservation at a site,
but instead the change in biodiversity conservation over the project.
To assess this change in conservation success, BCN will be collecting
several kinds of data. As shown in Table 2-3, these data include
assessments of 1) the change in total habitat area, 2) the change in the
population of the species targeted for exploitation by the project
enterprises, 3) the percentage of identified threats that have been
responded to, and 4) the development of institutions that can monitor and
respond to future threats. Depending on BCN's final analytical needs,
these data can either be analyzed individually or aggregated into a
"hybrid conservation index."2
Appendix B provides the guidelines collecting these data necessary to
assess conservation success. These data will be collected by the Program
Officers working in conjunction with project partners. Data will be
analyzed using indexes and simple descriptive statistics. The information
will also feed into the bivariate and multivariate comparisons described
above and will also be used in case studies and thematic studies.
3.3 Measurements of Interventions
In addition to measuring conservation success, we will also have to
measure all of the factors proposed to explain conservation success (the
independent "X-axis" variables). These independent factors can be
subdivided into two parts -- those that are directly linked to the proposed
project interventions and those that are related to the context in which
the project is operating.
The factors that are directly linked to proposed project interventions
were described earlier in Table 2-1. The specific method that will be
used to measure each factor will depend on the factor itself -- these
methods are described in detail for each factor in Appendix
C. As a
general rule we will, wherever possible, attempt to assess each factor
using quantitative data collected in a systematic fashion. If this
empirical assessment is not feasible, however, we will then employ as a
back-up a more subjective assessment of the factor made by BCN program
officers in consultation with project staff. These assessments will be
made in reference to strict criteria that will be standardized across all
projects with the goal being that any given data set would be scored the
same way by any given observer.
To illustrate this process, we might take the factor of the distribution
of cash benefits. Ideally this factor will be measured by using enterprise
records to determine empirically what percentage of the stakeholder
population in each village received benefits from the enterprise. If,
however, it proves to be the case that most projects are unable to come up
with this data (admittedly unlikely in this case), then we could
substitute a more subjective qualitative ranking of each site in terms of
its level of benefit distribution. Each program officer working with
project staff members could assign each site to a ranking on a scale of 0
- 9 based on criteria of how many people the enterprise employed and many
people received other benefits from the enterprise.
These quantitative data will be analyzed using graphs and simple
descriptive statistics. The information will also feed into the bivariate
and multivariate comparisons described above. These quantitative
assessments of the various factors will also be supplemented by in-depth
Thematic Studies that focus in detail on a particular factor. As in the
case of the Site Analyses, selection of these Thematic Studies should be
based on 1) addressing key policy needs and issues, and 2) developing
specific studies that address holes in the other analyses.
Table 2-3. Proposed Calculation of Components of Conservation
Index
See Measuring Conservation Project Success paper for full details of how
to calculate index.
| Item |
Measure |
Points |
Weight |
Comment |
| % Change in Total Habitat Area |
Area End/Area Start*100 |
100+
95 - 99
80 - 94
50 - 79
1 - 50
0 |
5
4
3
2
1
0 |
5 |
Area at the start will need to be carefully defined as outlined in the
site definition piece. |
| % Change in Population of Species Targeted for Exploitation (project
related) |
Resource Sustainability Index. |
100+
75 - 99
50 - 74
25 - 49
1 -
24
0 |
5
4
3
2
1
0 |
5 |
See Section 3.1 of Measuring Conservation
Success paper. |
| % of Identified Threats Responded to |
Threat Reduction Assessment
(TRA) Index. |
81 - 100
61 - 80
41 - 60
21 - 40
1 -
20
0 |
5
4
3
2
1
0 |
5 |
See Section 3.2 of Measuring Conservation
Success paper and TRA paper. |
| Future Conservation Success - Monitoring and
Response |
Assessment on a scale of 1 - 5 as to capacity of
stakeholders to detect
and respond to new threats. |
perfect
likely
potential
some
little
none |
5
4
3
2
1
0 |
5 |
Criteria for this ranking:
- Group present
- Group has monitoring plan
- Group is collecting and analyzing data
- Group has proven it can respond to new threats
- Group can anticipate upcoming threats and respond proactively |
3.4 Assessment of the Context
In addition to the factors that are directly linked to project
interventions, the success of the project can be affected by various
contextual factors that also need to be assessed. A preliminary list of
these factors is provided in Appendix D. Here again, the specific method
that will be used to measure each factor will depend on the factor
itself. Owing to the broader nature of these factors and BCNís limited
resources, however, these factors will generally be assessed more
qualitatively than quantitatively.
4. Potential Audiences and Outputs
Potential audiences for the results of this analysis include all of BCN's
clients (Table 2-4). The results of this study will document the degree
to which BCN and its partners have been able to achieve our hypothesis
testing goals.
These results will be packaged in different forms for different
audiences. Table 2-4 lists a few proposed outputs for each audience.
Table 2-4. Outputs for Topic II for Each of BCN's
Clients/Audiences
| BCN's SPECIFIC CLIENTS |
BROADER AUDIENCE |
SAMPLE OUTPUT |
| 1. Communities and local groups implementing
projects |
Field-based practitioners |
- 4 page summary (in relevant languages)
- Edited volume summarizing BCN program |
| 2. National and international groups implementing
projects |
Office-based managers |
- Final report
- Web site information |
| 3. USAID and US-AEP Missions and offices; Congress |
Donors |
- Final report
- Oral presentations |
| 4. Politicians and government workers in countries |
Policy makers |
- Oral presentations (in appropriate language)
- Briefing visits |
| 5. Researchers working with BCN-funded sites |
Researchers |
- Articles in technical journals (social, bio, economic,
development) |
| 6. US taxpayers |
General public |
- Web site information
- Press releases |
5. Implementation Steps
In terms of the
overall sequence, once the basic data sets have been assembled, the single
factor analyses can be run looking at the impact of each factor in
relation to the conservation success variable. Running these analyses
will involve doing initial pre-tests, cleaning or transforming the data as
necessary, and then running final tests. These data can then be combined
with other qualitative information to complete the Thematic Studies.
Once all the single factor analyses have been completed, then the
multi-factor analysis can be run. And once the results of this analysis
are known, we can make a determination as to how to proceed with the
offsetting factor analyses. If the multi-factor analysis is successful,
then we may not choose to do the offsetting factor analysis. If, however,
problems arise, then we will have to design the offsetting factor work.
1 There are several other units of analysis that we considered.
Using the overall project was rejected because a large percentage of the
projects in the BCN portfolio have multiple sites that vary in their
characteristics. Using individual enterprises as the unit of analysis for
the overall study was rejected because our ultimate goal is conservation
and to achieve conservation, it may make sense to have more than one
enterprise at a given site.
2 Full details regarding various options that were considered for
measuring the dependent variable can be found in Nick Salafsky and Richard
Margoluis (1998) Measuring Conservation Project Success: Developing a
Standardized Index for Comparing and Analyzing Biodiversity Conservation
Network (BCN) Funded Projects. Draft Working Paper, Biodiversity
Support Program, Washington, DC.
