Measuring Success of Adaptive Management Projects
Condition for AM is met, success likely
Success will be fraught with challenges but is achievable
AM project not likely to be successful, potentially abandon AM as an approach to management
Was/Is there a mandate requiring a natural resource management decision?
Yes: mandate may be motivated by law, policy, agency directive, social pressure, competition for resources, or critical resource scarcity
Yes, but it is weak: mandate may be informal or critical mass to make a decision has not been achieved thus stalling the process in later stages
No
Is there uncertainty about the system in question, i.e. are consequences (both social and ecological) of management alternatives unknown?
Yes
No
What is the scale and complexity of the AM project?
Small-scale or simple: for example, a single-species context in a protected area managed by one agency
Large-scale or complex: for example, a water allocation debate in a large, multi-jurisdictional basin with unsettled water rights
Is there strong baseline data for the system in question for an AM project?
Yes
No, but it is easily attained through research. Peform the necessary research and proceed. If possible, include relevant stakeholders in baseline information gathering to promote learning processes in this early stage
No
Can the system in question be manipulated easily, i.e. is there high controllability?
Yes
No
Are all the relevant stakeholders engaged?
Yes
Some, but not all. If the AM process can engage stakeholders directly affected by AM decisions and those with ability to stall or block the process, AM can proceed
Not enough. If stakeholders with a direct stake in resource decisions to be made through the AM process cannot engage, the process is likely to fail
Are objectives of AM clear, measureable, and economically, politically, and ecologically feasible?
Yes
No. Reformulate goals and objectives until they are explicit, measureable, and feasible
No and there is no ability to modify them
Do stakeholders agree on AM objectives?
Yes
No. Stakeholders can seek conflict resolution avenues to reach agreement and proceed
No and conflict resolution is not an option or has been exhausted
Can the ecological system including current resource management regime and potential alternatives be modeled?
Yes
Maybe but capacity to do so is lacking. Attain necessary capacity (knowledge, skills, funding, etc.) and proceed
No
Can monitoring of resource management be designed, funded, and maintained?
Yes
Maybe but capacity to do so is lacking. Attain necessary capacity (knowledge, skills, funding, etc.) and proceed
No
Can management actions be modified (including implementation of necessary policy changes) from monitoring data?
Yes
Potentially, but certain modifications to law or policy are required. If these modifications can be made or adjusted for, proceed with AM
No
Assessing the Suitability of Adaptive Management in Different Contexts
Given a general definition of adaptive management success—measurable progress toward explicit objectives through strict attention to rigorous process—it is helpful to review factors that may or may not lead to success in adaptive management projects. Knowing when (and when not) to apply adaptive management has important implications prior to any attempt to measure success in adaptive management applications—including the conservation of precious (and often scarce) evaluation resources. If a situation does not merit the approach, there is a high likelihood that the approach will not be successful. Although this feels like common sense, adaptive management literature clearly points out that the most common reason for a lack of measureable success in adaptive management projects is the inappropriate application of the technique (Gregory et al. 2006, Allen and Gunderson 2011, Mc-Fadden et al. 2011). So what are the characteristics of a management scenario in which adaptive management would be an appropriate approach? Applying adaptive management as an approach to NRM management is appropriate when uncertainty and controllability are high (Allen and Gunderson 2011). If implemented correctly, adaptive management creates significant potential for learning through implementing experiments as management actions, thus enabling decision making in the face of uncertainty and allowing managers to adjust these decisions according to what is learned from rigorous monitoring of the experimentation . It is critical to have high levels of ecological and social controllability in a situation ripe for an adaptive management approach. Without a significant ability to manipulate the environment, adjusting management from monitoring of adaptive management experiments would not be possible and thus the adaptive management approach could not function. Knowing when to appropriately apply an adaptive management approach is the initial key to ensuring success in adaptive management.
Although not generally where the most complexity and uncertainty in management is found (e.g. Lee 1993), small and simple management contexts (for example, a one-species management focus in a small protected-area) have been shown to support more successful applications of AM (McConnaha and Paquet 1996, Gregory et al. 2006, Morghan et al. 2006). The more complete the data surrounding the social and ecological components of the system, the more likely adaptive management experimentation will produce meaningful results.
In addition, the approach to adaptive management, passive vs. active, may have some influence on the success rate of adaptive management projects. Morghan et al. (2006) suggest that the first attempt at adaptive management should be a “pilot study” for the real adaptive management implementation; a more passive approach with heavy modeling and introductory collaboration which can be used to generate the necessary conditions for active adaptive management. McCarthy and Possingham (2007) describe such an approach by using Bayesian statistics to passively model active adaptive management alternatives prior to experimental implementation in a small watershed context. In this way, passive adaptive management can be thought of as what Williams and Brown (2012) refer to as the “set-up phase”—part of the adaptive management cycle, but a precursor to the active experimentation of adaptive management. Adaptive management projects that engage in passive adaptive management prior to active adaptive management may be more likely to succeed because they have had an opportunity to ‘work out the kinks’ (Zellmer and Gunderson 2009).
Since adaptive management by definition necessitates stakeholder involvement, the type, level, and coordination of this involvement is a significant factor that either contributes to or detracts from adaptive management success. Building a community of shared understanding rather than opting for strict consensus in collaborative decision-making is more likely to foster adaptive management success (Zellmer and Gunderson 2009). Further, developing a shared understanding of the adaptive management process as well as the ecological context and management objectives is critical (Broderick 2008). Occasionally, institutional barriers to adaptive management will be presented by the lead agency or coordinating organization. One way to avoid this is to rely heavily on a stakeholder group, an appointed neutral third-party, or a series of elected/appointed advisory committees to carry out much of the adaptive management decision-making and implementation (Zellmer and Gunderson 2009, Smith 2011). Zellmer and Gunderson (2009) specifically suggest that a technically-based agency, such as U.S. Army Corps of Engineers (ACOE) is not the best choice for a coordinating or lead agency on an adaptive management project due to the narrow nature of its mission and expertise. If a management scenario is ripe for adaptive management according to an assessment based on Table 6.1, adaptive management should proceed with strict attention to process laid out in the adaptive management cycle (see Fig. 6.1 and Table 6.2). To support adaptive management success, we suggest a thorough review of each phase of the adaptive management cycle prior to moving on to subsequent phases. That review can be conducted as a self-assessment, but a robust peer-review will be more effective in determining potential areas of concern.
Fig. 6.1
The adaptive management cycle
Table 6.2
Criteria organized by phase of the adaptive management cycle
Phase of AM Cycle | ||||||
|---|---|---|---|---|---|---|
Assess | Design | Implement | Monitor | Evaluate | Adjust | |
Questions to ask of phase (qualitative inquiry) | Are all relevant stakeholders involved or engaged? | Are objectives explict, prioritized, shared, and measurable? | Has the program moved to action? | Who is in charge of monitoring? Is there benefit to or danger in joint-monitoring responsibility? | Was any new information learned? | Were adjustments made to experiments (management policies) in light of new information learned? |
Has clear and comprehensive baseline information been established? | Can management alternatives be posed as testable hypotheses? | Are experiments performed with the rigor of the scientific method? | Is there funding secured for the necessary and realistic monitoring timeframe? | Do the results match what was expected? | Do objectives still make sense in light of new knowledge or should adjustments be made? | |
Potential Metrics for measuring each phase (quantitative inquiry) | Stakeholder mapping: map the management scenario to determine individuals and groups who may be affected by a management decision; inform map through interviews and/or spatial analysis | 1. Public and stakeholder surveys on objectives; 2. model management alternatives to determine potential outcomes; 3. design measurable indicators for measuring progress toward goals | Measure number and duration of completed experiments. What is the length and duration of committed funding for experimental policies? | Measure consistency of the monitoring program including overseeing agency, funding, and any interruptions during the duration of monitoring | Quantify new information learned. Integrate into models. Quantiatively compare observed vs. predicted data | Measure the number and breadth of adjustments made to management policies |
External Peer-review | Review of baseline ecological data for management scenario | Review of models used | Review the procedures for implementing policies as experiments- was there a control, were they replicable? | Review of monitoring procedures- were the best possible practices used for an adequate duration of time? | Review collected data to ensure accurate interpretations were made | Review adjustments to management policies as accurate interpretations of monitoring as well as reasonable iterations of the AM cycle |
Evaluating Phases in the Adaptive Management Cycle
The process of adaptive management is embodied in the adaptive management cycle (Fig. 6.1). Many variations of the adaptive management cycle have been proposed in the literature (see Murray and Marmorek 2003, Stankey et al. 2003, Duxbury and Dickinson 2007, Pahl-Wostl 2007, Williams et al. 2009, Fulton 2010). Although there are significant differences among some of the proposed models, there are consistencies as well, in that they all generally include six core phases: assess, design, implement, monitor, evaluate, and adjust. The value of this simplified six-phase conceptualization of the adaptive management process is its succinct portrayal of the feedback loop of learning that is created through iterations of the adaptive management cycle. Significant literature exists on the performance of each phase in the cycle and for the ease of use here, we have organized this information into a single table (see Table 6.2). We suggest that progress in adaptive management can be measured in terms of completion of the adaptive management cycle and the presence or absence of feedback (i.e. learning) in the next cycle evidenced in new information being applied to objectives, experimental design , and ultimately NRM policies (Schreiber et al. 2004). Any complications or failures in individual phases of the adaptive management cycle may adversely affect the quality of learning applied to subsequent iterations. For example, if left out of the ‘assess’ phase, important (or politically powerful) stakeholders will be removed from joint fact-finding efforts to collectively construct a baseline picture of existing ecological and social conditions. As a result, these stakeholders may stalemate future agreement on measurable project objectives during the ‘design’ phase or prohibit experiments through use of litigation during the ‘implement’ phase.
However, not all adaptive management projects can and will move at a desired pace, one that keeps in step with funding cycles and those agencies and organizations needing an evaluation of project success. So how can success be determined by an unfinished project, i.e. how can success in adaptive management be determined without a completed iteration of the adaptive management cycle? We suggest that a definition of success in adaptive management depends on effectively completing individual phases of the adaptive management cycle. Therefore, measuring