Appendix J: Strategy Selection Tools - Conservation by Design 2.0

Consequence tables

In many cases, good solutions can be found without fancy software or analysis, simply by looking carefully at the estimated consequences of the options being considered. A consequence table has the options being considered listed down one side and the objectives listed across the top, and the estimated consequence of each option on each objective populating the cells. Establishing a consequence table is also the perfect platform from which to use the next two types of methods we cover; multicriteria decision analysis and return-on-investment.

Remove options that are inferior for every dimension. An option is inferior if there is another option that performs at least as well against all objectives and better than it for at least one objective. If such a situation exists, there is no rational reason for choosing the inferior option so it can be disregarded as an option.
Remove options that are largely inferior for every dimension. Some options may be roughly equivalent for most objectives but significantly inferior in others. These also can be discarded.
If someone disagrees with removing an inferior option, consider whether this is because it met an additional objective that was not considered in the consequence table. If so, this might require going back and adding another objective.

Multicriteria Decision Analysis (MCDA)

Consequence tables. Multicriteria Decision Analysis (MCDA) is a broad term that encompasses a range of methods for incorporating multiple objectives into the evaluation of alternative options. Equivalent terms include multicriteria decision making (MCDM) and multicriteria analysis (MCA). Criteria should include all those considerations by which the performance of a strategy should be evaluated (but lists of criteria are no replacement for clear identification of objectives). The basic elements of a MCDA, are:

A set of criteria against which the desirability of strategies are to be judged.
Assign weights to criteria. Weights reflect the importance of criteria in determining the outcome. Weights might be assigned individually or as consensus amongst a group.
Combine an assessment of the performance of each strategy for each criteria with the weight for that criteria.
Aggregate scores for each strategy across all criteria to give an overall assessment of performance or utility.

Although there are many ways that criteria weights and performance scores can be combined to calculate utility, by far the most common is known as a linear additive benefit function or model:

where U_j is the overall benefit or utility of strategy j, W_i is the weight given to criterion i and V_ij is the value of option j for criterion i. In simple terms this means that for each strategy, the performance score for criteria 1 is multiplied by the weight given to criteria 1; the same calculation is made for each additional criteria, and then these are all added together. The result is an overall score for each strategy.

Normalizing data. Criteria can be measured on very different scales. In order to calculate utility using a linear additive function, it is necessary that the performance values of strategies for different criteria are measured on the same scale; otherwise those criteria measured on scales with larger numbers will be unintentionally weighted more heavily. The process of converting data to the same scale is known as normalizing.

There are different procedures for normalizing data but one that is considered robust for use in MCDA⁶ is given by the formula:

where V_ij is the performance of strategy j for criterion i, and min[V_i] and max[V_i] are the minimum and maximum possible values for criterion i. The prime mark after V on the left hand side of the equation is standard notation that indicates the parameter is a transformation of another parameter.

Weighting criteria. Criteria are very unlikely to all be of equal importance in determining which strategy is best. Weighting is intended to help ensure that the calculation of utility actually reflects the perceived importance of the different criteria. The weighting given to criteria can significantly change the outcome of an MCDA, and therefore it is really the systematic and transparent weighting of criteria that is the substance of an MCDA. An important and under-appreciated point about weights is that they are only meaningful with reference to the observed range of outcomes for each criterion. In other words, the importance of a criterion in influencing a decision depends both on its inherent importance to the objectives, and on how well or poorly the strategies under consideration perform for that criteria. For example, a stakeholder group might value recreation opportunities over biodiversity conservation, but if the options strategies being considered differ little and are all satisfactory in their consequences on recreation opportunities, then it does not make sense to assign the recreation criterion a relatively greater weight. Giving greater weights to criteria that do not vary much has the effect of making strategies seem more similar in utility value, which is not informative for decision making. This means that it is premature to weight criteria before knowing the expected performance of strategies against these criteria (hence the importance of some form of consequence table).
Use a defensible approach to develop criteria weights.Weights of criteria in an MCDA embed the value judgements of those doing the weighting. As such, weights should not be seen as objective assessments of the relationships between criteria, and might benefit from including several perspectives prior to final selection of weights. It also means that it is important to assign weights through a process that makes these value judgments clear. Good approaches to establishing weights include Swing Weighting and the Analytic Hierarchy Process.

Return on Investment (ROI)

In the field of conservation, return-on-investment (ROI) has come to be a rather general term for prioritization approaches that explicitly consider the cost of the strategies being considered. Conservation ROI analysis belongs to a general class of economic analysis known as cost-effectiveness analysis. Economists consider something to be a cost-effectiveness analysis rather than a cost-benefit analysis when the outcome or return side of the equation is not monetized (for example, expressed as a dollar value). The basic notion behind ROI or cost-effectiveness is that the expected return or outcome from a conservation action should be balanced against the cost of achieving that outcome. Although they are not always recognised as such, the Conservancy’s Ecoregional Assessments were often a sort of ROI analysis, particularly when they are implemented using optimization software such as Marxan.

The cost-effectiveness, CE, of taking action i in place j can be given by the general equation:

where B_ij is the benefit of taking action i in place j, Cij is the cost of taking that action, and Prij is the probability that if taken, the action will deliver the expected benefit. This last term is not strictly necessary but is good practice, easy to do, and increasingly expected. Again, it is worth emphasizing that neither the benefit nor the cost portions of this equation need to be explicitly stated in financial terms.

Knowing bad ROI is more important than knowing good ROI. In our experience, there is generally enough uncertainty around the costs of different strategies that minor differences in the relative ROI of alternatives should be interpreted with caution, especially as cost-effectiveness is not the only consideration in selecting an action. In some ways the most relevant information contained in an ROI analysis is at the bottom-end, that is, those options which appear to deliver a poor ROI. The the selection of a strategy relatively poor cost-effectiveness would require carefully justification.
Link ROI to strategy and opportunity mapping. ROI analysis can also be used to select strategies in a way that considers both action and location simultaneously, essentially answering the question, “what should we do where?” The strategy and opportunity mapping described earlier in this Guidance is a strong platform for ROI analysis.

Trade-off analysis

Trade-off analysis is a general approach to evaluating strategies when there are multiple objectives. Trade-offs are naturally presented as consequential relationships between things we care about, and they exist when achievement of one objective comes at the expense of the achievement of another objective. For instance, when planning the allocation of a landscape to different activities there is likely to be a trade-off between food production (the amount of food able to be produced) and biodiversity conservation (the number of species conserved). The more land we use to grow food, the fewer the species likely to survive, and similarly, the more land we dedicate to conservation, the less food we can produce from that landscape (assuming the same level of productivity). By identifying different combinations and extent of land-use placement, and plotting the expected food production and biodiversity conserved for each, we are able to see the consequence of improving one of these objectives in terms of loss for the other .In these cases the best solution only be decided through exploring actual trade-offs in predicted outcomes. For instance, in the food versus biodiversity example above, the weight local stakeholders give to biodiversity objectives relative to food production objectives will depend on the consequence of biodiversity conservation for food production: if the trade-off is small they might weight biodiversity conservation strongly, but if conservation requires giving up substantial food production, they are likely to weight biodiversity much lower.

For pragmatic reasons, trade-off analyses typically emphasize trade-off between two objectives. Fortunately there are many conservation planning problems that while not two dimensional, can be usefully summarised in terms of two dimensions. All analytical tools ultimately involve simplification of highly complex social-ecological systems and we believe that illustrating trade-offs can be a useful conservation planning tool, especially for problems involving two major considerations, say food and carbon, or carbon and biodiversity. Illustrating trade-offs between a couple of dimensions can be a strong advocacy tool through promoting constructive deliberation about unavoidable trade-offs, and provide a powerful basis for strategy selection.

Making sure strategies are as efficient as possible. One of the functions for which economists use trade-off analysis is to check that strategies are as efficient as possible, or in their language, Pareto optimal. A strategy is Pareto optimal when no improvement in one objective can be made without simultaneously diminishing the achievement of at least one other objective. Strategies that are not Pareto optimal are inefficient because they involve unnecessarily sacrificing achievement of one or more objective, whereas strategies along the efficiency frontier represent trade-offs that cannot be avoided.

Efficiency frontiers are easy to construct in theory but often difficult in practice. The standard approach is to take one objective, and across its entire range of potential values, find the best you can do on another objective. This is typically accomplished through optimization. Doing this optimization can be a serious computational task in its own right. A useful and more easily accomplished starting point is to simply plot the consequence for the two or three objectives of each of the alternatives being considered. Or alternatively develop a hypothetical set of alternatives that favour one or the other objective to varying degrees. For example, The Nature Conservancy and the University of Tennessee collaborated to develop a tool that helps propose a series of alternative infrastructure patterns for shale gas development and then site these along an efficiency frontier of project cost and environmental impact (Figure 15).

Figure 15: Environmental Impact vs. Additional Project Cost

Moving from illustration to strategy selection Arguably a more difficult task than illustrating trade-offs between strategies is actually deciding on the best option. In theory (and if we were truly passive objective observers, which of course we are not), all solutions located on the efficiency frontier are equally good. In practice this is where the influence of other objectives comes into play (either consciously or unconsciously). For example, in addition to the landscape scale trade-off between food production and biodiversity, a land allocation decision might be influenced by the economic interests of particular individuals. There are, however, ways to look at trade-offs to identify points which might be desirable. Trade-off curves often exhibit points where the rate of decrease in one objective increases rapidly as the other objective increase. These are referred to as points of inflection (Figure 16). Such points are not always obvious and there is no guarantee that these represent a desirable solution. However, they are often valuable to identify because they represent the point at which losses for each objective are minimized. With increasing distance from an inflection point, small improvements in one objective represent significant losses in another objective, and decision makers have to increasingly favour one objective over another.

Figure 16: Biodiversity vs. Food Production

Is it appropriate to frame something as a trade-off? Just because a trade-off can be illustrated as part of an analysis does not mean these things can actually be traded off. For example, some stakeholders might feel that things such as cultural heritage or the right to make a living off the land cannot be traded-off at all. Another way to think about this is that stakeholder’s assessment of consequence for a particular objective may have a clear threshold rather than being a continuous function, for example, cultural heritage can either be protected or not, there is no partial delivery of this objective. Wilderness advocates often hold similar attitudes. Great care must be taken to avoid trade-off analysis that appears callous because in many cases it is not some generalized commodity but actually people who will be affected by the trade-off. Similarly, from a political and communication point of view, formal trade-off analysis can present a challenge because it involves acknowledging the possibility that an action or policy may have negative consequences for values that people care about. It is not appropriate to frame something in this manner when the trade-off stands to negatively impact a vulnerable population.

Scenario analysis

Imagine that an effective global carbon market is in place and all nations have agreed to strict emissions caps that keep the price of CO2 offsets at $20 a ton or above. This guarantees a vast and sustainable flow of revenue to landholders, communities and governments who protect forests and peat bogs, virtually eliminating deforestation globally. This is a scenario, albeit an optimistic one. Scenarios such as this are not intended to be predictions, nor are they a choice that someone in charge of a conservation project could make, but rather they are learning tools. What would it mean for our potential strategies if the scenario above became reality? Analysing scenarios can help us understand how different strategies might fare in the inevitably uncertain future. Scenario analysis can therefore be a useful strategy selection tool where uncertainty is high.

All strategies have uncertainty associated with their outcomes because we cannot perfectly predict the future; scenarios describe some of this uncertainty through intentional manipulations of imagined futures. The basic premise of scenario analysis is to develop a small set of possible future scenarios that describe how some of the main uncertainties — such as demographic trends, policies, markets, budgets, degree of climate change or stakeholder support — might behave. Some of the most globally recognisable scenarios are the different emissions scenarios developed by the Inter-governmental Panel on Climate Change (IPCC) to explore uncertainties in national policies and social behaviour and the impact on climate change. The set of strategies being considered are then imagined to be occurring on the back-drop of these different scenarios with the aim of learning about how they would be expected to perform. For example, we might assess the performance of coastal adaptation strategies under each of the different IPCC scenarios or using sea level rise models with different assumptions. Sometimes scenarios might yield parameters that can be used in a formal predictive model (like the extent of sea level rise expected under each of the IPCC scenarios), but scenario analysis can equally be accomplished simply by describing how we think a strategy might perform under a set of future conditions. The aim of scenario analysis is to see how robust a strategy is to a range of possible futures, and therefore how much confidence we should have selecting it. This information can also be used to refine strategies to make them more robust.

As a straightforward example of evaluating strategies for conservation of urban ecosystems in and around Stockholm, Ulla Mortberg and colleagues (2012) explored the consequence of two possible urban growth scenarios. The first scenario they termed “Compact” in which urban growth policies emphasized energy efficiency and minimizing transport distances and costs. The second scenario they termed “Urban Nature” in which urban growth policies emphasized protection of urban green spaces and planned but distributed development. Mortberg and colleagues speculated that the first scenario would likely result in the loss of urban biodiversity and ecosystem values, which would subsequently diminish the engagement and support of urban communities with conservation. Some of the consequences may be offset, however, by reduced loss of biodiversity in the peri-urban area because of a reduction in urban sprawl. The second scenario, they suggested, maintained urban biodiversity and ecosystems, but required greater ongoing budgets for transport infrastructure and energy availability because of a growth in low density housing, some of which was likely to come at the expense of peri-urban areas. Even using these two scenarios highlighted the consequence for urban biodiversity projects under different policy environments, and made obvious the need to seek greater clarity around the biodiversity and ecosystem service value of urban ecosystems.

Once the performance of strategies has been evaluated under different scenarios, this information needs to be analysed in order to help select the best strategy. One of the most useful approaches is to combine scenario analysis with multicriteria decisions analysis.

When is scenario analysis appropriate? Scenarios are best suited to exploring situations where uncertainty is high and controllability is low. For example, climate change and global governance are largely beyond the control of conservation decision makers, even in a large region. In these situations, scenarios can help to illuminate the consequences of these global drivers of change and to formulate robust local responses.
How do you design scenarios? A generalized approach to scenario development would include the following process:
Identify the three to five most important drivers of change,
For each of these drivers, identify possible future trends with a small number of categories (for example, remain the same, small increase, big increase), or bifurcating decisions (for example, a policy is implemented or not),
Create a framework by grouping these trends along two to three axes of uncertainty,
Develop a set of coherent storylines (a narrative about what may happen in the future) that draw on the possible trends and cover as much of the space in the framework as possible.

There is a good deal of debate in the scenario analysis community about how ‘plausible’ each scenario and its storyline need to be. Part of the point of scenario analysis is explicitly not to focus on what is likely; extreme, low-probability scenarios can still be very useful in scenario analysis. It is critical, however, that scenarios are coherent; a scenario must encapsulate a coherent story about the future world. For instance, a scenario in which suburban areas, farmland and intact habitat all increased is unlikely to be a coherent story as development of suburban areas will generally occur at the expense of farmland or intact habitat.

References

Mörtberg, U., Haas, J., Zetterberg, A., Franklin, J. P., Jonsson, D., and B. Deal. 20012. Urban ecosystems and sustainable urban development–analysing and assessing interacting systems in the Stockholm region. Urban Ecosystems, 2012:1–20.