Evaluating the progress made in climate change adaptation using indicators of resilience, Part II: Metrics for adaptation

Part 2: Can We Measure Adaptation Using Terms of Human & Environmental Well-being?

As discussed in Part 1, adaptation is a highly ‘context-specific’ activity. What works in one geographic, livelihood or climatic context might not work in another. Assessment of adaptation will therefore require highly context-specific approaches. In short, there are no universal metrics for adaptation as there are for mitigation (i.e. avoided or actual emissions, or global greenhouse gas concentrations).

The closest we have to universal indicators (metrics) for adaptation are standard measures of human wellbeing that represent factors such as poverty, health, livelihood performance, losses from climate-related extremes and disasters, and any other measures of human development that are ‘climate-sensitive’.

We might also define well-being metrics for non-human systems, for example measures of the health or functioning of ecosystems, of biodiversity, or of the size or stability of plant and animal populations.

If adaptation is successful, then these measures of well-being should remain stable or improve despite the intensification of climate stresses and shocks that would otherwise undermine the well-being they represent.

However, using such measures to assess adaptation effectiveness is extremely problematic.

Adaptation will not reduce the effects of climate change on human well-being or natural systems to zero, meaning that these well-being-type metrics will still vary as a result of changes and variations in climate. Trends in these metrics therefore need to be tracked over long periods if we are to tell whether human well-being is indeed improving – or at least stable – in the face of climate change.

We might compare the consequences of two similar climate extremes or disasters before and after an adaptation intervention, but this requires the occurrence of comparable climate events on ‘convenient’ timescales, and the elimination of other factors that might have influenced the underlying vulnerability or sensitivity of populations and systems to such events.

Alternatively, we might compare the consequences of a climate extreme or disaster between those who have received adaptation support and those who have not, using methodologies such as randomised control trials (RCTs) or the ‘difference in difference’ approach. While these approaches are increasingly popular in the evaluation of development interventions, RCTs are resource intensive and applicable only to some interventions, and any comparison must be made between similar populations exposed to the same shocks or stresses (something that is not always feasible). Again, such assessments need to be conducted after sufficient time has elapsed for an intervention to have influenced people’s circumstances, and depend on the occurrence of a climate shock or stress at a time ‘convenient’ for the evaluation.

Finally, we need to face the fact that adaptation might be palliative. Development indicators may tell us that human well-being is deteriorating in the face of climate change, but adaptation may have prevented an even worse deterioration.[1]

The emerging landscape of adaptation M&E

Faced with the above challenges, the tendency among donors has been to develop results frameworks that focus on outputs (i.e. goods and services delivered by a programme, rather than positive changes resulting from these goods and services), spending and value for money. This was the conclusion of the Independent Evaluation Group of the World Bank in 2012, and not much seems to have changed since then.

The bottom line for most donors seems to be that moving beyond the evaluation of outputs is simply too difficult and resource intensive. As a result, monitoring and evaluation (M&E) systems for most climate funds and adaptation-related programmes will tell us little about whether they have been truly effective in delivering adaptation.

Nonetheless, techniques are emerging for assessing the success of adaptation using a combination of wellbeing metrics and climate data, and by focusing on shorter term changes in people’s circumstances that affect their ability to cope with climate shocks and stresses (i.e. their resilience).

The Tracking Adaptation and Measuring Development (TAMD) framework[2], funded by the UK government’s Department for International Development (DFID) and led by the International Institute for Environment and Development (IIED) , advocates using a combination of wellbeing, resilience/vulnerability, and climate risk management indicators to assess the extent to which actions by institutions are delivering improvements in the resilience of populations “on the ground", and whether these improvements in resilience are in turn leading to improved wellbeing in the face of evolving climate stresses and shocks.

Guidance on measuring resilience has been developed for the UK's International Climate Fund, and is being piloted by DFID'S BRACED (Building Resilience to Climate Extremes and Disasters) programme.[3]

[1] Methodologies for assessing adaptation effectiveness on long and short timescales using standard development or ‘well-being’ indicators in conjunction with climate data are currently being developed as part of the DFID-funded, IIED-led Tracking Adaptation and Measuring Development (TAMD) framework and associated work. However, these methods have not yet been tested and are at the early stage of development.

[2] Further information on TAMD can be found on the IIED website, from which working papers, briefing notes, country reports and other relevant TAMD publications can be downloaded. See www.iied.org or search for “IIED TAMD”.

[3] See the guidance on measuring changes in resilience developed for reporting against Key Performance Indicator 4 (KPI4: Number of people with improved resilience as a result of programme support) of the UK’s International Climate Fund (ICF).

This blog is in 5 parts:

Part I

Part II

Part III

Part IV

Part V