Too complex to measure? Exploring the dynamic processes of resilience and recovery

Different understandings of what makes someone, or something, resilient, influence how it is measured. Is resilience an end goal in itself? Is it a means to an end, towards a greater or higher-level development objective? Taking a version of resilience thinking that encompasses not just anticipating and coping with shocks and stresses, but longer term recovery and adaptation, resilience becomes a process of change towards reducing the impact of future events, particularly for the most vulnerable. These change processes are non-linear. Outcomes are often uncertain or even unexpected and can take a long time to manifest – often longer than the lifecycle of any single intervention. How do we measure something so intractable? How do we do this in a way that is meaningful, and that represents the complexity of development processes?

At the third convening of the Resilience Measurement Community of Practice in Uganda and days spent afterwards at the African Evaluation Association conference, several strands of resilience thinking started to come together. Four aspects struck me as challenges that provide opportunities to improve our approaches as a measurement community.

One size does not fit all

There are multiple frameworks for resilience measurement. Many of these frameworks share commonalities, but in recognition that one-size does not fit all, emphasis has shifted to selecting frameworks that are appropriate for each context. An understanding of the uniqueness of each context is not new, but the shift away from seeking convergence in resilience thinking and in measurement frameworks – from seeking ‘one universal approach’ – towards an emphasis on valuing a diversity of approaches is. This shift embraces the multiple dimensions of resilience, the variety of focal areas for resilience building approaches, and brings into focus the appropriateness of different ways of measuring. This plurality brings challenges in comparing resilience measurement, but having a clear ‘purpose’ to each approach will help us to share knowledge, reflect and learn from each other.

Networks and dynamics

The shift towards multiple approaches has brought into focus some gaps in resilience measurement, including the limited number of frameworks focussing on urban resilience and infrastructure. Resilience thinking developed within different disciplines and fields draws out different measurement dimensions that are important and relevant for each. New perspectives bring opportunities to learn, reflect on current practice and to think differently. For example, focusing on infrastructure and urban contexts highlights challenges in thinking about interconnections and dynamics, particularly centred on the interactions between people, nature and the built environment.

This builds on a growing understanding within the resilience measurement community of how to capture aspects of vulnerability and capacity, to also raise challenges in how to incorporate and measure the dynamics within an interconnected network, or ‘system’. This framing draws in concepts from systems thinking, particularly complex adaptive systems that are able to self-organise, to learn from the past and adapt to changing circumstances. This flexibility and dynamic behaviour is an important characteristic of resilience – one that people have exhibited for millennia in adapting to their changing environment. In complex systems, it is the interactions between different elements of a system – social, physical, environmental – reaching across scales, and the emergent properties of the ‘whole’ that become important.

Measurement needs to understand and describe the dynamics of a system: the networks of relationships, connections and interactions within each unique context. These linkages take many forms – social, physical, organisational, economic, informational – creating a complex web of interdependencies. This brings new challenges and new opportunities to resilience measurement. How can we measure a network of relationships? Which relationships are most relevant and influential in each context? For who? How can we use this new understanding to enhance resilience interventions within each unique context? There are approaches that may help to explore and understand this complexity, for example, network approaches and tools such as Social Network Analysis offer some promise. Overall, these questions mean digging deeper to understand the unique dynamics of each context and there is a need for methodological innovation and experimentation.

Recovery and changed contexts

Although there was a lot of discussion about the complexity of resilience, non-linear change processes, and uncertain outcomes during the convening and conference, it struck me as surprising that there was very little discussion of a critical gap – recovery. The ability to ‘recover’ is included in many of the definitions of resilience, but there is a real lack of empirical evidence to support when or how recovery happens, or how long it takes. Different groups of people experience different impacts from disasters and recover at different rates and in different ways. There is growing understanding that recovery processes may take years or even decades, but also an acknowledgement that sometimes people don’t recover.

A shock or stress that triggers a disaster may affect physical, social, environmental, and economic systems, and can lead to a change in the context that is no longer comparable to pre-event circumstances. This ‘new normality’ creates challenges for resilience measurement – how to we measure resilience meaningfully before and after a shock when the context is fundamentally different? How do you know when something or someone has recovered if it looks different to before? And how are recovery processes, and resilience, affected by multiple shock and stresses? In amongst all of this, there remains a focus on ‘shocks’, over and above more frequent, but lower-impact and often more geographically widespread, stresses. The reality is that both challenge people’s resilience. As a resilience measurement community, we need to explore new ways of understanding resilience in chronic crises and, looking at pre- and post-disaster contexts, find ways to meaningfully interpret resilience outcomes. This is again both a challenge and an opportunity for innovation and creative thinking in resilience measurement.

Unexpected outcomes

Overall, given the complexity of resilience, there is a drive to prioritise measurement and to measure more efficiently, which can force a narrow lens on the way that we view and measure resilience. This can promote the use of reductionist approaches to break resilience concepts down to component parts, to form (often many) measurement indicators. Through this narrowing, we may be missing unexpected outcomes and critical causal pathways that are triggered by non-linear change processes and dynamics. As the theme running through this thinking, holistic, system-wide views of resilience may help to guide resilience measurement in a way that accounts for interdependent relationships and dynamics, emergent or unexpected outcomes, and the many ways in which contexts may have changed when exposed to shocks and stresses.

Wider adoption of qualitative approaches and integration of network analyses offer opportunities to ‘dig deeper’ into complexity and dynamics. Of course, certain evaluation designs are also better able to account for emergent outcomes – theory-based evaluations, and approaches such as outcome harvesting, most significant change, collaborative outcomes reporting, realist evaluation and contribution analysis are better able to capture both expected and unexpected outcomes and explain how and why they came about. DFID’s flagship resilience programme, BRACED, is testing some of these approaches.

There is a real opportunity for innovation and experimentation to improve resilience measurement. By drawing from different disciplines and fields, embracing the variety of possible approaches, and focusing on what is most meaningful in each context we can continue to learn, diversify and improve resilience measurement practice. Not too complex to measure then, but different approaches are needed to do it well.

This piece follows on from a previous blog. Click here to view related M&E content.