Impact evaluation
In today’s world, it is no longer acceptable for organisations to talk about success in financial metrics alone. In fact, there is an increasing need to demonstrate meaningful contributions to society through impact. Impact is defined as demonstratable contributions to the economy, society, the environment and culture¹. Impact assessments are methods to evaluate impact through a robust framework, which includes consideration of economic, environmental, and social aspects.
Impact evaluation covers a range of methodologies to assess the impact produced by or attributed to a particular intervention or activity, both intended and unintended. Rather than assessing whether an outcome is achieved, such as in traditional outcome research, impact evaluation focuses on understanding how the outcomes experienced by stakeholders have changed because of the intervention, relative to a world without the intervention. Impact evaluation is used as a way to apply evidence-based decisions to the policy development process. Originally more developed for the evaluation of social sector programs, impact evaluation is becoming increasingly useful and applicable in other areas such as energy, transport, and health.
What is Social Return on Investment (SROI)?
In 2000 a San Francisco-based philanthropic fund, the Roberts Enterprise Development Fund (REDF), started approaching its grant distribution process using a more holistic approach². To do so, REDF started measuring both economic and socio-economic returns on the invested capital. The returns were then compared to the investment. This is how Social Return on Investment (SROI) began.
SROI is a type of impact assessment that tells a rich story of how change has been or is being created, by measuring social, environmental, and economic outcomes i.e. triple bottom line. represented by monetary values. By comparing these outcomes to the cost of implementation, change is understood, and the wider value creation story can be demonstrated.
Stakeholders are at the heart of the SROI measurement process. It is central to involve stakeholders, beyond the traditional ‘experts’, and to include everyone who is impacted by the intervention assessed. The involvement of stakeholders and the ‘bottom-up’ approach to evaluating outcome metrics is a unique feature of SROI, which enhances the data compared to traditional cost-benefit analysis and leads to a rich story of change.
Why use SROI?
SROI has progressively emerged as a preferred technique for measuring impact and outcomes. SROI offers organisations a comprehensive approach to understanding and communicating impact returns, both internally and externally. Assigning monetary values to social returns provides a shared language to better inform decision making across multiple stakeholders and industries. By using stakeholder input, SROI provides rich qualitative insights as well as quantitative metrics to describe impact. SROI is used by Government, at both a Federal and State level, to evaluate programs due to the holistic approach and the ability to quantify intangible benefits³. In addition, the findings of an SROI can be assured through a rigorous international assessment process, which increases confidence in the validity of the impact assessment process and value creation. Assurance is more than a rubber-stamping exercise, but rather an independent evaluation of whether the social value principles were adequately understood and used in the analysis.
With the world moving towards a more sustainable future, evaluations of interventions and programs need to capture holistic outcomes, including impact on the environment. Consumers and investors are modifying their behaviour and demanding companies assess and act on sustainability. The ‘triple bottom line’ included in an SROI assessment allows the evaluation of multiple impacts simultaneously. This can be used proactively to guide investment decisions or to evaluate impact.
What is involved?
The SROI methodology is based on core principles that can be adapted to different types of industries and investments. The principles represent building blocks forming the basis of any SROI and of any decision that takes into account a wider notion of value, to increase equality, improve wellbeing and foster sustainability.
The value of these principles lies in applying them collectively. By applying the SROI principles, it is possible to create a consistent and credible account of the value that is being created. To do so, six stages have been designed to carry out the analysis:
What can I use the results for?
SROI analyses can be used for multiple purposes. Broadly defined, these include evaluation, communication, and advocacy. The SROI is less about the ratio and more about the process, what the research uncovers about the value of an intervention and the impact of a program. Nonetheless, the ratio of return-of-investment is an objective measure. These aspects of the analysis alone show the versatility of SROI, and how it can be used to communicate to different audiences and for different purposes, but always with a solid and consistent message.
SROI can be used to evaluate various investment and funding options or to provide justification for already chosen ones. SROI can be undertaken either retrospectively, to measure outcomes that have already occurred, or prospectively, to predict how much value will be generated if the intervention meets the intended outcomes, providing flexibility in the analysis objectives and uses. Prospective analyses can be used to understand likely future impacts, as well as determine where to allocate resources to ensure change is implemented and tracked. Retrospective analyses are valuable to evaluate the performance of the program.
Regardless of how it is conducted and what it is conducted for, SROI provides rich data to understand where change is created and answers the critical questions of what changes matter and how valuable they are to the people impacted by them. There are multiple examples of this versatility, from assessments of Indigenous Protected Areas (IPAs) conducted for the Department of the Prime Minister and Cabinet, to the SROI of communities created by Mirvac and the forecast SROI conducted to quantify and communicate the social outcomes generated when people use Fair Finance Australia’s services .
How do I get started?
The key starting point in the SROI process is to determine the purpose of the analysis and the likely audience. The purpose will determine the SROI approach, including whether a prospective or retrospective analysis is more appropriate. The audience will determine the level of rigour and detail required. Asking these questions upfront will guide the decision on how to proceed in the assessment. Occasionally, depending on the answer to these questions, a full SROI will not be required, and different parts of the framework will be enough to understand the effectiveness and efficiency of a program. In other situations, a more robust and rigorous level of evaluation will be required.
After defining the purpose and audience of the SROI analysis it is important to consider feasibility. This includes starting to build a theory of change, where the changes due to the intervention or program are linked to outcomes and impact. This will help determine if enough change that would have not occurred otherwise is created, to demonstrate a positive return. The theory of change is also a powerful tool to test assumptions, plan the evaluation, and sharpen messaging.
How can HTANALYSTS help?
Through extensive experience in the healthcare space, HTANALYSTS has developed a scientifically rigorous approach to SROI. We offer two levels of support, including a Premium SROI analysis service and Coached SROI, where we support the client’s team to leverage internal company resources in conducting the analysis. In a coached SROI project, we start with a training workshop to educate the client’s team on the core principles of SROI, and we then collaborate to build a ‘Theory of Change’ tool and a roadmap for the project. We provide expert support and guidance along the way which is facilitated by our powerful and secure collaboration platform Ripple.
HTANALYSTS impact assessment experience and services go also beyond SROI, with examples in the public domain including assessment of the impact of salt reduction in low- and high-income countries, greater use of innovative diabetes medications, and unintended pregnancies in Australia. SROI examples include an analysis of high-cost innovative pharmaceuticals targeting rare diseases, as well as the SROI of surviving non-curative cancer for parents with dependent children.
__
