Brendan McDonnell, Director of Inspiring Impact NI partner CENI, introduces Measuring Change – An approach to outcomes for the voluntary and community sector, a new report which sets out CENI’s approach to helping organisations plan for and capture outcomes.
Voluntary and community organisations exist to make a difference to the lives of the most vulnerable and deprived in our society. This is complex and difficult work, which has received considerable investment over the years, yet as we all know, it still remains very hard to pin down the measurable difference made to people, organisations and communities.
Austerity has placed an even greater imperative on public funders to show the impact of their investments and for funded organisations to evidence the difference they make.
Recently Community Evaluation NI (CENI) published a report on an approach that has been developed and tested over a number of years: Measuring Change. This attempts to provide a practical, low-cost, yet rigorous approach to capture the ‘hard-to-measure’ qualitative difference that voluntary and community activity makes to people, organisations and communities.
Measuring Change is basically a synthesis of methods which draws on theories of change and empowerment evaluation approaches and uses a tool known as ‘nominal group technique’. Crucially it advocates collaboration between funders and funded organisations to specify the differences they want to make in advance and then gather data to estimate progress over time. This involves the following steps:
1. Define change:
Within a funding programme CENI works with key funder and project representatives to develop a theory of change and produce an agreed programme Outcomes Framework.
2. Capture change:
Employing the nominal group technique, CENI facilitates project stakeholder groups, drawing on their collective knowledge to produce agreed estimates of:
- Baseline position of the project against each outcome at a starting point; and
- Change or progress made against each outcome at key intervals.
This collects numerical estimates of change coupled with narrative data providing the rationale and evidence for change produced.
3. Show change:
Applied systematically across multiple projects, this produces a wealth of quantitative and qualitative data in a standardised format. The data can then be analysed to enable:
- Projects to demonstrate progress made from their baseline; tell the story behind the change; and evidence their contribution to achieving programme outcomes.
- Programmes to review projects’ progress within a common framework of outcomes; identify common themes and learning across diverse projects; and aggregate project data to provide an indication of programme impact.
The report also provides a series of case studies to illustrate how Measuring Change has been applied in different settings. These range from a pilot study with the Department of Agriculture testing the method to baseline community development infrastructure in rural areas, through to assisting a community health project to capture and report on some of the hard-to-measure qualitative outcomes from its work with older people.
While still a work in progress, the pilot work on Measuring Change is already attracting considerable attention and its potential benefits may be summarised as follows:
- Relatively robust and low–cost approach to outcome measurement at a time of financial austerity;
- Facilitates collaboration between funders and projects to identify and capture outcomes;
- Provides a strategic focus, enabling projects to align diverse activities against common programme outcomes;
- Supports critical peer review resulting in more realistic and honest evaluation which informs learning and improvement;
- Helps to shift focus of evaluation from counting activities to measuring change; and
- Generates useful data to inform decision-making at project and programme level.
Measuring Change – An approach to outcomes for the voluntary and community sector is available from CENI or can be downloaded from www.ceni.org.