Qualitative Impact Protocol

Published27th January 2017

The Qualitative Impact Assessment Protocol

The QuIP addresses the issue of how to attribute changes to different stakeholders or events, whilst minimising pro-project and other sources of bias, and avoiding the need to interview a control group. It is designed to work alongside existing quantitative monitoring of key indicators, adding qualitative, self-reported attribution of impact to provide sufficient evidence to test the theory of change behind the activity being evaluated.

Put simply, the QuIP provides a straightforward and cost-effective mechanism to ask intended beneficiaries about significant drivers of change in their lives, and to analyse and present the data collected.

Features

Key features

There are strong ethical grounds for asking people directly about the effect of actions intended to benefit them, and doing so can also contribute practically to detailed learning, innovation and wider accountability within your organisation. However, this approach entails finding credible ways to address potential response biases. The QuIP does so by arranging for qualitative data collection to take place without any reference to the project being evaluated, ensuring that field researchers and respondents are not briefed on the project being evaluated and thereby reducing confirmation bias as much as possible up to the point of data analysis. The analysis is then carried out by a separate party, who is fully briefed and can therefore interrogate and code the data against the theory of change. The aim of separating these roles is to ensure that the analysis remains as independent as possible.

The second key problem with recording detailed feedback from beneficiaries is what to do with all the data? Qualitative information is difficult to process, analyse and condense into a credible and transparent report. We have developed a simple and semi-automated approach to coding QuIP data which allows for easy analysis and reporting, ensuring that reports are brief, readable and that the frequency of findings cited and the source data are completely transparent. The coding system is detailed in the briefing paper and full guidelines.

By creating a systematic approach, we have speeded up the time required to undertake the assessment ‰ÛÒ thereby reducing the costs considerably. Whilst it isn‰Ûªt possible to fix a price on a QuIP as every country has different costs, it is safe to say that it will cost a fraction of the cost of randomised controlled trial, or any assessment requiring a control group. The following timings may help calculate approximate costs:

1 week of design and preparation

2 weeks of 2 field researchers‰Ûª time, both in the field and writing up

1 week of analysis and writing up

Services

Training and software to support analysis and ensure the tools remains robust

Case Study

http://qualitysocialimpact.org/wp-content/uploads/2016/04/SHA-Assela-QuIP-Report-2015.pdf

Existing Users

  • Oxfam
  • Diageo (CSR)
  • Acumen (Social impact investors)
  • Self Help Africa
  • Farm Africa
  • Tearfund

Sign-up to the principles

We need your help to put principles into practice