Small organisations guidance - Assess

Assess

This section covers data analysis – the process of bringing together the evidence that you have collected, and making sense of it in order to understand how much change has happened, for whom, and why.

In addition to covering key steps in analysis and making sense of change, this section also explains the questions you will need to ask yourself in order to get an objective picture of your impact overall. This includes thinking about the other factors that could have contributed to your impact, as well as how much change might have happened independently of your work.

3.1 We analyse our information to make sense of how and why changes occur for beneficiaries

This criterion focuses on the way in which you bring your evidence together during analysis to explain how and why your work makes a difference.

You will need to have systems in place to allow you to store and analyse your data. These may be on paper or using IT such as Microsoft Excel or a project or customer management system.

Explaining why changes occur is vital to understanding your impact. Using your data to explain change requires two steps to analysis.

  • Firstly, you will need to collate and analyse your data looking for patterns.
  • Secondly, you will need to interpret the data in the context of your work, trying to explain how and why changes might have happened. If you have collected information using more than one tool (eg, a survey and interviews) then you will need to synthesise then compare the data from those sources to see whether they support the same conclusions or tell you more about how and why changes occur.

This criterion is fully met if:

You compare and contrast different types and sources of data, in a way that helps you to understand why beneficiaries experience outcomes as a result of your work.

What next?

If you’ve met this criterion in full, you could improve your practice by…

  • Comparing different perspectives on change to try and understand why outcomes occur. For example, if you were working with young people, you might compare data from the young people themselves about outcomes with that from their parents and carers. Examining the way in which the two groups give different or similar explanations for change will help you to get a more in-depth picture of why outcomes occur and will strengthen your findings.

3.2 We look carefully at negative and unexpected outcomes, as well as positive outcomes

Considering your findings objectively is a tricky but fundamental part of data analysis.

A danger of all research is that, without objectivity, your data can end up reflecting a rosy picture of what you hoped to find, rather than an accurate picture of how things really are. This can mean that any negative or unplanned outcomes are ignored during analysis. However, the most important learning for your organisation may rest with the negative or unplanned outcomes that you discover, so it is important that these outcomes are given due weight during analysis.

Treating negative or unexplained outcomes with the same amount of care and attention as positive outcome data means spending the same amount of time on analysis, and asking the same key questions about why changes occur, who else is involved in creating change, and how experiences differ between groups.

This criterion is fully met if:

Negative and/ or unexpected outcomes are analysed just as carefully as positive and planned ones.

What next?

If you’ve met this criterion in full, you could improve your practice by…

  • Thinking carefully about how the way you collected data might have influenced your findings
  • Reviewing the way you collected your information will help you to make objective judgements about how important your findings (negative or positive) really are. This includes considering any technical problems during data collection or issues with missing data, as well as reflecting on how your own bias might have influenced the way you interpreted the information you collected.

3.3 We think through the other factors that might have influenced the outcomes that beneficiaries experience

In order to fully understand your impact, your analysis will need to consider the question of attribution – that is, an assessment of how much change was down to your work, and how much was down to the work of others.

Change is complex: in every context, there will always be a number of other agents involved in creating outcomes, including other organisations, other professionals, and even beneficiaries’ families and friends. Being able to recognise and describe the role that others play in achieving outcomes for beneficiaries is an important part of data analysis.

This criterion is fully met if:

You use what you know about the involvement of others in creating change to draw sensible conclusions about the extent to which other actors or circumstances may have supported or held back the achievement of your planned outcomes.

What next?

If you’ve met this criterion in full, you could improve your practice by…

  • Collecting information from beneficiaries about how other people support or undermine change
  • Asking beneficiaries directly about who else they have been accessing support from or other factors that may have helped or hindered their will help you to develop a basic understanding of attribution.

Resources for this section

Using IT systems to input and analyse data quickly and easily

Inspiring Impact’s Resource Hub lists database and case management sources for voluntary organisations.

ESS Support Guide 3.1 – Analysing Information for Evaluation is a set of guidelines from Evaluation Support Scotland that sets out how to analyse data, including numbers, narrative and visual materials.

ARVAC’s Getting Started has a chapter on data analysis for voluntary organisations that will support you through a step by step approach.

Sign-up to the principles

We need your help to put principles into practice