Data with destiny: How to turn your charity’s data into meaningful action
Sally Bagwell, Erica Bertolotto and Katie Boswell
Data is essential for an effective charity. And according to our State of the Sector research, many charities are already aware that evaluation findings can help them achieve their mission. But in our work we often find that charities are intimidated by data. While some are successfully collecting data, they are often failing to use it to review, learn and improve their work. Some are collecting vast amounts of data but struggling to turn it into insights. Others are gleaning insights from data but not acting on them. It doesn’t have to be this way. When used properly, data can help you:Data is essential for an effective charity. And according to our State of the Sector research, many charities are already aware that evaluation findings can help them achieve their mission. But in our work we often find that charities are intimidated by data. While some are successfully collecting data, they are often failing to use it to review, learn and improve their work. Some are collecting vast amounts of data but struggling to turn it into insights. Others are gleaning insights from data but not acting on them. It doesn’t have to be this way. When used properly, data can help you:
- Understand the needs of your users, the quality of your services, and the difference you are making to people’s lives. This will inform your strategy, programme design, and resource allocation.
- Support staff in their professional development based on what works in project delivery.
- Report to funders, commissioners and policymakers—influencing their views and the way they fund and support charities.
Data should play a key role at each stage of the iterative impact cycle—as charities plan how to achieve their impact, do their work and collect relevant data, assess their impact, review their findings, and then feed back into their plan.
We spoke to a dozen charities of different sizes that work on a variety of issues, to understand the challenges they face in turning data into action and get their advice on using data effectively. We asked:
- Do you collect data? What is the purpose of your data collection?
- Do you learn from the data? How?
- Do you use that data to inform your services and strategy? How?
- What are the benefits and challenges of using data in this way?
This paper shares our thoughts on how charities can move from data to action. It draws on our experience of working with large and small organisations across the sector for over fifteen years, and on interviews with charities about their experiences of using data.
The best charities are transforming data into wisdom and then action
There is real power in moving from data to wisdom, as demonstrated by many of the charities we have spoken to. Because it is this wisdom that enables charities to make informed, strategic decisions.Behind Closed Doors, for instance, used its data to understand that, rather than just working with parents, family-wide approaches are more effective in reducing domestic violence. This insight improved their services, and it also helped the charity to win funding and expand its work with children. Similarly, Street League’s data showed that it was predominantly engaging male participants through its football and employability programmes, but the few females who came through the programme successfully progressed to sustainable employment, education and training. The Street League team used their data and research to design a new programme in which 70% of participants were females. Street League has achieved greater impact and access to additional funding as a result.
Turning data into informed decisions is a process
The Data, Information, Knowledge, Wisdom (DIKW) model illustrates how raw data becomes the wisdom. It is this wisdom upon which we base our decision-making, the final stage of turning data into action. When applied to charities, the DIKW is helpful. It breaks down the steps needed to go from collecting data to using that data in order to implement programmatic and strategic changes.
Data is the raw qualitative and quantitative—words and numbers—input collected through a charity’s monitoring activities, such as:
‘100 boys and 50 girls’.
This data must be given meaning through an understanding of words and numbers, turning it into information:
‘100 boys, 50 girls reached by our programme’.
Information then needs to be analysed and understood within the context in which the charity operates to become actionable knowledge:
‘Boys and girls are equally at risk, but we are reaching twice as many boys as girls.’
The knowledge acquired provides an understanding of what has happened in the past and why. The step to wisdom involves applying the knowledge gained from the data to make decisions about what actions to take in the future:
‘We will adjust our strategy and deliver a targeted intervention to reach more girls through our outdoor education programme.’
This process requires several assets
The charities we spoke to identified three main elements that help them to use their data to review and improve their strategy and programmes. Each of these points is essential to move up one step in the DIKW model:
Systems allow for good quality data to be collected and organised, so that it can be understood and turned into information.
People with the right capabilities then analyse and learn from the information to transform it into knowledge.
An organisational culture that is committed to impact will act on the data so that it becomes wisdom about the future direction of the organisation.
Moving from data to wisdom is an iterative process. The DIKW model might suggest that it is a linear process, but in reality you will need to keep interrogating your data. This will help you understand not only what outcomes have been achieved but also why, and what to do about the findings. You also need to keep reviewing the way you collect and analyse data to make sure it is good quality and useful. Data collection is not an end in itself. It is only useful if charities analyse the data and understand what they have achieved. IntoUniversity’s Chief Strategy Officer Dr Hugh Rayment-Pickard emphasised that charities need to question data and reflect on what the analysis is suggesting to fully understand what they have achieved.
‘It’s easy to collect data, but hard to conclude what impact is. It’s important to constantly reflect on data and question it.’
Hugh Rayment-Pickard, Chief Strategy Officer, IntoUniversity
Systems, capabilities and culture are cross-cutting assets and in some ways apply to every step on the DIKW model. But it is helpful to look at the specific role each asset has at different stages of turning data into action.
Step by step: How to transform your data into wisdom
Data: Collecting the raw material
Collecting good quality data is essential—this is the raw material upon which everything else is built. The data that charities collect from their users and services can be grouped into five main types:
- User data: how many people are using a service and their demographic characteristics, such as age, gender, ethnicity, etc.
- Engagement data: how often people use a service, and for how long.
- Feedback data: what people think of the service and their experience of it (from users), and an internal assessment of the quality of the service provided (from staff and volunteers).
- Outcome data: the capabilities, strengths, assets, knowledge people gain as a result of the intervention.
- Impact data: what long-term change users experience beyond the lifetime of the intervention.
Consider also what external data you need to gather to benchmark your services and better understand the context in which you operate. Oxfordshire Mind collects its own data but also looks at external data, such as: demographic data; NHS Digital’s Qualities and Outcomes framework (QOF) data to map the incidence of mental health; and data on employment and social deprivation. Demographic data, for example, showed an increase in the Indian and Pakistani populations in Oxfordshire that was not reflected in Oxfordshire Mind’s users. This analysis led the charity to deliver targeted outreach activities to understand the barriers preventing Indian and Pakistani communities accessing their services.Behind Closed Doors compares its own data to the data collected by similar charities working in the same area, to understand trends and adapt its interventions. When the charity found that the number of 16–17 year olds accessing their service was increasing, it consulted other domestic violence charities in the areas to understand if they had also noticed a similar trend.
Data to information: Setting up systems
Raw data must be collated and organised in a systematic way before it becomes meaningful. Systems include processes, structures, tools and policies around data—for example, assigning responsibility for different aspects of impact management, deciding on timings, choosing an appropriate outcomes management or client relationship management (CRM) system, and setting appropriate policies around data. When handling user data it is essential that charities are aware of and respect data protection law. Developing an organisational policy may help you with this. Street League have a data governance policy on responsibility and ownership of data.
What should you consider when choosing systems?
- Do you need data to be collected on the move?
- Will different people need different levels of access to the data? Can it attach outcome data to individual service users’ records?
- Does the system need to be compatible with your existing systems?
- Do you want to analyse data in the data collection system or export to another programme?
- Do you want to have data saved on your own servers, or kept on a third party database?
- What resources and IT expertise are available to maintain the system?
- What is the payment structure? Flat rate or for every user?
Consult your people
Developing data collection systems should be done in consultation with the staff members who will collect and input the data into the system. Positively UK involved its frontline workers, volunteers and users in developing its data systems, to ensure that they would be proportionate to the organisation’s size, evaluation capacity and measurement priorities. The process involved reviewing their existing outcome measurement tool. As a result it developed a new tool that the staff use both to collect data and for case management.
Equip your team
Consultation is not just important for designing your systems, but also for implementing them. Action for Children spent a year consulting with different internal stakeholders and getting buy-in for developing new systems. Good communication was crucial to ensuring all departments and staff at all levels participated at this stage. It then spent a year designing the systems involving all staff—especially frontline staff to ensure the system is practical and suitable for the organisation. When it began to collect data it found that, because staff had been involved in developing the system, they trusted it. This meant they felt incentivised to input data and willing to use the data to influence the way they deliver services.It goes without saying that training staff on how to use your system is important, and will improve the quality of what you can collect. To ensure systematic data collection, Behind Closed Doors offers staff regular training to understand and use its system as part of their day-to-day work. Allocating a suitable amount of resources is also essential. Most charities are stretched for time and funding so it is important to think creatively and broadly about how to resource impact management. There are a variety of free tools and resources on impact management available online, for example on Inspiring Impact’s Impact Hub.
Review your system
Systems development ought also to be iterative and ongoing. Once set up, you will need to regularly review your systems to make sure they are suitable. For example, Place2Be reviewed its data collection system after delivery staff said they felt that the Strengths and Difficulties Questionnaires (SDQs) for young people receiving counselling did not provide a complete picture of each young person’s journey. The charity decided to add a new ‘Child Voice’ form to the information it collects. This allows staff to gather the young person’s perception of counselling and what they aimed to gain from going to Place2Be.Finally, remember that you do not need to collect data on everything you do. There is a vast amount of existing evidence on the effectiveness of different interventions. This can save you the time and resources of testing whether your approach works. You can then focus on ensuring you are delivering your intervention effectively.
Information to knowledge: Building your capabilities
Turning information into knowledge requires capabilities: the ability to analyse the information, interrogate it, understand it and learn from it.
What capabilities do charities need in order to analyse data?
Charities need to have access to data analysts with the ability to:
- Clean and manipulate data—for example, manipulating data in Excel or using tools like Open Refine (openrefine.org) to work with messy data.
- Extract insight from data through rigorous analysis, for example analysing quantitative data in Excel.
- Understand the data they are analysing, ask the right questions, and interpret results. Analysts need enough sector expertise to build appropriate hypotheses and understand the results of analysis.
Finding people with the necessary capabilities to analyse data can be a challenge. Many organisations specifically recruit people who are keen to learn from data and who have the capabilities to carry out data analysis. Others train existing staff to meet their new data analysis needs.
Ask for help
Not all charities need sophisticated data analysis capabilities in house, so consider whether recruiting someone with those skills is the best approach. Data analysis can be outsourced to organisations such as DataKind. Some organisations, such as Street League, have accessed pro-bono support to review their data and help with the analysis. Others, such as Save the Children UK, hire external independent consultants. These have specialist experience of conducting evaluations, as well as knowledge of the organisation’s thematic programme areas and country contexts.
Get the right people on board
Recruiting trustees with measurement and evaluation experience is also a way to access skills your charity might not otherwise have. It will also help ensure the board helps to drive good impact management. Positively UK felt it could not afford to recruit measurement and evaluation staff, so instead it recruited two trustees with measurement skills, who played an important role in making impact management a priority in the organisation. Similarly, Place2Be has a long-established research and advisory group as a subcommittee of the board. The committee reviews the organisation’s data analysis and informs its research.
Upskill your staff
Pivotal to moving from data to wisdom is the ability to understand, interrogate and interpret the data. Be flexible and regularly review whether you are analysing your data in the best way. Then adapt your capabilities as the data you collect and the systems you use for data collection change.
Presenting the data analysis in an accessible format to staff at all levels is a great way to get everyone involved in impact management. It will also ensure that information translates into changes at the programmatic and strategic levels. At Place2Be, school staff who collect data can see all data that is relevant to them on their system—including information for individual children and analysis of data at the school level, cluster level (local grouping of schools), by region, and nationally. This allows them to learn from the data and make changes to their practice.At IntoUniversity, the impact manager is responsible for analysing data. This analysis is then made available to all staff, through dashboards, almost in real time. In this way, frontline staff can see how their intervention compares to other parts of the country and make small changes to the way they deliver the programme. For example, one centre had much higher attendance than average at parents events. They realised that this was because they had sent out the invitations in gold envelopes, which attracted the attention of parents and made them more likely to open them. Now all communication to parents is sent in gold envelopes.
Knowledge to wisdom: Creating an impact culture
An organisational culture that is committed to impact enables charities to use their new knowledge to improve strategy or services. Changing culture is often the most challenging component in creating an impact-led organisation as it is intangible and hard to pin down. However it can often be the factor to make or break a successful strategy or use of data.
What should you consider when reviewing your culture?
- Impact management needs to be valued across the organisation, and the desired impact needs to be clearly defined and communicated to all staff.
- All staff need to feel part of the mission and responsible for impact. This includes making impact everyone’s job—not just the evaluation team’s. At Street League local teams can choose the evaluation questions they ask. Different regions and roles are encouraged to collaborate and learn from each other. Making everyone responsible for impact might require changing systems, reviewing job descriptions or training staff. There may be resistance to such changes and it is the leadership’s responsibility to manage change.
- A culture of impact is a culture of learning: data collection and analysis should be forward facing, aimed at planning the future rather than just for reporting.
- It takes courage for charities to interrogate their data, to learn from it, without fear of what it might reveal. If change is not feared, organisations will talk openly about failure and data can be used to improve.
Creating an impact culture is certainly challenging, but all the organisations we spoke to stressed its importance. Culture is difficult to define, but there are some key points that charities should think about in creating an impact culture.
To embed impact practice in its work, Street League is shifting from using data as a performance management tool to also using it as a professional development tool. This has contributed to creating an impact culture, where data is used to support staff to improve the way they work, rather than as a way of judging their effectiveness. Street League also has a data champion in each region. They are responsible for communicating the organisation’s impact priorities and ensuring that everyone across the organisation sees impact practice as part of their job. Data champions meet quarterly to share best practice, learning, and to reflect on why change is or is not happening a certain way.
Leadership that builds an impact-focused culture is essential to turning the knowledge gained from your data into the wisdom needed to improve programmes. Positively UK highlighted that it is the CEO’s responsibility to communicate what the charity learns from its data and to act on it. The CEO can set an impact culture but will need to enable others to own the organisation’s impact as well. A senior management team that is committed to impact will set up the right systems, allocate resources and recruit staff and trustees with the necessary capabilities and commitment.
The role of the board in leading for impact is also important. Save the Children UK have an internal impact board that meets monthly. This board reviews approaches and ideas that need strategic oversight, and makes recommendations. It also produces an annual internal impact report informed by external evaluations and data collected internally. The impact report helps them understand what allowed (or did not allow) the organisation to achieve impact, and whether the systems are fit for purpose. It also makes recommendations on how to improve.
An organisation in which the leadership is committed to impact will see this focus lived out in its actions and decisions. Place2Be created a new role that loops learning into practice and helps the organisation convert the learning from their data into wisdom and then action. It found that a small number of children were not benefiting from their one-to-one counselling service. So field workers were asked to collect additional contextual information to determine the reasons for this change. Place2Be now proactively reviews and records all these cases and works to identify other sources of support within the school or external agencies for these children. Cross-organisational buy-in was instrumental to making these improvements.
It’s a journey, so just get started
- Be proportionate. A large and well-resourced organisation implementing complex programmes will need more rigorous methodologies than smaller organisations with limited access to resources and capabilities. As IntoUniversity said, ‘randomised control trials aren’t necessary for most charities, what is essential is involving users in planning, implementing and reviewing your programmes’.
- Involve staff and users in developing your systems. Chickenshed wanted the process of developing a measurement framework to be staff-led, so it held a workshop with staff and the young people using its services, as well as a workshop with parents.
- Allow time for the process. It can take a few years to set up the right systems to understand your data, and to transform it into visible changes for users.
- Leadership needs to drive the process. Street League said its board and its CEO lead the organisation to make decisions based on its data.
- Make impact part of everyone’s job. As Positively UK told us: ‘Make data collection integral to the work of frontline staff, not an “add-on”.’
- Let your mission guide how you use data. Oxfordshire Mind’s top tip is: ‘If you’re going through a process of culture change to become more impact focused, stay true to your values and objectives.’
- Get some quick wins so that people see the value of data. Behind Closed Doors realised from its client feedback data that people wanted to be able to access services outside of standard working hours. It used this insight to redesign its services and is currently developing an online support offer to reach more people.
Introducing all of these elements may seem daunting, particularly for small charities or those who are just starting to think about their impact. But it does not have to be that way. Charities do not need to have achieved all of this to start meaningfully using data to make strategic decisions to improve their services. All the charities we spoke to are on a journey to make the most of their data. The most important thing is to start somewhere and build on what is already in place—whether that is a basic data collection system, a committed leadership, or staff capabilities.
Free guides and publications
Hoare, G. and Noble, J. (2016) How to make your data more meaningful. New Philanthropy Capital.
Boswell, K. and Kazimirski, A. (2016) Keeping it in proportion: Impact measurement for small charities. New Philanthropy Capital.
Gyateng, T. (2015) Protecting your beneficiaries, protecting your organisation: Ten considerations for charities on safe use of personal data. New Philanthropy Capital.
Noble, J. and Thorne, M. (2016) Stories and numbers: Collecting the right impact data. New Philanthropy Capital.
de Las Casas, L., Gyateng, T. and Pritchard, D. (2013) The power of data: Is the charity sector ready to plug in? New Philanthropy Capital.
Breckon, J. (2016) Using research evidence: A practical guide. Nesta.DataKind UK and Data Orchard developed a data maturity framework to help you assess at what stage you are in your journey using data. If you are at the more advanced stages of the framework, DataKind can help you use data science to improve your work. The Data evolution report includes the findings from a study of hundreds of charities and social enterprises on data maturity.
Our expert consultants run regular training sessions on impact measurement, understanding and presenting your data and more. Check the events section of our website for the latest dates:
Tailored consultancy services
NPC offers support covering the tools and approaches in this document, including:
- Developing a measurement framework and identifying measurement priorities
- Designing and using data collection tools
- Internal capacity analysis
- Data analysis to capture impact
- Using data to improve service delivery
- Strategic decision making structures
If you would like to talk to us about using data, or want to know more about how we can help, get in touch via info@thinkNPC.org or 020 7620 4850.
Did you find this paper useful? Spread the word on Twitter @NPCthinks.
- Hoare, G., Murray, P. and Shea, J. (2017) Charities taking charge: Transforming to face a changing world. New Philanthropy Capital. ↑
- Inspiring Impact (2013) The code of good impact practice. ↑
- www.behind-closed-doors.org.uk ↑
- www.streetleague.co.uk ↑
- Bellinger, G., Castro, D. and Mills, A. on the Systems Thinking website: www.systems-thinking.org/dikw/dikw.htm ↑
- Noble, J. and Thorne, M. (2016) Stories and numbers: Collecting the right impact data. New Philanthropy Capital. ↑
- www.intouniversity.org ↑
- www.oxfordshiremind.org.uk ↑
- http://content.digital.nhs.uk/qof-online ↑
- For more on the definition of impact management, see Rotheroe, A. ‘Impact management: New kid on the block’, on NPC blog 16 June 2017. ↑
- Gyateng, T. (2015) Protecting your beneficiaries, protecting your organisation: Ten considerations for charities on safe use of personal data. New Philanthropy Capital. ↑
- www.positivelyuk.org ↑
- www.actionforchildren.org.uk ↑
- www.inspiringimpact.org/listings ↑
- www.place2be.org.uk ↑
- See www.sdqinfo.com ↑
- See Noble, J. ‘Why charities should collect less impact data’, on NPC blog, 14 August 2015. ↑
- For a guide on using evidence from elsewhere, see Knowledge Translation Network (2017) Evidence from elsewhere: Gathering, analysing and using other people’s evidence. ↑
- For a short practical guide to analysing data, see Hoare, G. and Noble, J. (2016) How to make your data more meaningful. New Philanthropy Capital. ↑
- www.datakind.org ↑
- www.savethechildren.org.uk ↑