Small organisations guidance - Plan

Plan

Careful, realistic planning lays the foundation for good impact practice. Accordingly, ‘Plan’ is the longest section of Measuring Up! , and the questions this section asks you to consider are relevant to all organisations, whatever their size, shape or ambition in terms of being able to plan, evidence, understand, communicate and learn from their impact.

Planning requires you to reflect on how and why you think your work can make a difference. It is also an excellent first step for involving others in thinking about your impact – everyone can get involved in discussing and defining your organisation’s purpose, desired outcomes and impact.

Your goals for impact measurement will also determine the depth and detail of information that you set out to collect, as well as your plans for collecting it.

All organisations, whether funded or not, need to collect information about their impact. This information can help you to be accountable to your funders or beneficiaries and to learn about how to improve your work. The standard of evidence required for organisations who want to influence policy or to explore the efficacy of a new type of programme or initiative will be much higher. We have explored this issue of proportionality throughout the guidance provided with this section.

1.1 We use evidence to better understand the need for our work

A needs assessment is a way of researching the need for the work you do. It will identify the extent and seriousness of existing problems, any services currently available and the needs that are not being met. Once you have clarified the need for your organisation, you can start defining how you plan to make a difference, laying the foundations for good impact practice.

If your organisation has been running for some time, chances are that you will have conducted a needs assessment in the past. If you are a new organisation, however, you will have to begin from scratch. To begin with, look for local data and any relevant national statistics and reports to get information on:

  • The area or region your organisation will operate in
  • The state of current local service provision
  • The type of person, organisation, animal or environmental issue you want to create change for, and where they are
  • The sort of problems they face, and possible solutions.

This criterion is fully met if:

You can describe the need for your organisation, and evidence that need using available information.

What next?

If you’ve met this criterion in full, you could improve your practice by:

  • Collecting your own information to use alongside existing data. This will add depth to your needs assessment and could make it more relevant to your specific context. You could gather this information by speaking to current service users, potential service users, community groups, and other organisations delivering similar work.

1.2 We can describe who or what will benefit from our work

Being able to describe your target group – the main group or groups in society that will benefit from your work– is an important step in defining your purpose as an organisation. This means that it’s also an important step in focusing on your impact.

Your target group could be identified by geographical area, age, gender, sexual orientation, ethnicity, or by health status or disability. Or you might be looking to benefit animals, other organisations or the environment.

You can use the information gathered through your needs assessment to help you to define your target group or groups.

This criterion is fully met if:

You can describe in detail the group or groups that will benefit from your work.

What next?

If you’ve met this criterion in full, you can improve your practice by:

  • Deepening your understanding of who your target group is. You may want to find out more about the groups who will benefit such as what socio-economic group they belong to and whether there are sub-groups within your target group (e.g. single parents, children excluded from school or homeless people).

1.3 We have a clear mission statement setting out our purpose and reflecting our values

Setting out the overall purpose of your organisation through a mission statement is a good starting point to help you to define the biggest, most broad-reaching change that you would like to see as a result of your work (your impact). This can include effects on people who are direct users of a project or organisation’s work, effects on those who are not direct users, or effects on a wider field such as government policy.

This criterion is fully met if:

Your organisation is guided by a document which clearly sets out your overall purpose as an organisation, and the core values that underpin your work.

What next?

If you’ve met this criterion in full, you can improve your practice by:

  • Making sure that your mission statement and core values are agreed on and recognised by the whole organisation. Involving people in putting your mission statement together, and making sure that the end document is recognisable to everyone within your organisation (including staff, trustees, and volunteers) will make it a more powerful guiding document.

1.4 We can describe the positive outcomes that we want to achieve for beneficiaries

Your outcomes are the changes, benefits, learning or other effects that happen as a result of the work you deliver. Your work as an organisation has the potential to produce any number of outcomes for different groups, both positive and negative, planned or unexpected.

Outcomes are not the same as outputs, which refer to the services and products provided by an organisation. Your outcomes describe the difference that your work makes, and not the work itself.

For example: A youth group runs sexual health workshops with local young people. The output is the workshop – the service that they deliver. The outcome is what changes for the young people as a result of participating – an increased understanding of how to protect themselves from sexually transmitted diseases.

The following table may help to build understanding of possible outcomes for example areas of change:

 

Health Reduced physical health symptoms

 

Behaviour Reduced offending behaviour

Reduced consumption of alcohol

 

Attitude and self-perception Increased motivation to find housing

Increased self-confidence

 

Knowledge and skills Increased knowledge of benefits

Increased job-search skills

 

Circumstance Service users achieve more qualifications

More service users have suitable accommodation

 

This criterion is fully met if:

You have clearly defined positive outcomes that you hope to achieve.

What next?

If you’ve met this criterion in full, you can improve your practice by:

  • Developing your outcomes in consultation with current beneficiaries. Consulting with potential beneficiaries, where possible, will help you to develop an in-depth picture of the sort of outcomes that could result from your work, and how beneficiaries might describe and experience these changes. This will help you to define outcomes that are relevant to your beneficiaries and realistic, given the work that you do.
  • Creating a Theory of Change. A Theory of Change sets out what outcomes (changes or benefits) you expect to achieve in the short, medium and long term, to contribute to your intended impact. It shows how you expect your outputs (products or services) to deliver those outcomes and what assumptions underlie that theory. For more about Theory of Change, please see the resources section.

1.5 We know what information to collect to show which goods and services have been delivered, and to whom

You will need to be able to measure the outputs of your work, to help you put your outcomes data in context. Your outputs are the products and services that you deliver. If you are running more than one project, it might be easier to begin by defining your outputs at a project level rather than across your entire organisation.

Firstly, you will need to be able to define what your outputs are. You can measure your outputs using indicators – well-defined, easily measurable information, which shows how well your organisation is performing. You will need to set indicators that tell you about your outputs (as well as your outcomes, see 1.6).

Output indicators are pieces of information that will help you to work out whether or not you delivered your work in the way you planned.

Output indicators are usually quantitative – that is, they collect numbers and statistics. For example, if you provided a helpline, your output indicators might be:

  • Number of calls received
  • Number of individual callers
  • Length of calls.

You would also want to collect some qualitative information as well – that is, more descriptive information. For example, the helpline could collect information about the types of issues that were raised by callers.

The following table may help to build understanding of the types of indicators that could be set for example outputs:

Outputs Output indicators
Social events Number of events held

Type of events

Number of people attending

Profile of people attending

This criterion is fully met if:

You have set output indicators for a prioritised number of outputs, that include information about the products and services that you delivered through your work, and the type of people who accessed them.

What next?

If you have met this criterion in full, you can improve your practice by:

  • Setting indicators that measure client satisfaction for all of your outputs. Collecting information about the level of client satisfaction will help you to understand how people experience the services that you provide, and how you might be able to improve the way you work. (Note that client satisfaction is an output indicator, rather than an outcome indicator, since it collects information on how people felt about the service they received, rather than what changed for them as a result.)

1.6 We know what information to collect to provide evidence of our outcomes

As well as collecting information on your outputs (1.5), you will need to be able to measure the outcomes of your work. Firstly, you will need to be able to define what your outcomes are (see 1.4). Then you prioritise which of these are most important (you will not be able to measure them all!)

As with outputs, you can then measure your outcomes using indicators. You will need to collect information that will help you to work out whether or not your work made the difference that you hoped to make.

Outcome indicators are the signs that the outcome has happened, or that progress is being made towards it. They can be quantitative (measuring the number of changes that happened, or the amount of change) and qualitative (describing people’s perceptions and experiences).

You are likely to find a number of possible indicators for each outcome. As with outcomes themselves, identify and use only the most relevant ones, to keep your evidence collection proportionate and streamlined.

To provide good evidence of change, you will usually need to select more than one indicator for each outcome. For example, if you wanted to measure the extent to which your networking event had resulted in the outcome of ‘increased partnership working’, you indicators might be:

  • Number of new collaborations or partnerships. This would give you information about how many new partnerships had occurred.

If you want information about the nature and depth of those new partnerships, a relevant indicator might be:

  • Level of information sharing

The following table may help to build understanding of the types of indicators that could be set for example outcomes:

Outcomes Outcome indicators
Reduced isolation Level of isolation (self-reported)

How often older people socialise

Size of social circle

 

Reduced stigma around mental health How comfortable older people would be living next door to someone with mental health problems

How comfortable older people would be leaving their grandchildren with someone who has had mental health problems in the past

 

Please see the resources section for links to further information on indicators.

This criterion is fully met if:

You have selected a number of priority outcomes and set at least one outcome indicator for each. Your outcome indicators can provide you with the information you need to assess whether and how your organisation has achieved the outcomes that it planned.

What next?

If you have met this criterion in full, you can improve your practice by:

  • Making sure that you have a good mix of qualitative and quantitative outcome indicators that include different people’s perspectives on the difference your work made

Selecting a range of qualitative and quantitative indicators will give you a good range of information about which outcomes were achieved, and how people perceived and experienced the change. Collecting different perspectives on change allows you to compare different people’s accounts of the difference your work makes, and will make your evidence for change more detailed and robust.

1.7 We are clear about the information that we need most urgently, both for reporting to funders and for internal learning

Before you begin collecting evidence, it is important to be clear about what information you need in order to be able to meet your reporting requirements externally to funders and commissioners, and internally to your trustees or board. This is important for all voluntary groups, irrespective of size. It will also help you to make sure that you have all the information that you need internally to learn and improve your service. Clarifying this at the planning stage will help you to avoid collecting information that you don’t need, or missing out something vital.

You will then be able to set priorities about the information you most need to collect – this is fundamentally important for building a plan for collecting evidence that matches your available resources. It may not be realistic to measure everything!  Focusing on measuring a prioritised number of things well, rather than trying to measure everything, is often a more straightforward and robust approach.

This criterion is fully met if:

You have defined which information you need to in order to be able to report to funders and commissioners, and to your trustees or board.

What next?

If you have met this criterion in full, you can improve your practice by:

  • Thinking ahead about the information you might need to make future bids for service and organisational development. Thinking strategically about the information you might need in the future to develop your work will help you to make informed decisions about the way you grow and fund your organisation.

1.8 We choose data collection tools that meet our information needs and suit our context

Once you have prioritised the outputs and outcomes that you want to collect information on, and set your indicators, the next planning stage is selecting data collection tools – for example, questionnaires, focus groups and interviews.

In order to collect good quality evidence, your data collection tool needs to be appropriate to your context. This involves thinking through five key considerations.

  1. What sort of information do I need?

If you want to collect basic quantitative data (numbers and statistics), you may find that a simple questionnaire suits your data collection needs. However, if you want to collect in-depth information about how people experienced different outcomes and what they thought about them, you will need to choose more ‘open’ data collection tools, such as interviews and focus groups.

  1. Which tools would work best in the context of my work?

Not all data collection tools can be applied equally successfully all different contexts. A questionnaire designed for older people might work well in context, but very poorly in a youth work setting if young people find it dull to complete or difficult to understand. Before choosing a data collection tool, you will need to think about your particular context, and how you can make data collection easy and interesting for the people you want to collect information from.

  1. Which tools will give me data that I can easily collect, store and analyse?

Think carefully about the skill level of the people who will be collecting the data and the time they have available. Your tool will need to work for your staff and volunteers, as well as for your beneficiaries. Thinking ahead to storage and analysis is also sensible.

  1. How much information do I need to collect?

You should be proportionate in the amount of information you collect using your tools. For a small organisation or project you might choose to use only one or two types of tool. Collecting information from a sample of your service users could also help you to be proportionate (see 2.1). In most cases, small organisations should only need to collect information on their output and outcome indicators; use these to form the basis of the content of your tools.

  1. Use existing tools and processes

If your existing tools and processes are working well, you may not need to change them or they may only need minor tweaking in order to meet your impact needs.

This criterion is fully met if:

You have selected data collection tools that can capture all of the information you need, which make data collection easy for your beneficiaries, and which suit the time and skill level of those people responsible for collecting and analysing the data.

What next?

If you have met this criterion in full, you could improve your practice by:

  • Adapting your data collection to take into account the evidence requirements of your stakeholders. You should consider what kind of information your staff, funders or other stakeholders require or even prefer. For example some funders prefer quantitative data, others like illustrative case studies; some prefer a combination of the two.

1.9 We have a plan which sets out how and when evidence will be collected, and by whom

Having decided on your priorities for collecting evidence and selected your tools, you are now ready to draw up your framework or plan. Your framework clarifies which outputs and outcomes are going to be measured, with what tool, when and by whom.

Essentially a road map for collecting evidence, your framework is a crucial tool for making sure that nothing important gets left out of your data collection plan, and that everyone understands their particular role in collecting information.

Example evaluation framework:

Outcomes Indicators Information collection tools Who and when
Improved job search skills Number of people with a CV

Ability to identify suitable jobs

Ability to complete application form

Appropriateness of presentation

Job log database

 

Case file

Self-assessment form

 

Staff observation

Case worker at assessment/ review sessions

 

This criterion is fully met if:

You have a framework which sets out: what information you want to capture on outputs and outcomes; how this data will be collected; when it will be collected; and who is responsible for collecting it.

What next?

If you have met this criterion in full, you can further improve your practice by:

  • Including a description of how the information will be used. Adding this information to your framework will provide an additional check to make sure that you aren’t collecting anything unnecessary, and that all of your priority outputs and outcomes are included.

Resources for this section

Using local information to provide evidence of the need for your work

  • Neighbourhood Statistics is an online government portal that allows you to search 2011 census data and other government data sets. Enter your postcode to find statistics on demographic profile, crime rate, education, health, housing, deprivation, lifestyles, work and environment.
    In Wales, you can access additional regional data on the StatsCymru website.
  • Scotland has a devolved body, The National Records of Scotland, which is the central location for government data.
  • Northern Ireland has the Northern Ireland Statistics and Research Agency which also holds government data and other social research.
  • The Joseph Rowntree Foundation has a repository for indicators of poverty including unemployment, housing benefit and educational attainment.

Charity Commission

Learn more about the difference between outputs, outcomes and impact

  • There is a jargonbuster defining relevant terms on the Inspiring Impact website.
  • New Philanthropy Capital’s Four Pillars Approach is a document detailing New Philanthropy Capital’s (NPC) four-pillar approach which provides practical guidance on developing an impact measurement framework.
  • Evaluation Support Scotland Guide 1.1: Clarifying your Aims, Outcomes and Activities is a guide from Evaluation Support Scotland (ESS) that will help voluntary organisations to work out what their outcomes are and how to write them. It clarifies the connection between what you do (your services and activities) and the impact you are trying to make (your outcomes). It introduces the Weaver’s Triangle (otherwise known as the CES Planning Triangle).

Theory of Change

Sample questionnaires and guidance for designing them

Output and outcome indicator examples

Developing a plan

 

Sign-up to the principles

We need your help to put principles into practice