Choosing a data management system for your organisation: A frontline perspective

  • Date of release: 2018
  • Type: Database & case management systems
  • Sectors:
    Arts and culture, Children and families, Citizenship and communities, Community development, Crime and public safety, Disability, Domestic violence, Education and learning, Employment and training, Environment, Housing and homelessness, Infrastructure support, International development, Mental health, Older people, Physical health, Sport, Substance use and addiction, Volunteering, Young people
  • Difficulty: Very Easy
  • Cost: Free
  • Developer/Author: Tim Crabbe, Substance

Data is increasingly important and deciding how to collect and manage it can be difficult. Organisations in the charity sector have diverse requirements, and there are many things you need consider for your system will be suitable for your organisation and those you work with.

This paper covers insights from the following areas:

  • Highlight practical challenges encountered by others
  • Help others navigate the process and avoid common pitfalls
  • Ensure they consider all the options.

Choosing a data management system for your organisation:

A Frontline perspective

Tim Crabbe


Data is increasingly important and deciding how to collect and manage it can be difficult. Organisations in the charity sector have diverse requirements, and there are many things you need consider for your system to be suitable for your organisation and those you work with.

Here, we want to build on existing guidance from Inspiring Impact and others about planning a good database, choosing the right supplier and choosing an electronic case management system.

This paper offers honest insights from organisations to:

  • Highlight practical challenges encountered by others.
  • Help others navigate the process and avoid common pitfalls.
  • Ensure they consider all the options.

Our approach

For this paper we decided to ‘ask the audience’ and gather experiences of how different organisations from across the sector had approached the challenge of reviewing and improving their data collection processes and associated system selections. The paper shares examples from these organisations, as well as tips for how to get started.

We spoke with a national funder, an impact assessment support network, a national survivor helpline and a local children’s charity about:

  • Their motivations for looking at their data management processes and systems.
  • Who they involved in the exercise.
  • Decision making and the process of implementation.
  • Outcomes from the process.
  • Key learning points to emerge.

Getting started

‘What are you hoping to achieve? Getting the purpose [of the system] in place is key.’

The decision to implement a new data management system is rarely taken lightly. It invariably involves a degree of organisational disruption and expense, focused around the consultation process, handovers, staff training, system procurement and indeed the possibility of resistance and sometimes failure. So, what was it that tipped the balance for those organisations we spoke with? Unsurprisingly, it was something of a mixed bag.

A case management system can streamline processes

For the funder it was a blend of factors relating to their existing usage of a range of incompatible spreadsheets and the arrival of a new member of staff who suggested the implementation of a dedicated system:

‘We were using a series of spreadsheets which were doing a fair job, they weren’t falling over, but we were sometimes struggling to get one version of the truth and they didn’t speak to each other at all.’

It was far from plain sailing from there though, as there was no obvious solution. CMS for funders are more often focused on managing applications for funding but this wasn’t what they were looking for. They wanted to think more about better management of the projects they had funded:

‘I found it quite difficult. We are a funder and what I did first was phone an old colleague from [another charity], to see what they were using… they used a specific thing for funders. I can’t remember the name, but it seemed quite limited and they weren’t blown away by it.’

She felt she was getting led down different paths when speaking with different colleagues and maybe asking too much of one system:

‘We explored different options and found it difficult to find out what the best thing would be. It has taken a year from recognising that things needed to change to [the process] being complete.’

A case management system can support integration in a merger

For the support network, which was mainly concerned with the management of membership data, the driver was less around operational requirements and more about the merger of two organisations and the need for the national network to adopt the system employed across the international operations of the new entity.

A case management system can help with regulation compliance

For the national helpline provider, whilst there was a certain degree of dissatisfaction with their existing arrangements, ultimately, implementation of the change was externally driven. They had won a contract with the NHS and therefore had to become IAPT (Improving Access to Psychological Therapies) compliant and decided to look for a system that had already been approved:

‘It was so much faffing [for the other system to be IAPT compliant.’

A case management system can support staff to work effectively

By contrast, at the children’s charity, the driver was internal. They were using an Access database which wasn’t fit for purpose and led to problems with the sharing of data when staff were out of the office and needed to access and update records on the fly.

Key Learning Points

‘What are you hoping to achieve? Getting the purpose [of the system change] in place is key.’

Take your time and read about the subject. These resources may support your discussions:

NPC’s guide to choosing an electronic case management system.

Power Objects’ guide to defining the scope of a dynamic CRM project.

Defining Requirements

‘Manage expectations and don’t run before you can walk.’

Once committed to the process of data management system change, a process of system requirement definition needs to commence which can be lengthy and complex, particularly where there are multiple stakeholders. Even before getting into the detail, NPC’s guide to case management system selection highlights a number of high level considerations which are relevant to wider data management system selection, including:

  • Levels—or extent—of use across the organisation.
  • The adaptability or flexibility of the system.
  • Support arrangements needed in terms of training and guidance.
  • Ongoing system development to ensure sustainability of the solution.
  • Integration options with other systems in use across the organisation.
  • Security and compliance with GDPR.
  • Whether the system will work with existing hardware.

Beyond these starting points, our consultation uncovered a series of wide-ranging requirements that were specific to the individual organisations. This itself points to the need to involve users and key stakeholders as far as possible.

A case management system is a development, not a solution

Part of the issue here is the tendency for organisations embarking on an exercise of this type to see it as a solution that will ‘do everything’ or at least capture everything.

This was the case for our funder, although they were also alert to the dangers of this desire and the difficulty of maintaining focus as well as facing the reality that there might not always be a one stop solution. As the Head of Legacy and Impact revealed:

‘We wanted to have lots of information at our fingertips. If [the system] was set up in the right way, we would be able to have data at the press of a button… rather than… do a lot of stuff manually.’

‘Trying to work out what was in scope… to be able to understand what was possible, I found that hard.’

‘Some things initially ended up being out of scope. We wanted dashboard management, so all the information could be refreshed easily. For example: A split of projects across portfolios. How much money was going out, total spend. Quick stuff that it can report on.

We wanted it to do a little bit more to answer impact questions, so organisations could feed in outcome measures but in the end [that] felt too complicated [and] might be something we do later. Think it’s a bit clunky on [the selected system] … Maybe [we were] looking for one system to do too much.’

Choosing a case management system is a team process

That is where consultation across the organisation proved vital in revealing the different requirements associated with different roles, including headline dashboard reporting of spend and progress for the Board and executive leadership team alongside more detail at the manager level:

‘When did our grant manger last speak with projects? Which projects have been visited or are due a visit or chat? RAG ratings for projects. Which projects have red comments on at the moment? Keeping track on day to day progress to be used in 121 meetings.’

Start simple and adapt the system as you learn

For the support network, the regular user group was more defined and on a day to day basis consists of the internal team rather than members themselves. The team produce monthly headline reports and quarterly board reports. Again, there were varied functional requirements which ultimately couldn’t be satisfied by a single system. The main driver was to have a well-functioning CRM system to help manage the membership. Ideally the system would manage wider impact data relating to their own delivery work linked to member records as well to allow the team to create their own reports:

‘We would love to have a CRM or data management system which would be able to integrate the two.’

The organisation itself works with software solution providers from across the social impact sector and provide accreditation services to these clients and might look to integrate the systematisation of this offer in the future but recognise that this is ‘quite a big undertaking’ particularly given the variety and complexity of the systems they review which they feel would benefit from being ‘stripped back and made simple.’ That cry for simplicity resonates with the Chief Executive of the national helpline who was clear about his needs, whilst recognising that these might change, and the system would need to be able to adapt with these changes.

‘I was putting all the data into the system and wanted to extract it in a reporting way that I could understand and that would give me complete insight into what was actually going on.’

‘It had to be able to be adaptable. [I] didn’t want a system that if I needed something changing and it was really simple, that I would have to wait 16 weeks to get an IT person to make the changes. I wanted to be able to do bits and bobs myself. I also wanted a system that we could adapt to our needs.’

Prioritise your requirements

The children’s charity also emphasised the importance of a progressive and indeed pragmatic approach and not trying to systematise things before you are ready. On the one hand, working as they do with children and young people, there was an immediate need to have basic personal records focused on who they are and what they access in terms of services provided. On the other, there is also a desire to have systems to help them conduct impact measurement and wider monitoring and evaluation but a recognition that other steps need to be taken first.

The organisation is currently developing a theory of change and associated monitoring and evaluation plan which has been ongoing for the past year with the intention of it being ready for the 2018 summer holiday programmes. Whilst their contact database has been built on an existing widely used CRM platform which can be used for wider evaluation purposes, the team do not feel ready to have this element built into the system yet and see this as something for the future:

‘We spoke to other charities who use it specifically for impact measurement. The consultant we work with suggested other charities use it in that way. We know there is scope there.’

A case management system needs to be proportionate

As such their immediate decision was driven by the solution’s ability to handle their routine contact management requirements and, critically, cost. After consulting with staff, they felt confident that the standard configuration would deliver most of what they wanted and had a lot of features they liked, including:

  • Ease of use
  • Reporting potential
  • Configurability

Critically though, for an organisation of their size and limited resource, the software itself was free for non-profits:

‘The software is free for up to ten licences. This is sufficient at the moment.’

There were costs for implementation, using an external consultant, and this may well have influenced the decision to be pragmatic about the extent of the functionality required and to develop in a piecemeal fashion as readiness and resources allow.

Key Learning Points

Security of the data and data protection is key and really important, especially with GDPR:

All guidance assumes that an organisation is meeting all legal requirements. A case management system can help with compliance. There are lots of GDPR resources out there including, guidance from the Information Commissioner’s Office, and there lots of resources on NCVO Knowhow Non-profit.

Be able to extract whatever data [you] put on the system—extract and cross reference in any way despite what order the data was entered in:

Discussing the organisations requirements for inputting, storing and using data can help ensure that the system meets your needs and the providers understand your requirements.

Make sure it has something that every member of staff can see advantages in, so each job role could have quick wins. Make sure there is a real thing for each member of staff [before committing].

Think about the functionality and that it can integrate to other systems used in the organisation. Don’t double up information elsewhere:

Reviewing your current data collection methods and systems can help you understand your system requirements and integration requirements. Complete this exercise to give you an overview of the data already being collected within your organisation and how it is currently being used.

Manage expectations and don’t run before you can walk:

Involving staff and key stakeholders in discussions about the system requirements and prioritisation of requirements can help manage expectations.

Decision making and implementation

‘Get staff to buy in. Most staff are now keen to get it in place and get it up and running. This will make a big different to how it is used and how it is used properly.’

Whilst it is unlikely that any system will ever satisfy everyone’s needs, the consultation process should help organisations prioritise their needs.

At the funders, it was internal staff who came up with the specification following extensive discussion and brainstorming which led to buy in from the Chief Executive, the identification of a preferred solution and the budget sign off. Whilst the budget was below the threshold for a full tender process, given the preference for a database configuration built on an existing CRM platform, there was still a need to identify the right consultant:

‘That was torturous. We ended up going with the first person out of the three quotes. Lots didn’t get back to us, one went bust! Was quite surprised by that.’

Once appointed, in many ways this merely marked the start of a new phase in the process as the consultant went through a detailed scoping exercise focused on how to structure the data and how to engage staff with the system:

‘We had a couple of long sessions with him. Our Grant Manager spent a lot of time with him.

Training was provided towards the end of the process for all 10 staff members. It was a whole afternoon and then follow up notes and activities. We had internal follow up training for any issues and created a channel to put in concerns or questions a month after the implementation.’

Implementation of a new system takes time and support

For the support network, whilst the decision was driven by the merger and therefore a need to use the other organisation’s system, implementation required staff training and shared responsibility across the team. The software also needed to integrate to other systems across the organisation given that they use a different system for communications, another for surveys as well as the organisation’s own website which includes a payment area. The new CRM platform does integrate with some of these but there is ongoing work to make the experience of the team and members better, post implementation.

At the national helpline, given that the NHS IAPT requirements had been such a driver of the process, the team had to keep checking their systems against the IATP standards:

‘They (IAPT) gave us instructions about what we needed to do and ideas on what would be useful and we kept coming back to [the chosen system] and how adaptable [it] looked like it could be.’

Involve all staff in the implementation process

Once the decision was made the process of implementation has been led and cascaded by the Chief Executive and Head of Operations with the assistance of the system provider’s support team:

‘[The consultant] was absolutely brilliant. He worked really closely with us. I added some development money so [the system provider] could get the system ready for us. [We] never necessarily had any training, more sitting down and talking to us [before we introduced it through internal training].’

Train staff and test the system

The process at the children’s charity went on for the best part of a year with the hope that it could be implemented in July 2018 ahead of their busiest period. After consulting widely on the system needs and then consulting with other similar charities about their arrangements, a range of systems were reviewed with the final decision being made by the Office Manager and Chief Executive. Whilst not yet implemented, there is a recognition that staff training will be a major consideration in the roll out process once they are convinced that everything is working as they want it to. Cost considerations mean that they have not signed up for ongoing support though and so this will be accessed on an ad-hoc basis.

Key Learning Points

Consult with other charities who do a similar thing to you. Tap into networks:

There are lots of networks you can tap into for advice and suggestions. You could post on the Inspiring Impact LinkedIn Group, attend a Charity Meetup, attend an Inspiring Impact peer learning session or contact others through a membership body like the Small Charities Coalition.

Have a clear idea of what you want prior to implementation. What data and how you want to use it:

Reviewing your current data can help you decide your requirements for a case management system.

Get staff to buy in. Most staff are now keen to get it in place and get it up and running. This will make a big different to how it is used and how it is used properly:

Involving staff at every stage will help you get staff buy-in. This could involve consultations of the system requirements and prioritisation of needs, staff testing and training on the system and internal communications throughout.

Find the right implementer. Dedicate a member of staff who can be the main liaison. Don’t underestimate the amount of time it takes:

This is not always possible, especially for smaller organisations, but having consistent points of contact, where possible it can save time and resource.

Make sure everyone knows what they are doing and encourage continued use:

Regular staff training checks, top tips and manuals can be helpful for ensuring continued use. A regular review of your systems ensure issues are highlighted and improvements can be made.


‘The process never ends’

The decision to change systems can improve working practices and the outcomes achieved by beneficiaries. Amongst those we spoke to, for one, the implementation phase was only just getting under way and so it was too early to comment on the result; for another the decision to change system had never been theirs in the first place. For the other two, there were promising signs and genuine enthusiasm, if not for the process, then for the outcome.

The funder was clear that the project had met their original aims, even if those had been scaled back, partly because of the learning that came from the process:

‘We’ve had it since January and it’s been really great… but we narrowed the scope a little bit as we worked out what [it] could or couldn’t do… people are starting to see nice things like the charts and how it can become more integrated to people’s working day and be part of the fabric of how we do things. Would like to see how we can use it further with the outcomes, and tidy things up either with [further configuration of the system] or integrating [with other] systems. Just getting used to this…The quality of information they are getting out is better but want to streamline the admin.

The national helpline was similarly positive but keener to highlight the ongoing interplay between use of the new system and changes in working practices:

‘The postcode finder was something [the system provider] was looking at and it became really important for us. It changed the way we do the activities. We’ve made the system IAPT compatible to make sure it fits with the national IAPT database. There’s been huge changes like the way that questionnaires are formatted and extracted.

There are still things that need to be changed. I need to see a date and time stamp. So, I need an activity and notes and I need that date and time stamped so that it can’t be amended once submitted… We still keep some paper records for this reason. Therefore, not using [the system] for all storage… but in the main [it’s meeting our needs].’

Key Learning Points

The process never ends:

Implementing a case management system isn’t the end. Your requirements will change regularly, and any system will need to adapt to this. Acknowledging this and using the Inspiring impact mantra of Assess > Plan > Do > Review could help you to review whether or not it’s working.

Security of the data and data protection is key…especially with GDPR.


The selection of a new data collection and management system is an important exercise that should not be taken on lightly.

Regardless of the preference for off-the-shelf, tailored or bespoke developments there are several factors to consider prior to, during and after the selection process itself.

The organisations we spoke to, whilst being motivated by a range of factors, coming from different sectors and operating at different scales, all emphasised the importance of widespread but appropriate stakeholder/user involvement to get the specification of requirements right.

It was also clear that expectations should be managed. No system will address every organisational need and might not address any need if users cannot see the benefit. It is probably better that the preferred solution, it is effective in a few key areas rather than trying to work across areas.

In many ways system selection should be the start of a process rather than the end. Implementation can take time and require significant resource in terms of staff time. It is also sometimes best conducted in an iterative fashion, building and extending from a strong platform using the Inspiring impact mantra of Assess > Plan > Do > Review.

Finally, if you plan to work with personal data remember to be responsible and adhere to the requirements of GDPR.

Key Takeaways:

  • It takes time so make sure you understand the purpose of why you need a case management system.
  • Take time to understand what data you have and what you need to do with the data.
  • The system won’t meet all your needs. You will need to prioritise your requirements and manage expectations.
  • It is a process, not a solution.
  • Ask questions and involve staff, users and key stakeholders throughout.
  • The system will keep evolving over time.


Resource download

Sign up to our newsletter

Stay up to date with the latest news from Inspiring Impact