If you worked in the public sector and someone offered you a tool which could “prioritize risk more strategically, deliver services more efficiently, enforce laws more effectively and increase transparency”, you’d take it, right?
According to the Mayor’s Office of Data Analytics (New York), there is such a tool. It’s known as data analytics (defined by Nesta as the discovery, interpretation, and communication of meaningful patterns in data) and it’s being embraced by public sector organisations around the world.
However, not everyone agrees that data analytics is a force for good. In fact, the use of data analytics in the public sector appears to largely divide people into two camps: evangelists and sceptics.
Evangelists unapologetically promote data analytics as making government work faster and smarter, while sceptics suggest that the inherent limitations and risks associated with data analytics undermine any significant gains.
I’ve been struggling to understand why this is, because I sit somewhere in the middle. I am a data analytics moderate. Reflecting on this following a recent London Office of Technology and Innovation (LOTI) workshop, I think that a key driver of the polarisation of views is the fact that data analytics as a field is often not clear enough about its strengths and limitations.
What can data analytics do?
At the recent LOTI workshop, the analytics team in New Orleans’ Office of Performance and Accountability shared six public sector problem-types where data analytics works particularly well:
The following example of the positive impact of data analytics was shared at the workshop:
In New Orleans, a fire broke out in a family home, killing all of the young family living inside. Despite the fact that the Fire Department had a programme in place to distribute free smoke alarms, the investigation revealed that the house didn’t have a working alarm — a fact which almost certainly contributed to the tragic consequences of the fire.
This tragedy deeply affected the city of New Orleans and brought to light deficiencies in the Fire Department’s programme — despite having free smoke alarms to distribute, the Department was struggling to find the small number of vulnerable families that needed them. The tragic loss of a young family prompted them to try something different.
Using public data, the analytics team developed a risk model to support the Fire Department to target the neighbourhoods least likely to have smoke alarms. As a result of this project, homes in need of smoke alarms were identified at twice the rate as they were when the approach was random.
The power of adopting a data-informed approach to distributing free smoke alarms was recently validated when there was a fire at one of the homes which had received a free alarm, and all 11 people living in that house escaped safely.
What can’t data analytics do?
While the preceding story shows that data analytics is very good at certain things, it also has significant limitations.
In focusing on actionable insights, data analytics practitioners are deliberately — and for good reason — looking for narrow, discrete and relatively technical problems to which a solution can be applied. However, public sector challenges rarely, if ever, fall into this category.
Discrete problems, which are amenable to a data analytics solution, are, in reality carved out of bigger, more complex, messier ones.
For example, even a challenge as seemingly technical as optimisation of traffic flow has deeper, more tangly, roots. While a clear cause of congestion might be poor traffic-light coordination (an issue very amenable to a data analytics solution), there are also more complex contributory factors and questions to explore and address. For example, could we be encouraging more people to ride bikes? Are people feeling unsafe, therefore choosing to drive rather than walk?
While data analytics can be very effective at addressing discrete, technical issues, it offers little in terms of understanding or helping to address the broader and more complex challenges in which the discrete challenge is situated.
Drawing again on the example used above from New Orleans — data analytics offers a solution to the problem at hand: “let’s identify which houses don’t have smoke alarms and give them a free one”. But it doesn’t ask the deeper and arguably more important questions: why do some neighbourhoods in this city have disproportionately low levels of smoke alarms, and what changes need to be made to shift patterns of need, vulnerability and intergenerational disadvantage?
Data analytics and the three horizons
A useful way of framing and thinking about the strengths and limitations of data analytics is Bill Sharpe’s Three Horizons Framework, explained nicely here by Kate Raworth.
Very briefly, the Three Horizons framework suggests that, in the context of thinking about the future, there are three possible paths:
- Horizon 1 (H1): Business as usual (i.e. no change)
- Horizon 2 (H2): Disruption/modifications to existing processes
- Horizon 3 (H3): A radical rethinking and reshaping of existing processes and paradigms.
Many of the most commonly used and well-known innovation tools are H2 innovations; in other words, innovations such as new technologies or services, which disrupt business as usual. However, it is important to recognise that there are two forms of H2 innovations:
- those that disrupt the status quo, but then result in a reversion to business as usual (i.e. back to H1) — known as “H2 minus innovation”; and
- those that disrupt the status quo in a way which catalyses fundamental systems shifts — known as “H2 plus innovation”.
Data analytics as second horizon innovation
Data analytics clearly operates at the second horizon — disrupting and evolving existing services and systems. This is largely because data analytics can’t penetrate issues deeply enough to offer more radical shifts or visions for the future.
Big data generates broad insights, but not deep insights. I’ve written about the limitations of insights generated by big data previously — although from a different angle — in this article.
Nevertheless, while data analytics alone cannot define the deeper service and systemic changes needed, it can play an important role in informing third horizons thinking.
For example, say a Local Council decided to use data analytics to optimise the allocation of early help workers across wards. The insights generated from the analytics could be used for two purposes:
- to allocate early help workers more efficiently; and
- to highlight geographic areas in need of more early help workers, and invite a different kind of project, drawing on different disciplines and skills, to explore why there is more need in those areas, and what might be done differently at a more systemic level to reduce this need.
This is an example data analytics operating as a “H2 plus” innovation — catalysing systemic change. However, this is not always how data analytics projects operate. Too often, data analytics projects seem to apply a “H2 minus” model — disrupting the status quo to some degree, but then allowing business as usual to continue.
In order for data analytics to support not just evolution, but revolution of public services, humility is needed. There must be an acknowledgement that to truly drive change, data analytics projects should not be seen as an end in themselves, but rather, as a key player in a much larger project.
What would this look like in practice?
I think it would mean tweaking the current methodology adopted by many data analytics projects.
Current methodologies begin by asking, “can I answer this question with data?” This is great, because it creates a real clarity and offers the opportunity to generate actionable insights and real, tangible change.
However, the challenge with this framing is that it appears to be seeking a binary answer — “yes” or “no”. In fact, what I’m suggesting is that there should probably be a third acceptable answer which is, “in part”.
For example, while data analytics might help to identify at-risk children, it offers very little in helping to address the underlying causes. The solution offered by a data analytics project might be something like a dashboard to help social workers make faster, more accurate decisions. However, while this dashboard might mean that children are seen more quickly by social workers, it does nothing to address the actual root causes which place the child at risk in the first place.
As such, we need to start framing data analytics projects differently and acknowledging that data analytics can only ever play a limited role in tackling complex social challenges. This should trigger a process of thinking about who else needs to get involved in the project to give it a better chance of solving the challenge more holistically. Do we need ethnographers? Systems change practitioners? People with lived experience? Service designers?
I am not suggesting that this will be easy. Bringing together different disciplines to work together is always challenging. However, ultimately, I believe that framing data analytics as a relatively small part of a much bigger process is imperative if data is going to play a key role in helping to meaningfully tackle social challenges.
Why we should all be data analytics moderates
Data analytics clearly offers valuable opportunities to disrupt and optimise existing public services, and there is certainly a need for this. However, too often, the conversation stops there, as though the job is done when we have worked out how to hand out smoke alarms in a more targeted way or identify at-risk children more accurately.
But stopping there feels wrong. Rather, these discrete projects should be matched with broader, slower, more ambitious projects, which draw on different fields and disciplines to address the root causes of the problems in a way that data analytics cannot.
Data analytics can never be the revolution we need in the public sector. However, it plays a role in evolving existing systems and, if appropriately framed as part of a larger piece of work, can generate important insights to support truly transformational change.
