Skip to content
Article Article November 30th, 2016

Data detectives: how AI can solve challenges large and small

Article highlights

We have only begun to scratch the surface of data science and machine learning

Share article

Governments could be saving billions of pounds and improving services through AI's use

Share article

Government silos make it very hard for datasets to work together

Share article

Partnering for Learning

We put our vision for government into practice through learning partner projects that align with our values and help reimagine government so that it works for everyone.

Partner with us

Artificial intelligence. Two words to strike a mix of emotions into audiences far and wide. Wonder? Fear? Optimism? Nervousness? Bewilderment? Maybe all of the above - and yet we have only begun to scratch the surface of artificial intelligence (AI).

It is extraordinary that the world's best Go player can be beaten by a machine, but that is a long way from emulating human creativity or wisdom. And yet machine learning - the process by which programs grow, change and learn when exposed to data - could already be having an impact. Governments today could be saving billions of pounds, releasing funds to the front line and improving services, if they could only make better use of machine learning and other forms of AI.

Flagging up our fellows

I work for ASI Data Science, a London-based firm which applies AI to solve business problems, but I served previously in government, including a stint as one of the founding directors of the Government Digital Service.

About 10% of the UK's science, technology, engineering and maths PhDs apply to us for an ASI Fellowship. Three times a year, we take 20 of the best-performing students and place them on eight-week projects with top companies and, increasingly, with central and local government. Through this programme, we get a sense of the problems that companies and governments are trying to solve - and how AI can help.

Our most recent fellowship covered a wide range of subjects, one of which was fraud detection. Local authorities and police services spend an inordinate amount of time pursuing fraudsters. Rogue landlords, for example, cause misery to their tenants - often across dozens of properties and years of criminal behaviour - and each year they make £4.2 billion illegally.

Catching them requires a painful, manual process of tracking names across different registries and databases. It takes dozens of hours. In just a few weeks, one of our fellows automated this process and made it 600 times faster. The program is so effective that the local authority client is applying it across all of its fraud detection. This might seem simple, even boring. But if simple moves can save billions in time and effort, it is a signal of the scale of public money that could be saved with the intelligent application of AI.

We also looked at predictive recruitment. In Michael Lewis's brilliant book Moneyball, immortalised by Stephen Spielberg and Brad Pitt, the Oakland Athletics' basketball manager Billy Beane used analytics and data instead of human judgment to recruit his team. The Athletics became competitive with the New York Yankees, despite spending far less on player fees. Many of us who have hired and managed large teams know how difficult recruitment is. We are all of us subject to natural human biases and reactions. In a recent project, one fellow devised a machine learning recruitment engine for a British sports team, and the program performed better than top recruiters.

Education, too, came under our microscope. The past is littered with attempts to digitise education, from warnings that the radio would replace the teacher to online schools and Skinner's behavioural boxes. To date, none has given us the kind of results we'd hoped for. But that may be about to change.

Machine learning, coupled with our growing cognitive science base, allows us to have a far more sophisticated and differentiated view of the human brain and its capacity to learn. How quickly you learn - and forget - and how best to reinforce material was the subject of another fellowship. If you could automate this on a personal level, it would free up the educator to concentrate on feedback and human interaction. Personalising the balance of revision as against teaching new material - this was the issue for the ASI fellow, who was working on a company's online language-learning application.

These are just three of almost sixty examples across the year. Each took only eight weeks to complete, which tells you something about the maturity of AI and its potential to improve performance across the public sector is, and how little it is currently being deployed in governments across the world. The UK is among the more sophisticated of these governments, and yet it is still only noticing the barest fraction of possibilities.

What next?

The discussion of AI is all too often focused on job losses and social dislocation. These are real concerns that require a response, but we ignore the fact that better jobs might replace them. Much of the work that AI can replicate is laborious and repetitive - it's no-one's dream job to compile endless lists, for example. Calculators and software have changed the tasks of accountants, but they haven't removed the need for them. Instead, they have enabled accountants to focus on the more interesting and complex aspects of their work, which require human interaction, creativity and expert judgment.

Unfortunately, making the most of AI depends on some hard slog. The biggest problem for AI practitioners is the poor quality of the data, and government siloes make it very hard for datasets to work together. For example, we don't check benefit records against the death register to avoid paying benefits to people who have died, because the two information sources are run by separate departments. We don't have one consistent dataset of companies in the UK, because HMRC and Companies House maintain their own separate lists.

And maximising the use of AI also requires new people in the public sector - ones with the skills to create or commission the right projects. We need to recruit and train differently (perhaps by using AI itself, as our sports recruitment project indicates!), but there are plenty of well-qualified graduates ready to do the work.

Our fellows may be inexperienced as data scientists, but they have first-class minds and learn quickly. We are proud of the fact that 100% of them go on to be hired by organisations who partner with us on the fellowship. They complement perfectly our more experienced consulting teams, who work alongside public and private sector clients to build capability and demonstrate business value.

If you would like to apply for an ASI Fellowship or become a partner organisation, do let me know. My email is



  • From the machines of today, to the artificial intelligence of tomorrow. IP Australia is pioneering the use of artificial intelligence in the Australian government. Its general manager of Business Futures, Rob Bollard, tells us how services - and citizens - benefit from its deployment
  • March of the machines: how governments can benefit from AI. The rapid advances of artificial intelligence are poised to reshape our world, says Philip Evans. He explains why governments should embrace, and not retreat from, this upcoming revolution
  • Changing times: Why it's time to plan, not panic for AI. Although artificial intelligence can lead to many positive results, policymakers should be vigilant about the implications and direction of this fast moving technology, says Sebastian Farquhar. He explains why clear goals and objectives are paramount
  • How AI could improve access to justice. With the law never having been less accessible, Joel Tito explains how the smart deployment of artificial intelligence can help
  • Transforming technology, transforming government. Rare is the policymaker who doesn't see digital as a doorway for strengthening public services. But as Miguel Carrascoexplains, the pace of the digital evolution means there is always more to do
  • Power to the people. Few countries have embraced the digital era as successfully as New Zealand. We talk to one of its government's key digital transformation leaders, Richard Foy,about how they've done it.
  • Computer says yes. Governments are increasingly reliant on digital technology to deliver public services - and Australia's myGov service is a potential game-changer, says Gary Sterrenberg
  • Digital dawn. It may not be obvious, but US policymakers have had an important role to play in the creation of today's digital era. But sometimes it involves stepping back rather than stepping up, suggests David Dean

Share this article: