How to turn data into action – lessons from US intelligence community and beyond

Data is not new in government. Agencies have been collecting data on any number of things as long as government has existed. It's been used to answer accountability questions like how much money it is spending and how many people are receiving social services.

What has changed over the last decade (probably two), is that we started to see across government a realisation that data could be used as core part of its mission, services and operations. Far more than just a reporting and metrics tool, it can actually drive the delivery and impact of services, as well as predict what is likely to happen in the future, based on what has happened in the past.

I'm lucky enough to have witnessed - and participated in - this evolution first-hand. In my 16 years working with the National Security Agency and across the intelligence community I was focused on boosting our capacity to turn data into something actionable.

After running several projects, leading data organisations, and working on several operations floors, around 2013 I decided to explore the policy side of government decision-making. Having spent much of my career on the tech side, I wanted to get involved in how decision-makers are making the policy decisions and how government it trickles down through Agencies to civil servants.  So I spent a year working in the office of Senator Cory Booker as a Brookings Fellow and legislative aide on cybersecurity and governmental affairs.

It was in this role that I met senior members of the Obama administration, including DJ Patil, the US chief data scientist, and Megan Smith, the US chief technology officer (CTO). I never considered working for the White House, but after several discussions, it turned out my technical, government, and policy experience would support the Administration's focus on building capacity for maximising the value of data across the federal government.

Capacity-building as a nation

As senior policy advisor to the CTO, my priorities for my tenure were improving individual access to workforce data for informing training and employment decisions as well as building out a community of practice across the federal government to build capacity through collective development of best practices.

Whenever I look to ramp up an initiative, it starts with a conversation primarily with the people who are most affected by the change. For example, before starting the Data Cabinet, we brought together 15-20 chief data officers to have a conversation about the issues they were facing in terms of using data to drive decision making. It was these conversations which helped focus and drive the work of what became approximately 200 data leaders across 40 agencies developing and leveraging three key best practice tools for creating the Data Cabinet, which was tasked with leveraging data as a strategic asset.

The Workforce Data Initiative, meanwhile, was a collaboration between the Department of Labour, the University of Chicago and several other entities. Its aim is to open up workforce data from the public and private sectors, as well as academia and create a taxonomy that allows for tools to be developed which better inform job-seekers economic opportunities.

The premise is that in order for an individual to make informed decisions about skills, careers and future opportunities, they need information which spans across the sectors. So that's what we did, we brought data from across the sectors together, created an Open Skills Repository and exposed it in a way that can spur innovative solutions to help inform individuals, researchers, and policymakers in helping them get the information and tools they need to keep the workforce ecosystem moving forward.

Power up to open up

What made the Obama Administration's approach to data different was the decision not only to “open it up” but also to turn it into something useful. This is where its power comes from. And to do so in a collaborative fashion with purposeful diversity ensures the responsible democratisation of data necessary to maximise its value to people. One example of many is The Opportunity Project, which brought government together with industry to turn open data tools that help communities.  Tools such as the Opportunity Score, developed by RedFin, leverages commerce data to show which jobs are available within 30 minutes without a vehicle.  

This type of work shows how technology and data could and should be used to not only increase efficiency and effectiveness, but at the same time transform the way we approach and examine society, and change the way we solve problems.

What's next?

The ability of data to transform the way government can deliver social services was the main driver for my decision to leave government - not because I didn't love being a public servant - but because what I wanted to do next was focus on how to build capacity at the state and local level and build capacity within the data science community to help do more with data.

I've been focusing on two areas. I joined a company called BrightHive as chief operating officer. BrightHive delivers multi-party data sharing services to enable state and local governments to run outcomes-focused social service programmes. As a data-technology start-up, we build multi-sector ecosystems that legally and securely support the use of data across policy and infrastructure silos.

For example, in today's modern workforce, employers worry about finding and retaining the talent they need to remain competitive, and workers stress about how to obtain and hone the skills employers want.  How do you help employers to signal their need and help jobseekers to understand the right training opportunities to improve economic opportunities? You will need employment data, wage data, training outcomes data, all of which are housed in different systems with different rules. So you need to bring all of these siloed data sets together in a way that can solve some of these complex problems. The good news is that this is very much achievable and BrightHive is working with states such as Colorado to make it happen.

None of this can have a sustainable impact without also building capacity through community engagement of the data science community. There is so much data being used and collected on a daily basis but where are the frameworks and where are the tools that help data science practitioners know how to use it responsibly? Where are the strategies that help organisations fit into their business models?  

That is why I lead an initiative with Bloomberg and Data For Democracy to develop Community-driven Principles for Ethical Data Sharing (CPEDS).  CPEDS focuses on bringing anyone that shares data together to identify and define the behaviors of responsible data sharing. Those behaviours can then be used to drive development and awareness of tools and guidance.   

Coming up with the answers will not happen overnight but there's no doubt that progress is being made - just like government's use of data and technology. 

FURTHER READING

• Government by numbers: doing data differently. How can government extract the most value from its data? BCG's Evan Stubbs offers up some ideas
• PODCAST: In conversation with… New Zealand digital leader Maria Robertson
• Briefing Bulletin: Going digital - how governments can use technology to transform lives around the world
• Power to the people. Few countries have embraced the digital era as successfully as New Zealand. BCG's Miguel Carrasco talks to one of the New Zealand government's key digital transformation leaders, Richard Foy, about how they've done it.
• Ever evolving, e-Estonia. As Estonia's prime minster Taavi Roivas, played a pivotal role in the country's technological advancement. We hear how he did it
• A new New Zealand: public services reimagined. New Zealand already has one of the world's most advanced systems of digital government - but it's not pressing pause. We hear from Maria Robertson, deputy chief executive, service delivery and operations, about tackling tomorrow's challenges 
• Unlocking the digital door for developing countries. Although universal access to the internet remains some way off, Hans Kuipers explains what steps can be taken to bridge the enduring digital divide