Can Generative AI solve a 300-year-old government challenge?

Just over a year ago, the UK Government’s latest Levelling Up White Paper promised a new direction in its attempts to “improve productivity, boost economic growth, encourage innovation, create good jobs, enhance educational attainment and renovate the cultural fabric” of some of the more deprived areas in the country.

If the rhetoric sounded familiar, that’s because it was. The Institute for Government’s 2015 All Change report found that over the previous 40 years, 40 different programmes and organisations had been created to close the UK's regional divide. Add to that list freeports; growth deals; various competitive funds; and metro mayors, and by the time the White Paper was published that figure stood at 44 programmes in 47 years.

The challenge of public sector knowledge management

The paper itself acknowledged the extent of this policy churn, claiming that previous efforts “have tended to be short-term, lacked scale and coordination, and were hamstrung by a lack of data and effective oversight”. This lack of coordination is not limited to any one party’s policies, or any single policy area. Their short-lived nature speaks to more than just individual ministers wanting to make their own impact.

Of course, the UK’s regional divide is far too complex a challenge - and its causes far too numerous - to be attributed solely to a lack of coordinated policy. But the figure of 44 programmes in 47 years points to a wider breakdown of institutional memory in government. Policymakers are increasingly called upon to respond effectively to complex challenges far broader than any single department's remit. One of the principal arguments for a permanent civil service is to build an organisational memory that helps them do just that, an antidote to the churn of the political cadres that often last just a couple of years in any given role.

And yet, as successive reports have shown, governments worldwide continue to be plagued by poor institutional memory. At best, knowledge is stored in subfolders of a remote Google Drive link and, at worst, remembered only in the mind of a civil servant who left the organisation years ago.

There are a variety of reasons for this. "Chief among them is the high staff turnover in the Civil Service, by accident and by design. According to the Institute for Government, 13.6% of the Civil Service workforce either move between departments or leave the civil service entirely year-on-year, while as many as 45% of Senior Civil Servants can leave a department. Add to this the prevalence of contractors, regular restructuring of departments, and changing IT and data management systems, and it is easy to see why knowledge management in the public sector is such a problem.

This is not a new challenge. His Majesty’s Stationary Office (HMSO) has been publishing millions of parliamentary papers, passports and pension books since 1786 to preserve institutional memory. There is even a dedicated Knowledge & Information Management profession within the Civil Service. But it is a challenge that may have a novel solution in the form of Generative AI.

One of the principal arguments for a permanent civil service is to build an organisational memory that helps them do just that, an antidote to the churn of the political cadres that often last just a couple of years in any given role.

“The fuel that keeps the government running”: Generative AI’s public sector promise

It seems that barely a day goes by without someone identifying a new use case for Generative AI. It could be the plugins from Expedia or Shopify that let you use OpenAI’s ChatGPT to arrange your holiday or pick this season’s outfit, or the abundance of high-value startups in a market with a predicted 32% compound annual growth rate.

No doubt, the excitement around these innovations is being fuelled by OpenAI’s own content library of case studies, including ChatGPT’s use in Morgan Stanley’s information management. Knowledge management certainly does not capture the imagination in the same way as AI-generated podcasts. Still, OpenAI’s work with the investment bank’s wealth management portfolio is no less of a profound shift in content creation.

Drawing on hundreds of thousands of reports and analyst insights, the partnership has produced an internal-facing chatbot trained on “the cumulative knowledge of Morgan Stanley Wealth Management”. The chatbot can answer questions on as broad a spectrum as “what is our company’s view on the future performance of Tesla stock?” to “what are the biggest challenges facing the growth of the fast-food market”.

As Jeff McMillan, the company’s Chief Data and Analytics Officer, puts it, the chatbot is like “having our Chief Investment Strategist, Chief Global Economist, Global Equities Strategist, and every other analyst around the globe on call for every advisor, every day”.

The cross-sectoral opportunities of this use case are plentiful, nowhere more so than government. As one 2017 UK Cabinet Office report noted, “information […] is the fuel that keeps government running”. Generative AI offers the chance to amalgamate all that data and information in one place, available to a civil servant with just one prompt. It can be used to build policy memory; unlock access to best practices in government; save time and money in avoiding duplicative initiatives; and break down silos between departments and local governments.

Take the example of the policymaker who, faced with an apparently new problem for the first time, might spend weeks brainstorming and focus-grouping policy solutions only to find out that another department has attempted similar work before. Rather than reinventing the wheel, the civil servant could ask the chatbot for a summary of the different policy responses to similar problems in the past, and receive back several case studies and recommendations for policy responses.

Embracing the UK’s first-mover knowledge advantage

As governments begin to explore the integration of Generative AI in day-to-day operations, the UK government is almost uniquely placed to get a headstart.

But reaping the rewards of this modern knowledge management is not always straightforward and is premised on the strengths of an organisation’s existing knowledge infrastructure. Morgan Stanley was founded almost a hundred years ago, and its mass of research and analysis forms a core part of its business model. These documents are stored mostly in a machine-readable PDF format, meaning they can easily be used to train the AI chatbot. 

Government data is rarely as well-maintained. Clearly, a great deal of systemic change still needs to happen before all government data is machine-readable and able to train a chatbot. But, as governments begin to explore the integration of Generative AI in day-to-day operations, the UK government is almost uniquely placed to get a headstart.

Besides the existing infrastructure of HMSO and the Knowledge & Information Profession, the jewel in the UK’s knowledge management crown is undoubtedly its archives. Since 1958, the government has been required to preserve records for public access after a certain time (currently 20 years), while Hansard keeps a record of every speech in parliament. The national archives contain more than 9 million items available for download (and another 23 million items yet to be digitised).

Then there is GOV.UK. In replacing nearly 2,000 government websites with just one, the website has inspired governments from Australia to Israel. Serving millions of users a day, the website's developers have been working for years on structuring and tagging data across more than half a million pages with the help of supervised machine learning. Today, not only is the site a wealth of government blogs, news and policy, all of this already exists in a structured format leveraging supervised machine learning.

Of course, the practicalities of chatbot implementation are not the only barrier to their adoption. Last month, the Italian government banned ChatGPT, citing privacy concerns and pending investigation into GDPR compliance. Generative AI models ought to be tested against the UK’s strict data protection laws, which include, but are not limited, to GDPR (particularly when it comes to personal information or security-sensitive data). In the meantime, civil servants have already been warned not to use AI chatbots to carry out government work, and the government is right to caution against the immediate hype.

But Generative AI is impossible to ignore. When it comes to its use in government, Morgan Stanley’s knowledge management platform is a striking vision of the future. To maintain its first-mover advantage, Whitehall - and especially the Cabinet Office’s Central Digital and Data Office (CDDO) - now needs to explore the opportunities to embrace it sooner rather than later.