The work underscores what can go wrong when an AI that manages city transport, safety, health and environmental monitoring predicts the future and intervenes in the present, significantly influencing urban governance and public policy development.
Generative Artificial Intelligence (AI) is boosting anticipatory forms of governance around the world, helping state actors to predict the future and focus their efforts in the present where the AI predicts they can have the greatest positive impact.
Particularly evident in China
This phenomenon is particularly evident in China, but similar forms of governance mediated by generative AI are also becoming increasingly popular in Europe and the seeds of this trend are already visible in Ireland.
In this context, 'city brains' represent an emerging type of generative AI currently employed in urban governance and public policy in a growing number of cities.
City brains are large-scale AIs residing in vast digital urban platforms, which use Large Language Models (LLMs) to generate visions of urban futures: visions that are in turn used by policymakers to generate new urban policies. In China alone, there are more than 500 cities developing city brains.
However, one of the main foreseeable dangers is the formation of a policy process that, under the influence of unintelligible LLMs, risks losing transparency and thus accountability, with another being the marginalisation of human stakeholders (citizens, in particular) as the role of AI in the management of cities keeps growing and governance begins to turn posthuman.
And by focusing on a real-world city brain project operating in the Haidian district of Beijing (China), which has a population of about 3,000,000 people and gathers data from more than 14,000 CCTV cameras and more than 20,000 environmental sensors, the researchers have been able to show these dangers are not just theoretical.
The dashboard of an existing City Brain system in China. Image credit: Dr Ying Xu.
Dr Federico Cugurullo, associate professor in Trinity’s School of Natural Sciences, is a leading expert in AI urbanism and the first author of the research, which has been published in the journal, Policy and Society.
Gigantic panopticon
He said: “We can think of the Haidian city brain project as a gigantic panopticon that constantly observes what is happening it the city. It is operated by AI and focuses on three main areas of governance that are shaped by its predictions: environmental risk management, traffic management and public security.”
“For example, in the case of an impending natural disaster, policies are rapidly implemented to build new infrastructure meant to reinforce riverbanks and increase the efficiency of the city’s drainage systems. Outcomes also include direct interventions when, for example, police officers are dispatched to prevent illegal activities in an area where, according to the city brain, crimes are likely to take place in the near future.
“However, the predictions of the Haidian city brain are far from being infallible. Our research reveals that the accuracy rate of what the city brain predicts varies from 60% to 90%, which leaves a significant margin of error around such important decisions, for which there is no understanding as to why they have been implemented. This is particularly dangerous when it comes to predictive policing, since any error made by AI means that an innocent person will be targeted by the police for hypothetical crimes that never took place.”
This research forms part of the ORACLE project led by Professor Cugurullo, which was funded by Research Ireland (formerly the Irish Research Council).
The published journal article is available to read, open access, on the publisher's website.