The effectiveness of the Cloud and AI Development Act regarding data centres

 

Annelieke Mooij, Assistant Professor, Tilburg Law School

Photo: Facebook Clonee (Ireland) data centre

Photo credit: Thomas Nugent, via Wikimedia commons

 

1.    Introduction

The debate about sovereignty and specifically digital sovereignty is fierce. Member States, struggle to achieve digital sovereignty which impacts the continuity and safety of the digital services. To speed up the realization of the sovereign cloud the Commission has proposed a new act the Cloud and AI Development Act (CADA). The act covers three important facets: AI, Cloud and Data centres. This contribution is limited to the proposed rules regarding data centres and analyze their effectiveness. The proposed Regulation should not be considered a stand-alone Regulation but rather forms part of the European Union’s broader strategy to strengthen digital sovereignty. The EU aims to reduce dependence on foreign digital service providers and expand Europe’s cloud and data centre capacity. At its core, the proposed Regulation seeks to create the conditions necessary for a sovereign European cloud ecosystem. A system that can support economic growth, innovation, and public-sector resilience.

The pursuit of cloud sovereignty, however, depends on more than software, governance frameworks, or industrial policy. Cloud services, AI and other digital services ultimately rely on physical infrastructure. Data centres provide storage, computing power, and connectivity upon which cloud services and AI applications depend. Without sufficient data centre capacity, ambitions for European technological autonomy cannot be achieved. The Commission aims to stimulate the increase of the data centre capacity through the CADA. The CADA therefore introduces a regulatory framework aimed at accelerating the development of data centres.  The Commission aims for the EU capacity to have tripled by 2030, and by 2035, all critical infrastructure will be hosted in EU data centres. These objectives are ambitious but there are good reasons for the Commission to emphasize digital sovereignty.

2. Why Digital Sovereignty Is Necessary & Difficult

Before delving into the CADA, it is important to understand why digital sovereignty is important to the EU. Cloud computing provides its users with remote access to data storage, software, and computing resources hosted on external servers. By outsourcing storage and computing functions to the cloud, organizations can reduce the need to maintain their own IT infrastructure. Whilst benefiting from greater scalability and operational flexibility. Public authorities increasingly rely on cloud services for precisely these reasons.

At present, however, the European cloud market is heavily concentrated. American providers dominate the sector, with Amazon Web Services and Microsoft accounting for approximately 75% of the European market. The largest European provider holds only a marginal 2% share. This concentration creates a structural dependency on foreign companies for critical digital infrastructure. One of the principal objectives of CADA is therefore to reduce this dependency and strengthen Europe’s technological autonomy.

The strategic rationale for reducing dependence on non-European providers extends beyond concerns about market concentration. Control over cloud infrastructure increasingly translates into geopolitical influence. A recent example is that of the latest AI-model created by Anthropic. The US government prohibited Anthropic from releasing its newest and most powerful model to its European customers. The official reason was due to considered possibilities of jailbreaks. The possibility of this happening was strongly denied by Anthropic. The EU Commission, however, stressed the possible negative impact on EU cybersecurity and cyberdefense.  It has been illustrative of how the US can cut-off new technologies to the EU, without serious repercussions. These incidents have not remained limited to simply limiting foreign accessibility but also to demonstrate power to individuals and institutions. Illustrative of this is the disruption of e-mail communications involving the International Criminal Court. The ICC’s email was cut-off after President Trump disagreed with actions from its main prosecutor. The case illustrated how political pressure exerted through private technology providers, without court permission, may affect the continuity of essential digital services. This dependence can be dangerous for Europe as it includes technology that is necessary for military purposes.

To achieve the desired increased EU cloud and AI capacity, physical infrastructure (or hardware) is necessary. Cloud and AI systems require data centres to operate on. The CADA therefore introduces a framework to create the necessary infrastructure.

3. The CADA’s rules on achieving data centre capacity.  

3.1. Specific Objectives

The CADA seeks to establish what the Commission describes as a coordinated and integrated ecosystem approach to cloud computing and artificial intelligence. According to the Commission, divergent national approaches to data centre permitting barriers to the efficient functioning of the internal market and hinder the development of a competitive European cloud ecosystem.

Against this background, the Regulation aims to create the conditions necessary for the large-scale deployment of cloud and AI infrastructure throughout the Union. In addition to reducing regulatory fragmentation, the proposal seeks to strengthen technological sovereignty, improve operational resilience, and support public-order objectives. The Commission further presents the Regulation as an instrument for promoting innovation and sustainability in Europe's digital infrastructure.

Regarding data centres specifically, the proposal seeks to address perceived shortages in computing and storage capacity through a combination of harmonisation measures and accelerated deployment procedures. This should lead to a specific result namely; triple the data centre capacity by 2030 and have sufficient EU data centre capacity for critical infrastructure by 2035.

3.2 Role of the Commission

The responsibility for achieving the operational objectives established by the CADA rests primarily with the European Commission. To achieve its goals, the proposal relies heavily on existing and future funding programmes intended to stimulate the development of cloud and AI technologies. These programmes seek, among other things, to improve the efficiency with which computing resources are used.

Particular emphasis is placed on technological innovation. High Performance Computing (HPC), for example, may increase the amount of computing output generated from a given level of infrastructure. More efficient use of computing resources can reduce the relative amount of storage and processing capacity required to achieve a particular outcome. Nevertheless, such efficiency gains do not eliminate the need for physical infrastructure. High-performance computing still depends on data centres and therefore remains subject to the same underlying constraints relating to energy, water, and spatial planning.

The effectiveness of this strategy consequently depends largely on the success of research and innovation projects supported through European funding programmes. There is ample reason to believe these strategies can be successful, economic literature has long recognised that research subsidies can stimulate innovation by reducing investment costs and encouraging experimentation. The chance of success, however, depends on the knowledge of the subsidy provider. In the past EU subsidies have proven a successful strategy. It is to be expected that the aim of development through subsidies will be successful again.

3.3. Data centre Acceleration Zones

To facilitate the expansion of data centre capacity, the CADA introduces so-called Data Centre Acceleration Areas (article 10). Each Member State is required to designate at least one such area for the accelerated development of data centre infrastructure, within six months of the Regulation entering into force.

When identifying acceleration areas, Member States must consider existing and future infrastructure capacity, energy availability, and broader sustainability considerations. The proposal further requires national authorities responsible for spatial planning to consider future data centre development and the necessary supporting infrastructure in those zones.

A developer wishing to develop a data centre in such an acceleration zone, will have the right to be assisted by a single point of information (article 11). This single point of information can assist the developer by sharing and coordinating the necessary permits and environmental and habitat assessments. The latter will be a sped-up procedure in accordance with Regulation 2026/XXXX on speeding-up environmental assessments. This Regulation was proposed in December 2025 with the aim to simplify environmental assessments. These rules aim to ensure that new projects have completed the permitting procedures within a year. The latter is the maximum that permitting procedures are allowed to last.

Taken together, these measures are intended to reduce administrative burdens and increase legal certainty for developers. The underlying assumption is that lengthy and fragmented permitting procedures constitute a significant obstacle to data centre deployment. To the extent that regulatory complexity delays investment, the proposed measures may indeed facilitate development. It is, however, questionable whether regulatory procedures are the primary challenge. The extent to which these objectives can be achieved in practice is, however, less clear. The realization of data centres come with significant challenges. When in operation, data centres become increasingly hot. With temperatures rising to 70 degrees in an hour. To continue their operations data centres need to cool. The cooling process requires high amounts of energy and clean water. E.g. data centres in the Netherlands constituted for approximately 5% of electricity demand in 2024. Recently a data centre by Microsoft made headlines that it uses 1% of the total national energy in the Netherlands. This whilst on the other hand there are significant shortages in energy supply for new housing and net congestion is increasing. Thereby creating serious debates on whether power should be diverted to data centres. On an EU level the targets for energy consumption are not yet met. The reduction target is approximately 18% away from its 2030 target. In 2024 the EU was 17% from renewable energy targets for 2030. In 2024 data centres consumed roughly 3% of the EU’s energy. Tripling this number and increasing it further till there is sufficient capacity for digital sovereignty creates a significant challenge.

These concerns are not limited to energy, the Netherlands is estimated to have a drinking water shortage by 2030. This whilst the data centres require approximately 3.7 million tonnes of drinking water per year, roughly 0.3% of Dutch tap water consumption, in the EU it is estimated to total 5.747.764.000 (nearly 6 billion liters). The CADA does not provide solutions to these underlying constraints. Instead, it requires Member States to create data centre acceleration zones and take infrastructure into account when designating these zones. Within these zones permitting procedures must be conducted within 12 months. While this may improve planning and coordination, it does not generate additional electricity capacity, alleviate network congestion, or increase the availability of water resources. The permits may become a hollow factor. A good example of the potential irony is that of the data centre in the Netherlands. The data centre had the required planning permits but were put on a waiting list for their energy connection.

The requirements created by the CADA may seem with a large margin of discretion as it uses language such as “take into consideration”. This language does not exert pressure on Member States. The CADA, however, also includes the earlier mentioned hard objective to triple the data centre capacity by 2030. Here lies another difficulty with the proposed framework. The CADA does not introduce a division key for how much capacity must be realized by each individual Member State. There are, however, big gaps between Member States in the current capacity.

Hungary for example only has 7.3MW of total capacity whereas Germany has 2.6GW of IT power. Arguably the capacity can be divided equally over all Member States, using the GDP as percentage divider. GDP is an indication of how much IT power is consumed in the economy. Generally, the higher the GDP the higher the IT consumption is. There is, however, little data on the demand for critical infrastructure in the EU. This is likely to change as article 15 of the CADA charges the EU Commission with obtaining that data. From an environmental perspective it is, however, ineffective to simply divide along GDP. Countries with cold climates and large coastal areas can build new more efficient data centres as the can use ocean water or outside air to cool. A submerged data centre on the coast is more sustainable than a data centre in a desert. In theory, the incentive to build data centre capacity by these countries is profit. Countries with favorable circumstances can build capacity cheaper than others and sell the capacity for profit. This theory of absolute advantage seems undermined by the next section of the CADA; the introduction of the European Cloud Federation.

 

4. The European Cloud Federation

In addition to measures aimed at expanding data centre capacity, in articles 34 and 35 the CADA introduces the proposed EuroCloud Federation. Participation in the Federation is voluntary and open to EU institutions and public-sector bodies. The purpose of the Federation is to facilitate the sharing of public cloud and data centre resources among participating members.

The underlying rationale is straightforward. Public authorities do not always utilize their available computing resources at full capacity. By enabling participating organisations to share infrastructure, the Federation seeks to improve the utilisation of existing resources and reduce unnecessary duplication of investments. In principle, such an approach may contribute to a more efficient use of public infrastructure.

To facilitate this objective, the proposal establishes a framework governing access to and sharing of infrastructure within the Federation. A notable feature of this framework is the limitation placed on financial compensation. Under Article 35(5), members providing infrastructure may recover their costs but are not permitted to generate profit from sharing their capacity with other participants.

From the perspective of short-term efficiency, this approach is understandable. Allowing access at cost price reduces barriers for participating entities and may encourage greater use of available infrastructure. The arrangement may therefore improve the allocation of existing capacity within the public sector.

The longer-term effects are less clear. The development of additional infrastructure requires significant investment and involves financial and operational risks. Where providers are unable to obtain any return beyond cost recovery, but cost recovery is not guaranteed, the incentive to create surplus capacity that can later be shared within the Federation may be reduced. Public entities may conclude that it is more attractive to rely on the capacity of other participants than to invest in additional infrastructure themselves.

 

5. Conclusion: a failed attempt?

The aim of the CADA is to increase the total EU data centre capacity. The CADA, however, does not create a division key. This is a fundamental gap within the regulation, it is too easy to state that all Member States should triple their data centre capacity equally. At present there are high differences in capacity between the different Member States.

The proposed Cloud and AI Development Act represents an ambitious attempt to strengthen European digital sovereignty through the expansion of cloud and AI infrastructure. Central to this ambition is the objective of significantly increasing data centre capacity across the European Union. To facilitate this development, the Regulation requires Member States to designate acceleration areas, develop national cloud and AI strategies, and participate in a broader framework intended to support the growth of sovereign digital infrastructure.

The proposal therefore sends a clear political signal. Data centres are no longer regarded as purely commercial infrastructure but as strategic assets that are essential for economic competitiveness, public administration, and technological autonomy. In that respect, the CADA forms part of a broader shift in European policy towards reducing strategic dependencies in critical digital technologies. Nevertheless, the CADA does not solve issues regarding natural resources. The introduction of the EU Cloud Federation furthermore has the potential to undermine a sustainable and economically efficient capacity division.  

 

From Large Language Models to Long Lasting Manipulations: the AI Act and generative AI advertising

 

Annelieke Mooij*  and Anuj Puri**

*Assistant Professor at the Public Law & Governance Department, Tilburg Law School

** Post-Doctoral Researcher at the Public Law & Governance Department, Tilburg Law School

Photo credit: piqsels, via Wikimedia Commons

 

Introduction

General purpose Large Language Models (LLMs) are amongst the most discussed innovation of the century, with AI developers even being named persons of the year by the Times magazine. Amongst the leading General purpose LLMs, perhaps the most famous one is ChatGPT – which is offered by Open AI. In light of such success, it may be surprising for many to learn that Open AI operates at huge losses. Its annual revenue is predicted at 13 billion dollars, which suffices to only a fraction of its computing costs which totals approximately 1.4 trillion dollars over the next eight years. It was therefore not entirely unexpected that Open AI was preparing ChatGPT for the inclusion of advertisement. The potential introduction of such advertisements raises significant ethical and legal concerns.

Consider the following excerpt from ChatGPT’s Memory FAQ,

ChatGPT can remember useful details between chats, making its responses more personalized and relevant. As you chat with ChatGPT, whether you’re typing, talking, or asking it to generate an image, it can remember helpful context from earlier conversations, such as your preferences and interests, and use that to tailor its responses.

Depending upon one’s penchant towards customization or preference for privacy, such features may either increase usability or raise privacy concerns or both. The inclusion of advertisement within General purpose LLMs should, however, concern even the least privacy conscious users.

ChatGPT has already taken steps to provide a user with customized in-conversation and instant check-out shopping, thereby creating new potential avenues for manipulation of consumers. Hence, it is not surprising that its plan to introduce ads was met with backlash. Most of the critique, however, seemed focused on the inclusion of advertisement in ChatGPT pro-plans and the lack of quality of the suggested ads. The advertisement plans have purportedly been currently put on hold to improve ChatGPT’s core features including personalization. It is not unlikely that ChatGPT may roll out an improved version that includes personalized ads. Hence, there exists an urgent need to examine the possibility of such advertisements manipulating consumers.

Manipulation Risks

Consider a potential scenario where an individual is in distress over the fall out of a personal relationship and reaches out to a general purpose LLM like the ChatGPT for advice. The LLM responds by advising the user to spend time and money on self-care by shopping for products such as clothes, shoes etc. with “helpful” links to shopping websites and perhaps a “helpful” image of the product. The user’s prior usage history may lead to their vulnerable situation being exploited for surveillance capitalist purposes. Such plausible uses of user’s information by the firms developing and deploying general purpose LLMs raise concerns pertaining to the use of manipulative techniques.

              From an ethical perspective, manipulation can be understood in various ways— such as manipulation in the form of introduction of non-rational influence (which renders it closer to subliminal technique), manipulation as a form of pressure, and manipulation as a form of trickery (which is conceptually linked to deception). Susser et al have defined manipulation as imposing a hidden or covert influence on another person’s decision-making and offered a widely accepted account of online manipulation as the use of information technology to covertly influence another person’s decision making. Manipulation understood in this manner raises concerns pertaining to the covert exploitation of a LLM user’s emotional vulnerabilities for commercial exploitation purposes. Before we address the question of existing legal remedies, it would be helpful to highlight some of the backdrop conditions which pave way for the potential manipulation of the consumers.

              Two common conceptual concerns lie in the backdrop of the purported use of manipulative AI— trust and anthropomorphization. The propensity of users to trust general purpose LLMs with queries pertaining to all aspects of their lives, even when they are not trustworthy, is at the heart of the  manipulation risks. Secondly, the conversational nature of the interaction with the LLM increases the odds of the user getting exploited on account of the tendency to anthropomorphize such interactions. It is worth noting that the undeserved inducement of trust and anthropomorphization are borne out of the design choices made by the developers. The potential to rely on previous conversations, the covert nature of the exercised influence, trusting propensity of the users along with the tendency to anthropomorphize the conversation lead to a fertile ground for potentially long-lasting manipulation of the user. This is where the legal remedies provided under the EU AI Act have an important role to play in protecting vulnerable users.

Legal Remedies

Article 5 of the AI-Act prohibits the deployment of manipulative AI. It is, however, difficult to define what constitutes manipulation. According to the Commission’s Guidelines on the AI Act, “[m]anipulative techniques are typically designed to exploit cognitive biases, psychological vulnerabilities, or situational factors that make individuals more susceptible to influence” This raises the question when ChatGPT’s advertisement exploits a psychological vulnerability and/or situational factor. And whether legal distinctions of vulnerabilities can reasonably be made.

One way of addressing the question of vulnerability is by examining the tendency to anthropomorphize general purpose LLMs. Users trust such LLMs as confidants instead of realizing that their data is being used for commercial exploitation. In view of such tendencies and dependencies, one could argue that general purpose LLM advertisements are inherently exploiting the vulnerability of the users. Thus, such advertisements are manipulative by design. This, however, fails to recognize that some users may only use the LLM as a search engine. This ambiguity in usage demonstrates the legal conundrum surrounding the identification of vulnerability.   

When it comes to consumer protection, the question of exploitation of vulnerability has been addressed in the Unfair Commercial Practices Directive. In consumer cases, the Court of Justice of the EU has held that the in order to be considered unlawful advertisement should manipulate a reasonably informed and circumspect consumer. The average consumer is an interpretative standard that the CJEU develops based on the product as an expression of proportionality. The average consumer is defined in relation to a product’s target audience, certain groups, such as children, are considered inherently more vulnerable. Gaming platforms for children, for instance, must therefore comply with stricter advertisement rules. From the perspective of vulnerability determination on the basis of product, it is an open-ended question what does being a reasonably informed and circumspect user of AI entail? Should it be assumed that AI users always have a minimum level of knowledge that all their interactions with the LLMs are aimed at commercial gain? Should the reasonable consumer be circumspect that all prompts are potential data fodder for exploiting (future) vulnerabilities; and be suspicious of all AI results at all times? Even when they look up the recipe for apple pie? There are some who would argue that advertisement based on algorithms and big data are inherently manipulative. If we accept this argument, general purpose LLMs should not be able to include any form of advertisement.

As stated before, development and deployment of AI systems is currently extremely resource intensive. A proponent of inclusion of advertisement in general purpose LLM may therefore argue that ad-driven revenue generation model reduces digital exclusion. This argument however begs the question whether access to AI systems in garb of exploitation of user’s vulnerabilities through manipulation is equitable access at all. A more sustainable solution is perhaps not to prohibit advertisement, but to regulate against exploitation. This, however, requires a shift in approach.

A possible solution could be to train AI systems to differentiate between (extremely) vulnerable prompts (questions) such as how to deal with a break-up and prompts with lower vulnerability such as how to bake an apple pie. This would require a shift in perspective. Rather than defining the average consumer, it would require defining the average prompt or AI interaction, whereby prompts such as “how to get over a break-up” indicate a vulnerability that is legally protected from exploitation. However, such a classification attributes the power to AI systems to distinguish between users that are in a potentially vulnerable  state and those that are not. Even if such a hypothetical position were to be possible, it would not address all the underlying ethical and legal concerns. Algorithmic determination of vulnerability is as likely to reflect the normative choices made by the developers and the computational trade-offs made in the training data sets. It is unlikely that these accurately reflect vulnerabilities without bias, as development of AI systems is not known to reflect diversity.

Another avenue to explore is not to regulate the prompts, but the amount of personal history that general purpose LLMs may access to generate advertisement. Such regulation would do justice to the argument that big data & algorithm is inherently manipulative. Limiting the amount of data that can be used for advertising has the additional advantage of clarity. If for instance only ten data points can be used for advertising, this provides legal certainty. The difficulty however is enforcement – as it requires verifying source codes. Further it is difficult to construe a safe harbour provision for data collection, depending upon the nature of data points, even limited data can be used for undermining an individual’s autonomy. Furthermore, it does not reflect the reality of people who use a LLM as a confidant, even though it is not trustworthy, on account of the seemingly anonymized and private interaction with the AI system. Thus making them vulnerable towards its manipulative influences.

Questions for the future remain

While the potential introduction of advertisements in general purpose LLMs such as the ChatGPT might have been paused for now, the financial incentives to justify the Silicon Valley optimism remain, which are currently also driving the policy measures across the Atlantic. The advent of these potential advertisements would not be the last attempt to test the regulatory resolve in the technological battle to impinge upon human autonomy. But by taking a strong stand, and withstanding the geo-political pressure, the EU institutions can make it amongst the first red lines that should not be crossed.

 

 

Policing Facial Recognition — Between Risks, Misconceptions, and the Need for a More Honest Debate

 

Asress Adimi Gikay (PhD) Senior Lecturer in AI, Disruptive Innovation and Law, Brunel University of London

 

Photo credit: Abyssus, via Wikimedia commons

 

Live facial recognition on the rise

 

Live facial recognition (LFR), is quickly gaining ground across Europe, with countries like Germany having used it to target serious criminal offences. The technology scans people’s faces in real time and matches them against police watchlists (e.g., people suspected of committing serious crimes). The EU’s Artificial Intelligence(AI) Act, allows police in member states to use LFR for serious crimes such as terrorism. However, the implementation of the EU AI Act in member states will likely face challenges as technical issues such as accuracy and legal boundaries are yet to be adequately tested.

 

Meanwhile, the UK Metropolitan Police have gained an extensive experience in managing the risk posed by the technology, arresting more than 1,000 people between January 2024 and August 2025.  In August 2025, despite opposition from 11 civil liberty groups, the Metropolitan deployed LFR at the Europe’s largest street festival celebrating African-Caribbean culture,  Notting Hill Carnival,  making  61 arrests.

 

The Metropolitan Police have taken the most step to address one of the biggest challenges in the use of the technology, i.e., ethnic bias. However, a controversy remains as to whether ethnic bias has been adequately tackled with data being interpreted differently to support the specific narrative being advanced. Misconception or misframing of critical notions in the field surveillance also shape public perception and could potentially inform policy and regulatory choices that are not necessarily evidence based.  I believe the prevailing positions adopted by academics and civil society groups also partly reflect such a state of affairs— selective use of data, unwarranted anxiety about surveillance and misconceptions around core legal concepts.

 

The view  predominantly advanced today by academics and civil liberty groups is a proposal for banning or imposing moratorium on the use of LFR on the ground that it is inaccurate, ethnically biased, susceptible to racially discriminatory use and enables mass surveillance. Whilst these are valid concerns, the Metropolitan Police’s experience over the past decade and the debate it sparked illustrates that the debate over governing the technology often doesn’t fairly weigh human rights and public safety concerns. Based on the experiences from the use of LFR technology in UK policing, in this post, I cover issues that often don’t surface wider-public discourse, some of these issues being crucial in providing insights into how LFR technology can deployed in the EU under the AI Act as well as other jurisdictions.

 

From backlash to acceptance 

 

Critics often describe policing facial recognition as Orwellian surveillance tool.  Yet history shows facial recognition is not the first or only technology to raise such a fear.

 

When Transport for London released a poster in 2002 announcing CCTV on buses, the design featured a double-decker bus gliding under a sky, with floating eyes. Its slogan read— “Secure Beneath the Watchful Eyes.” Simon Davies, the then head of Privacy International described it as “acutely disturbing.”  Two decades later, CCTV is widely accepted as an essential tool for solving crimes.  

 

 

Big Brother Watch, initially opposed airport facial recognition e-gates, warning that the system creates  privacy intrusive massive database of personal information and is prone to risk of error. Today, automated border control in Europe is considered a privilege, allowing faster passport control, available primarily to European passport holders. ‘Other travellers’ undergo more intrusive security control, including through fingerprints.

 

New technologies usually caused alarm, until their public benefits become clearer and they gain legitimacy. I don’t believe policing facial recognition is any different.

 

Measuring the impact of ethnic bias is tricky

 

Concerns about bias in facial recognition stem from early studies of commercial gender-classification algorithms and Metropolitan Police’s initial deployments that showed poorer accuracy especially for black women.  

 

However, a 2023 audit by the National Physical Laboratory (NPL), commissioned by the Metropolitan police found that when the system is optimally set, it works without significant ethnic disparities.

 

A crucial factor is the ‘recognition confidence threshold,’ or ‘face match threshold’ which determines how accurately the software matches faces. It ranges between 0-1. Higher settings reduce errors but yield fewer face matches while lower settings give more matches with less accuracy. The Metropolitan Police currently uses 0.64, a level recommended by the NPL to reduce ethnic bias significant enough to treat is as not concerning(statistically insignificant).

 

The NPL’s test involved 400 volunteers embedded in an estimated crowd of 130,000. The test showed that at a 0.64 setting or higher, there was no ethnic disparity in accuracy. At thresholds of 0.62 and 0.60, ethnic bias was statistically insignificant, while at 0.58 and 0.56, the system struggled to identify black faces.

 

Pete Fussey, a recognised expert in this field, contends the sample was too small to support  such a conclusion and notes that “false matches were not actually assessed at the settings where ethnic bias was non-existent”.   This essentially rests on the fact that for a technology that scans millions of faces, testing it on faces of 400 volunteers is less likely to generate a sufficient evidence base. In their book, Facial Recognition Surveillance: Policing in the Age of Artificial Intelligence (p, 58), Pete Fussey and Daragh Murray argue:

 

“Also of note are claims that no demographic bias is discernible above the 0.64 threshold. This is because no false positives occurred at this level. Put another way, no bias was observed because the system was not adequately tested in this range. Notable here is that such arguments rest less on how FRT operates and more on how statistics work  A suitable analogy would be the claim that 90 per cent of car accidents occur within a quarter-mile of home. This is less because such locales are inherently hazardous and more because almost all car journeys happen within a quarter-mile of home. Fewer journeys occur 600 miles away so accidents in that category are rarer. ”

 

However, a counter-argument to above is that the test in question did show steady decline in ethnic disparities with higher face match thresholds— at 0.56, 22 vs. 3 (Black vs. White); at 0.58, 11 vs. 0; at 0.60, 4 vs. 0; at 0.64, 0 vs. 0.  Despite the sample being smaller, the consistent decline implies that face match threshold clearly determines accuracy. The insistence on testing the technology until bias is completely removed is also unrealistic. So, if no inaccuracy was recorded at 0.64 and ethnic bias declined gradually up to that point, it would not be unreasonable to conclude that the technology works optimally at the given setting.

 

The NPL’s test is consistent with the risk management system in the EU AI Act, which sets strict standards for high-risk AI systems.  In its provisions requiring risk management for high-risk AI systems, in particular article 9(3), the AI Act requires that

 

“The risk management measures referred to in paragraph 2, point (d), shall be such that the relevant residual risk associated with each hazard, as well as the overall residual risk of the high-risk AI systems is judged to be acceptable.”

 

It means that the expectation in terms of risks including risk of ethic bias is not a complete elimination rather it is mitigation to the extent that some acceptable(tolerable) level of risks could still exist. By these standard, NPL’s testing is likely considered robust, since at 0.64 ethnic bias would reasonably be seen as low enough to be acceptable in view of the technology’s benefits

 

 

Subsequent Metropolitan Police’s deployment data is also indicative of this.  Between January and August 2025, the Metropolitan Police have misidentified only eight people using LFR, leading to no arrests. While ethnic breakdown for these false matches is not studied, the small number makes any ethnic disparity likely negligible.

 

Currently, there is one pending legal action brought against the Metropolitan Police by Big Brother Watch concerning prolonged police engagement with a mistakenly identified individual. This was not officially documented as false arrest, and therefore the official record in the UK is that there has not been a single false arrest following misidentification by LFR in the UK.

 

The above highlights that statistics alone doesn’t capture the complex ways LFR really affect people. Human oversight, responsible police judgment, and procedural safeguards play a crucial role; and the current debate discounts these components.

 

Policing by consent isn’t policing by of everyone’s consent

 

A common misconception is that overt(transparent) LFR surveillance undermines policing by consent, as people don’t meaningfully consent to being surveilled. 

 

Peter Fussey and Daragh Murray argue that, for instances, signages placed by the Metropolitan Police at deployment spots to inform the public of LFR operations were insufficient to obtain informed consent, as they contained inadequate information, lacked visibility and offered no opportunity for refusing consent.

 

Echoing this, former director of Big Brother Watch, Silkie Carlo stated in an interview, “there’s no meaningful consent process whatsoever. You certainly can’t withdraw consent.”

 

I think this view misrepresents both the law and the idea of policing by consent. The relevant UK Surveillance Camera Code of practice requires overt surveillance to be based on consent, specifically clarifying that consent in this context  should be regarded as “analogous to policing by consent”.

 

Policing by consent is  traced to the 9 point principles of Robert Peel, UK’s Home Secretary set out in the general instructions issued to new officers in 1829. Essentially, it requires public consent for police to serve the community where the legitimacy of policing power drives from public support. It does not require individual member of the public to consent to specific policing operations.

 

Similarly, surveillance by consent requires the community broadly to agree to visible camera systems as a legitimate tool for public safety, not whether everyone agrees to the surveillance. Besides facilitating legitimacy, transparent police surveillance ensures that those aggrieved by potentially unlawful surveillance can take legal actions.  The Surveillance Camera Code of Practice itself which is the basis for transparency in overt surveillance confirms this point by not only specifying that consent in this context is equivalent to policing by consent but also indicating the reason why consent is required. Section 3.3.2. states that “Surveillance by consent is dependent upon transparency and accountability on the part of a system operator. The provision of information is the first step in transparency and is also a key mechanism of accountability.” Nowhere in the code or any other legislation is it stated that surveillance by consent entitles individuals to consent to or withdraw consent to specific operations on individual level. Despite quoting the SCC including the relevant reference to policing by consent in their recent book, Peter Fussey and Daragh Murry don’t engage with the notion of policing by consent when they discuss consent in the context of overt surveillance, instead engaging with data protection law notion of consent. If consent of everyone who could be captured by LFR camera or even a normal CCTV came is to be secured, most public facing CCTV cameras would have to be removed.  

 

It is therefore legally and conceptually unfounded to claim that overt LFR surveillance requires the consent of everyone who walks by the LFR camera. Neither can this be realistically achieved in practice.

 

Surveillance harms, but context matters 

 

Opponents often alert that surveillance in public space, can deter people from speaking freely, attending protests, or joining public events, a phenomenon called the ‘chilling effect.’

 

In the context of LFR, Daragh Murray asserted that it might discourage attendance at the 2025 Notting Hill Carnival, citing uncertainty about how the technology is used and historical allegations of institutional racism against the Metropolitan Police.

 

The 2024 Carnival experienced two murders, multiple assaults, and stabbings, and yet an estimated two million people attended the Carnival this year, undeterred by the potential violence. Suggesting that surveillance would deter participation in such a cultural event is clearly implausible.  At the very least, there is no evidence to back this claim.

 

The chilling effect of surveillance is a concern in the context of political protests, where authorities may target opposition groups and threaten civil liberties. It can also be argued that excessive policing of minority communities may create a chilling effect to some extent, though this is highly context dependent. For example, the 2025 Carnival had 7,000 police officers with supporting technologies, and their presence was requested by the organisers and generally welcomed by the public. To suggest that adding LFR to this setting would have altered the behaviour of potential attendees is hardly credible. The blanket claim that surveillance suppresses civil rights  and alters behaviours in all contexts is not supported by evidence.

 

The bottom-line

 

Facial recognition will inevitably become routine policing tool. Rather than pushing unrealistic proposals of bans or moratoriums, regulatory debate should properly weigh the trade-offs between human rights and public safety in ensuring the proportionate use of the technology.   Questions about when LFR should be used and considered proportionate and other issues such as oversight should be debated carefully. However, the UK police’s use LFR, and the ongoing debate highlights that policy and regulatory proposals could be based on shaky interpretation of data and understanding of essential legal concepts.