From Large Language Models to Long Lasting Manipulations: the AI Act and generative AI advertising

 

Annelieke Mooij*  and Anuj Puri**

*Assistant Professor at the Public Law & Governance Department, Tilburg Law School

** Post-Doctoral Researcher at the Public Law & Governance Department, Tilburg Law School

Photo credit: piqsels, via Wikimedia Commons

 

Introduction

General purpose Large Language Models (LLMs) are amongst the most discussed innovation of the century, with AI developers even being named persons of the year by the Times magazine. Amongst the leading General purpose LLMs, perhaps the most famous one is ChatGPT – which is offered by Open AI. In light of such success, it may be surprising for many to learn that Open AI operates at huge losses. Its annual revenue is predicted at 13 billion dollars, which suffices to only a fraction of its computing costs which totals approximately 1.4 trillion dollars over the next eight years. It was therefore not entirely unexpected that Open AI was preparing ChatGPT for the inclusion of advertisement. The potential introduction of such advertisements raises significant ethical and legal concerns.

Consider the following excerpt from ChatGPT’s Memory FAQ,

ChatGPT can remember useful details between chats, making its responses more personalized and relevant. As you chat with ChatGPT, whether you’re typing, talking, or asking it to generate an image, it can remember helpful context from earlier conversations, such as your preferences and interests, and use that to tailor its responses.

Depending upon one’s penchant towards customization or preference for privacy, such features may either increase usability or raise privacy concerns or both. The inclusion of advertisement within General purpose LLMs should, however, concern even the least privacy conscious users.

ChatGPT has already taken steps to provide a user with customized in-conversation and instant check-out shopping, thereby creating new potential avenues for manipulation of consumers. Hence, it is not surprising that its plan to introduce ads was met with backlash. Most of the critique, however, seemed focused on the inclusion of advertisement in ChatGPT pro-plans and the lack of quality of the suggested ads. The advertisement plans have purportedly been currently put on hold to improve ChatGPT’s core features including personalization. It is not unlikely that ChatGPT may roll out an improved version that includes personalized ads. Hence, there exists an urgent need to examine the possibility of such advertisements manipulating consumers.

Manipulation Risks

Consider a potential scenario where an individual is in distress over the fall out of a personal relationship and reaches out to a general purpose LLM like the ChatGPT for advice. The LLM responds by advising the user to spend time and money on self-care by shopping for products such as clothes, shoes etc. with “helpful” links to shopping websites and perhaps a “helpful” image of the product. The user’s prior usage history may lead to their vulnerable situation being exploited for surveillance capitalist purposes. Such plausible uses of user’s information by the firms developing and deploying general purpose LLMs raise concerns pertaining to the use of manipulative techniques.

              From an ethical perspective, manipulation can be understood in various ways— such as manipulation in the form of introduction of non-rational influence (which renders it closer to subliminal technique), manipulation as a form of pressure, and manipulation as a form of trickery (which is conceptually linked to deception). Susser et al have defined manipulation as imposing a hidden or covert influence on another person’s decision-making and offered a widely accepted account of online manipulation as the use of information technology to covertly influence another person’s decision making. Manipulation understood in this manner raises concerns pertaining to the covert exploitation of a LLM user’s emotional vulnerabilities for commercial exploitation purposes. Before we address the question of existing legal remedies, it would be helpful to highlight some of the backdrop conditions which pave way for the potential manipulation of the consumers.

              Two common conceptual concerns lie in the backdrop of the purported use of manipulative AI— trust and anthropomorphization. The propensity of users to trust general purpose LLMs with queries pertaining to all aspects of their lives, even when they are not trustworthy, is at the heart of the  manipulation risks. Secondly, the conversational nature of the interaction with the LLM increases the odds of the user getting exploited on account of the tendency to anthropomorphize such interactions. It is worth noting that the undeserved inducement of trust and anthropomorphization are borne out of the design choices made by the developers. The potential to rely on previous conversations, the covert nature of the exercised influence, trusting propensity of the users along with the tendency to anthropomorphize the conversation lead to a fertile ground for potentially long-lasting manipulation of the user. This is where the legal remedies provided under the EU AI Act have an important role to play in protecting vulnerable users.

Legal Remedies

Article 5 of the AI-Act prohibits the deployment of manipulative AI. It is, however, difficult to define what constitutes manipulation. According to the Commission’s Guidelines on the AI Act, “[m]anipulative techniques are typically designed to exploit cognitive biases, psychological vulnerabilities, or situational factors that make individuals more susceptible to influence” This raises the question when ChatGPT’s advertisement exploits a psychological vulnerability and/or situational factor. And whether legal distinctions of vulnerabilities can reasonably be made.

One way of addressing the question of vulnerability is by examining the tendency to anthropomorphize general purpose LLMs. Users trust such LLMs as confidants instead of realizing that their data is being used for commercial exploitation. In view of such tendencies and dependencies, one could argue that general purpose LLM advertisements are inherently exploiting the vulnerability of the users. Thus, such advertisements are manipulative by design. This, however, fails to recognize that some users may only use the LLM as a search engine. This ambiguity in usage demonstrates the legal conundrum surrounding the identification of vulnerability.   

When it comes to consumer protection, the question of exploitation of vulnerability has been addressed in the Unfair Commercial Practices Directive. In consumer cases, the Court of Justice of the EU has held that the in order to be considered unlawful advertisement should manipulate a reasonably informed and circumspect consumer. The average consumer is an interpretative standard that the CJEU develops based on the product as an expression of proportionality. The average consumer is defined in relation to a product’s target audience, certain groups, such as children, are considered inherently more vulnerable. Gaming platforms for children, for instance, must therefore comply with stricter advertisement rules. From the perspective of vulnerability determination on the basis of product, it is an open-ended question what does being a reasonably informed and circumspect user of AI entail? Should it be assumed that AI users always have a minimum level of knowledge that all their interactions with the LLMs are aimed at commercial gain? Should the reasonable consumer be circumspect that all prompts are potential data fodder for exploiting (future) vulnerabilities; and be suspicious of all AI results at all times? Even when they look up the recipe for apple pie? There are some who would argue that advertisement based on algorithms and big data are inherently manipulative. If we accept this argument, general purpose LLMs should not be able to include any form of advertisement.

As stated before, development and deployment of AI systems is currently extremely resource intensive. A proponent of inclusion of advertisement in general purpose LLM may therefore argue that ad-driven revenue generation model reduces digital exclusion. This argument however begs the question whether access to AI systems in garb of exploitation of user’s vulnerabilities through manipulation is equitable access at all. A more sustainable solution is perhaps not to prohibit advertisement, but to regulate against exploitation. This, however, requires a shift in approach.

A possible solution could be to train AI systems to differentiate between (extremely) vulnerable prompts (questions) such as how to deal with a break-up and prompts with lower vulnerability such as how to bake an apple pie. This would require a shift in perspective. Rather than defining the average consumer, it would require defining the average prompt or AI interaction, whereby prompts such as “how to get over a break-up” indicate a vulnerability that is legally protected from exploitation. However, such a classification attributes the power to AI systems to distinguish between users that are in a potentially vulnerable  state and those that are not. Even if such a hypothetical position were to be possible, it would not address all the underlying ethical and legal concerns. Algorithmic determination of vulnerability is as likely to reflect the normative choices made by the developers and the computational trade-offs made in the training data sets. It is unlikely that these accurately reflect vulnerabilities without bias, as development of AI systems is not known to reflect diversity.

Another avenue to explore is not to regulate the prompts, but the amount of personal history that general purpose LLMs may access to generate advertisement. Such regulation would do justice to the argument that big data & algorithm is inherently manipulative. Limiting the amount of data that can be used for advertising has the additional advantage of clarity. If for instance only ten data points can be used for advertising, this provides legal certainty. The difficulty however is enforcement – as it requires verifying source codes. Further it is difficult to construe a safe harbour provision for data collection, depending upon the nature of data points, even limited data can be used for undermining an individual’s autonomy. Furthermore, it does not reflect the reality of people who use a LLM as a confidant, even though it is not trustworthy, on account of the seemingly anonymized and private interaction with the AI system. Thus making them vulnerable towards its manipulative influences.

Questions for the future remain

While the potential introduction of advertisements in general purpose LLMs such as the ChatGPT might have been paused for now, the financial incentives to justify the Silicon Valley optimism remain, which are currently also driving the policy measures across the Atlantic. The advent of these potential advertisements would not be the last attempt to test the regulatory resolve in the technological battle to impinge upon human autonomy. But by taking a strong stand, and withstanding the geo-political pressure, the EU institutions can make it amongst the first red lines that should not be crossed.

 

 

Policing Facial Recognition — Between Risks, Misconceptions, and the Need for a More Honest Debate

 

Asress Adimi Gikay (PhD) Senior Lecturer in AI, Disruptive Innovation and Law, Brunel University of London

 

Photo credit: Abyssus, via Wikimedia commons

 

Live facial recognition on the rise

 

Live facial recognition (LFR), is quickly gaining ground across Europe, with countries like Germany having used it to target serious criminal offences. The technology scans people’s faces in real time and matches them against police watchlists (e.g., people suspected of committing serious crimes). The EU’s Artificial Intelligence(AI) Act, allows police in member states to use LFR for serious crimes such as terrorism. However, the implementation of the EU AI Act in member states will likely face challenges as technical issues such as accuracy and legal boundaries are yet to be adequately tested.

 

Meanwhile, the UK Metropolitan Police have gained an extensive experience in managing the risk posed by the technology, arresting more than 1,000 people between January 2024 and August 2025.  In August 2025, despite opposition from 11 civil liberty groups, the Metropolitan deployed LFR at the Europe’s largest street festival celebrating African-Caribbean culture,  Notting Hill Carnival,  making  61 arrests.

 

The Metropolitan Police have taken the most step to address one of the biggest challenges in the use of the technology, i.e., ethnic bias. However, a controversy remains as to whether ethnic bias has been adequately tackled with data being interpreted differently to support the specific narrative being advanced. Misconception or misframing of critical notions in the field surveillance also shape public perception and could potentially inform policy and regulatory choices that are not necessarily evidence based.  I believe the prevailing positions adopted by academics and civil society groups also partly reflect such a state of affairs— selective use of data, unwarranted anxiety about surveillance and misconceptions around core legal concepts.

 

The view  predominantly advanced today by academics and civil liberty groups is a proposal for banning or imposing moratorium on the use of LFR on the ground that it is inaccurate, ethnically biased, susceptible to racially discriminatory use and enables mass surveillance. Whilst these are valid concerns, the Metropolitan Police’s experience over the past decade and the debate it sparked illustrates that the debate over governing the technology often doesn’t fairly weigh human rights and public safety concerns. Based on the experiences from the use of LFR technology in UK policing, in this post, I cover issues that often don’t surface wider-public discourse, some of these issues being crucial in providing insights into how LFR technology can deployed in the EU under the AI Act as well as other jurisdictions.

 

From backlash to acceptance 

 

Critics often describe policing facial recognition as Orwellian surveillance tool.  Yet history shows facial recognition is not the first or only technology to raise such a fear.

 

When Transport for London released a poster in 2002 announcing CCTV on buses, the design featured a double-decker bus gliding under a sky, with floating eyes. Its slogan read— “Secure Beneath the Watchful Eyes.” Simon Davies, the then head of Privacy International described it as “acutely disturbing.”  Two decades later, CCTV is widely accepted as an essential tool for solving crimes.  

 

 

Big Brother Watch, initially opposed airport facial recognition e-gates, warning that the system creates  privacy intrusive massive database of personal information and is prone to risk of error. Today, automated border control in Europe is considered a privilege, allowing faster passport control, available primarily to European passport holders. ‘Other travellers’ undergo more intrusive security control, including through fingerprints.

 

New technologies usually caused alarm, until their public benefits become clearer and they gain legitimacy. I don’t believe policing facial recognition is any different.

 

Measuring the impact of ethnic bias is tricky

 

Concerns about bias in facial recognition stem from early studies of commercial gender-classification algorithms and Metropolitan Police’s initial deployments that showed poorer accuracy especially for black women.  

 

However, a 2023 audit by the National Physical Laboratory (NPL), commissioned by the Metropolitan police found that when the system is optimally set, it works without significant ethnic disparities.

 

A crucial factor is the ‘recognition confidence threshold,’ or ‘face match threshold’ which determines how accurately the software matches faces. It ranges between 0-1. Higher settings reduce errors but yield fewer face matches while lower settings give more matches with less accuracy. The Metropolitan Police currently uses 0.64, a level recommended by the NPL to reduce ethnic bias significant enough to treat is as not concerning(statistically insignificant).

 

The NPL’s test involved 400 volunteers embedded in an estimated crowd of 130,000. The test showed that at a 0.64 setting or higher, there was no ethnic disparity in accuracy. At thresholds of 0.62 and 0.60, ethnic bias was statistically insignificant, while at 0.58 and 0.56, the system struggled to identify black faces.

 

Pete Fussey, a recognised expert in this field, contends the sample was too small to support  such a conclusion and notes that “false matches were not actually assessed at the settings where ethnic bias was non-existent”.   This essentially rests on the fact that for a technology that scans millions of faces, testing it on faces of 400 volunteers is less likely to generate a sufficient evidence base. In their book, Facial Recognition Surveillance: Policing in the Age of Artificial Intelligence (p, 58), Pete Fussey and Daragh Murray argue:

 

“Also of note are claims that no demographic bias is discernible above the 0.64 threshold. This is because no false positives occurred at this level. Put another way, no bias was observed because the system was not adequately tested in this range. Notable here is that such arguments rest less on how FRT operates and more on how statistics work  A suitable analogy would be the claim that 90 per cent of car accidents occur within a quarter-mile of home. This is less because such locales are inherently hazardous and more because almost all car journeys happen within a quarter-mile of home. Fewer journeys occur 600 miles away so accidents in that category are rarer. ”

 

However, a counter-argument to above is that the test in question did show steady decline in ethnic disparities with higher face match thresholds— at 0.56, 22 vs. 3 (Black vs. White); at 0.58, 11 vs. 0; at 0.60, 4 vs. 0; at 0.64, 0 vs. 0.  Despite the sample being smaller, the consistent decline implies that face match threshold clearly determines accuracy. The insistence on testing the technology until bias is completely removed is also unrealistic. So, if no inaccuracy was recorded at 0.64 and ethnic bias declined gradually up to that point, it would not be unreasonable to conclude that the technology works optimally at the given setting.

 

The NPL’s test is consistent with the risk management system in the EU AI Act, which sets strict standards for high-risk AI systems.  In its provisions requiring risk management for high-risk AI systems, in particular article 9(3), the AI Act requires that

 

“The risk management measures referred to in paragraph 2, point (d), shall be such that the relevant residual risk associated with each hazard, as well as the overall residual risk of the high-risk AI systems is judged to be acceptable.”

 

It means that the expectation in terms of risks including risk of ethic bias is not a complete elimination rather it is mitigation to the extent that some acceptable(tolerable) level of risks could still exist. By these standard, NPL’s testing is likely considered robust, since at 0.64 ethnic bias would reasonably be seen as low enough to be acceptable in view of the technology’s benefits

 

 

Subsequent Metropolitan Police’s deployment data is also indicative of this.  Between January and August 2025, the Metropolitan Police have misidentified only eight people using LFR, leading to no arrests. While ethnic breakdown for these false matches is not studied, the small number makes any ethnic disparity likely negligible.

 

Currently, there is one pending legal action brought against the Metropolitan Police by Big Brother Watch concerning prolonged police engagement with a mistakenly identified individual. This was not officially documented as false arrest, and therefore the official record in the UK is that there has not been a single false arrest following misidentification by LFR in the UK.

 

The above highlights that statistics alone doesn’t capture the complex ways LFR really affect people. Human oversight, responsible police judgment, and procedural safeguards play a crucial role; and the current debate discounts these components.

 

Policing by consent isn’t policing by of everyone’s consent

 

A common misconception is that overt(transparent) LFR surveillance undermines policing by consent, as people don’t meaningfully consent to being surveilled. 

 

Peter Fussey and Daragh Murray argue that, for instances, signages placed by the Metropolitan Police at deployment spots to inform the public of LFR operations were insufficient to obtain informed consent, as they contained inadequate information, lacked visibility and offered no opportunity for refusing consent.

 

Echoing this, former director of Big Brother Watch, Silkie Carlo stated in an interview, “there’s no meaningful consent process whatsoever. You certainly can’t withdraw consent.”

 

I think this view misrepresents both the law and the idea of policing by consent. The relevant UK Surveillance Camera Code of practice requires overt surveillance to be based on consent, specifically clarifying that consent in this context  should be regarded as “analogous to policing by consent”.

 

Policing by consent is  traced to the 9 point principles of Robert Peel, UK’s Home Secretary set out in the general instructions issued to new officers in 1829. Essentially, it requires public consent for police to serve the community where the legitimacy of policing power drives from public support. It does not require individual member of the public to consent to specific policing operations.

 

Similarly, surveillance by consent requires the community broadly to agree to visible camera systems as a legitimate tool for public safety, not whether everyone agrees to the surveillance. Besides facilitating legitimacy, transparent police surveillance ensures that those aggrieved by potentially unlawful surveillance can take legal actions.  The Surveillance Camera Code of Practice itself which is the basis for transparency in overt surveillance confirms this point by not only specifying that consent in this context is equivalent to policing by consent but also indicating the reason why consent is required. Section 3.3.2. states that “Surveillance by consent is dependent upon transparency and accountability on the part of a system operator. The provision of information is the first step in transparency and is also a key mechanism of accountability.” Nowhere in the code or any other legislation is it stated that surveillance by consent entitles individuals to consent to or withdraw consent to specific operations on individual level. Despite quoting the SCC including the relevant reference to policing by consent in their recent book, Peter Fussey and Daragh Murry don’t engage with the notion of policing by consent when they discuss consent in the context of overt surveillance, instead engaging with data protection law notion of consent. If consent of everyone who could be captured by LFR camera or even a normal CCTV came is to be secured, most public facing CCTV cameras would have to be removed.  

 

It is therefore legally and conceptually unfounded to claim that overt LFR surveillance requires the consent of everyone who walks by the LFR camera. Neither can this be realistically achieved in practice.

 

Surveillance harms, but context matters 

 

Opponents often alert that surveillance in public space, can deter people from speaking freely, attending protests, or joining public events, a phenomenon called the ‘chilling effect.’

 

In the context of LFR, Daragh Murray asserted that it might discourage attendance at the 2025 Notting Hill Carnival, citing uncertainty about how the technology is used and historical allegations of institutional racism against the Metropolitan Police.

 

The 2024 Carnival experienced two murders, multiple assaults, and stabbings, and yet an estimated two million people attended the Carnival this year, undeterred by the potential violence. Suggesting that surveillance would deter participation in such a cultural event is clearly implausible.  At the very least, there is no evidence to back this claim.

 

The chilling effect of surveillance is a concern in the context of political protests, where authorities may target opposition groups and threaten civil liberties. It can also be argued that excessive policing of minority communities may create a chilling effect to some extent, though this is highly context dependent. For example, the 2025 Carnival had 7,000 police officers with supporting technologies, and their presence was requested by the organisers and generally welcomed by the public. To suggest that adding LFR to this setting would have altered the behaviour of potential attendees is hardly credible. The blanket claim that surveillance suppresses civil rights  and alters behaviours in all contexts is not supported by evidence.

 

The bottom-line

 

Facial recognition will inevitably become routine policing tool. Rather than pushing unrealistic proposals of bans or moratoriums, regulatory debate should properly weigh the trade-offs between human rights and public safety in ensuring the proportionate use of the technology.   Questions about when LFR should be used and considered proportionate and other issues such as oversight should be debated carefully. However, the UK police’s use LFR, and the ongoing debate highlights that policy and regulatory proposals could be based on shaky interpretation of data and understanding of essential legal concepts.

 

Revolutionizing Law? Exploring experimental regulations and artificial intelligence Sandboxes in the AI Act

 

 

Davide Rauhe

 

Photo credit: Chief Photographer, MoD

 

Executive Summary

 

This blog post explores the emergence of experimental regulations and policies with focus on its gained prominence within the European Union in the recent years.

 

As emerging technologies, especially in the field of artificial intelligence (AI), continue to shape our societies, there is a growing need for flexible regulatory approaches that can adapt to rapidly evolving technological landscapes. Experimental regulation and Regulatory sandboxes have gained popularity as a potential solution to foster innovation while ensuring the maintenance of minimum standards in fundamental rights and ethical questions.

 

Therefore, special focus shall be laid upon the significance of regulatory sandboxes and their implementation in the new AI Act on EU-level. This post analyzes the significance of these new law-making methods and answers the question whether or not lawmakers can benefit from them.

 

I. Introduction

 

Regulation and legislation can have strong impacts on the economy and society of a country. However, both are still regarded as a merely bureaucratic action, despite its influence on economic and social development. Indeed, legislation is by some even considered a major force in enabling capitalistic structures, assuming that law itself can create capital by allocating resources either by allowing, shaping or even prohibiting certain economic behaviors.^[1] This is particularly the case when sudden and impactful technological improvements are made as with them usually a shift in political and economic power is recorded as well – its design and scope is therefore of even greater importance.^[2] This applies especially to the European Union as legislator, because the EU as a regulative body influences other jurisdictions all over the world – a process often describes as the so-called Brussels-effect.^[3]

 

With the emergence of increasingly complex technological and innovative economic models in various economic sectors such as finance, commerce and others, there has been a greater demand for more diverse and innovative regulatory approaches in various sectors. However, the protection of fundamental rights standards takes a key role in this discussion as well as the age of “information” or “surveillance” capitalism comes with an increased danger for fundamental rights. Furthermore, new technologies often tend to change the distribution of wealth within societies and therefore possibly lead to either the reduction or the growth of inequality depending on how they are regulated.^[4] The misalignment of innovation and regulation can therefore be extremely problematic for societies.^[5] Major reasons for a misalignment have been localized in information gaps of regulators, the inability of regulators to fully grasp the innovational model itself and the invisibility of certain innovational flaws only until they become critical and therefore unmissable.^[6]

 

Experimental regulations and regulatory Sandboxes may provide a fitting and promising remedy to these conditions. Both concepts promise to handle innovative business models more effectively through the implementation of different adaptability and adjustment measures while also securing a sophisticated human rights standard. The discourse over these specific forms of regulation gained more prominence with the recent improvements of artificial intelligence (AI), a generic term that encompasses various technologies that are considered to have some form of intelligent behavior, and its spreading usage in various economic and scientific sectors. Especially with the public roll out of ChatGPT and other artificial intelligence powered Large Language Models (LLM), the potential of Artificial intelligence became more apparent to a greater circle of persons, including policymakers.^[7]

 

With the next technological milestones in artificial intelligence development already on the horizon, generally referred to as superintelligence, a dedicated and consistent approach is essential here.^[8] The goal in this regard is to reach so-called superalignment, which translates to agile and adaptive regulation combined with state of the art monitoring and reactional measures for any form of super-intelligent machines.^[9] But there is still a long way to go. Until then, however, the goal must remain to reach an alignment as far as possible.

 

Due to the rapid evolution of this technology and its improvements and changes, experimental regulation and sandboxes seem to serve here as the right method to effectively regulate this technology while not preventing economic and/or scientific growth using it. As seen in the recent implementation of the AI Act and its use of experimental legislation and sandboxes these lawmaking forms found their way into one of the most discussed and anticipated EU regulations in the recent years. The EU's AI Act represents therefore a significant milestone, using the concept of regulatory sandboxes on the EU-Level for probably the technology of the 21st century.

 

This blogpost tries to assess whether this legislative approach is the right measure to tackle artificial intelligence by analyzing its historical background and the legal implications of it. After that, the case of experimental regulations and legislative sandboxes and their effective provision in the AI act will be addressed through a case study on the AI act.

 

II. Experimental Regulation and Regulatory Sandboxes

 

To comprehensively analyze the concept of experimental regulation and regulatory sandboxes, particularly in the realm of AI, it is essential to clarify its specific meaning.

1. Experimental Regulation

 

Experimental Regulation is inherently designed to serve as a more adaptive and collaborative approach to regulatory compliance in comparison to more conventional regulation methods, offering a framework that fosters innovation while ensuring accountability. Generally speaking, experimental regulation can be seen as legislation which authorizes, monitors and executes legal experiments.^[10]

 

In its core, experimental regulation serves as an evidence-based form of law making in opposition to the conventional “trial & error” approach.^[11] Experimental regulation tries to limit the unpredictability of that approach to an acceptable amount.

 

The most notable features of experimental regulation are its temporal nature, its derogation from current, already existing statutes and finally the evaluation of the results stemming from the execution of the experiment.^[12] It can be therefore defined as a form of legislation that includes legislative measures on trial that serve the sole purpose of evaluating its effectiveness and practicability before its widespread and definitive implementation.

Experimental regulations are therefore a method for piloting fresh rules on a limited scale to assess their practical efficacy, tailor them to evolving conditions, and empower regulatory authorities to gain insights from the outcomes acquired in real-world scenarios.^[13] In this regard experimental regulation serves as a form of anticipatory and flexible regulation which encompasses preventive citizen protection while promoting innovation at the same time; its experimental framework and limited scale allows private actors and state authorities to analyze possible outcomes of technologies as well as of the regulation itself in a more detailed, safe and overall sophisticated way.^[14] Due to real-time feedback and constant evaluating of outcoming results, it is specially feasible for subjects and industries that require a fast and reactive regulative response to practical issues.

Furthermore, experimental regulation can easily adapt to cultural changes in economic behavior as itself is changing and adapting constantly as well.^[15] While it does not make sense to apply old regulation, which for instance was made for regulating taxi and cab services to more innovative transport companies like Uber, experimental regulation could adapt and adjust its rules to the seemingly similar, but essentially different business models of new market players more easily.^[16] The case-by-case approach of experimental regulation promises here a coherent and up to date regulation even in times with increasingly faster changing industries and economic realities.^[17] Moreover, it can even lead to a better cost-effectiveness of state spending as potential negative monetary impacts can be detected faster and the regulation can be adjusted before its widespread establishment.

2. Regulatory Sandboxes

 

Another and more recent form of experimental regulation is Regulatory Sandboxes. As has been the case with experimental regulation, one reason why sandboxes are being promoted is that traditional legislation is no longer regarded as adequately fitting to regulate innovative business models. These frequently outpace regulatory development, which can stifle innovation or lead to unregulated deployments, which could be seen in the case of Big Tech. Here, traditional legal measures like competition law weren’t capable of regulating these companies effectively during their rise-ups.^[18] In fact it could be even argued that their entire business-model cannot be monitored with previous existing measures.^[19]

The alternative to the mere adjustments of already existing regulatory frameworks could be the introduction of regulatory sandboxes. Introduced for the first time in 2016 within the UK for the enhancement of innovation within the fintech sector, regulatory sandboxes can serve as a method to effectively promote innovation while mitigating compliance with regulation.^[20]

Sandboxes facilitate close cooperation between public and private entities and provide secure environments for fostering innovation by either temporarily applying an alternate regulatory framework to a (pre-)selected group of companies or by providing guidance on compliance through public actors.^[21] Usually, but not necessarily, both is the case. It therefore is a safe space for (often) start-ups and established companies to test new technologies, products, or services that are usually not compliant with current legislations within a limited, well-defined scope and under public supervision.^[22]

 

These controlled, yet real-world environments allow for a sophisticated testing of services, products and/or market approaches while minimizing the risks associated with unchecked and new technologies as they affect here only a limited circle of individuals and/or companies with proper safeguards provided.^[23] The sandboxes‘ duration depends on the decision-making authority, but they usually last up to 12 months.^[24]

While the private parties involved in the sandbox regime gain important information on client impressions, lawmakers can learn from emerging technologies and refine already existing regulations, ensuring this way the maintenance of an ethical but also efficient regulation.^[25] In contrast to experimental regulation in the narrow sense, regulative sandboxes do not always foresee the derogation of existing legislature within the laboratory-like framework.^[26] Here, the focus lies more on the collaborative factor between companies and the regulator and sometimes even only between companies and other private actors.^[27]

 

By offering a safe space for experimentation, sandboxes shall promote ideally innovation and learning on both sides, facilitating a two-way dialogue between innovators and regulators. This dialogue is instrumental in fine-tuning the regulatory framework as technologies advance since the knowledge necessary for effectively regulating increasingly more complex economic models becomes more and more complex itself.

 

The micro-optimizing and technology-specific approach promises to lead to satisfactory results, that can then be applied to a greater scale or other sectors/technologies.^[28] It also minimizes knowledge gaps between regulators and innovators as the constant exchange of information lets the legislator gain a wider understanding of new products, which makes it in turn easier to adjust regulation to the specifics of the product.^[29] Here, it is crucial to adjust regulation in the early stages of the development process as later changes may be already outdated or even harmful to the new standard, which the innovation usually gains more quickly after a certain period of time.^[30] Overall Sandboxes can therefore lead to better informed and tested regulation, making it likely to prevent flaws in legal regimes like the before-mentioned competition law.

 

Furthermore, sandboxes can accelerate efficient, coherent, and ‘bullet-proof’ regulation, thus also improving legal certainty for businesses.^[31] On top of that they also make it easier for companies to comply with upcoming regulation, as the experiences from the sandbox can already be used to amend or adjust the companies’ respective services, products and mechanisms while the legislative process is still running. Therefore, the time for these products and services to be deployed onto the respective markets can be significantly shortened. An established and well-planed learning and knowledge sharing mechanism could then foster the achieved results and make them useful for future sandboxes and thus amplify the gained knowledge. Sandboxes therefore promise to serve as a framework for nurturing innovation, but also compliance. Regulation thus often turns here into some sort of Governance based on enhanced communication.^[32]

 

III. The ‘smart’ legal framework in practice: The AI Act

 

The most recent and significant use of sandboxes can be found in the new AI Act of the EU, which came into effect in August 2024, trying to regulate Artificial Intelligence and its usage.^[33]  In general, the EU chose to follow a horizontal regulating approach with implementing the AI Act.^[34] In this regard many artificial intelligence tools already fall under current legislation regarding several different sectors, like data protection law or competition law.^[35] This is usually not due to the peculiarities of the artificial intelligence used, but rather of varying reasons connected to other issues regarding the product or the company.

 

However, the EU legislator attempted to at least minimize negative consequences of artificial intelligence in particular before the implementation of the AI Act in a non-centralized and somewhat chaotic approach, enshrining some regulative measures in different legal initiatives like the GDPR, cf. Art. 22 or 35.^[36] Most of these regulations were of vertical nature, mostly born out of pressure to quickly react to fast-changing technologies and the legal vacuum they nurtured from.^[37] The amendments were necessary due to the lack of a general law constraining and defining the powers and limits of this technology.^[38] With the AI Act such a law now exists, crossing the threshold of regulation being predominantly reactive to being increasingly more structural and therefore preventive.^[39] Through its implementation the EU now seeks to create a comprehensive framework and ecosystem to enable citizens to nurture the benefits of artificial intelligence while simultaneously minimize its risks EU-wide.^[40]

 

The EU followed in this regard a mostly risk-based approach, meaning that it categorizes artificial intelligence systems and foundation models into different risk categories with different compliance standards according to the specific risk level the respective artificial intelligence systems falls under.^[41] Providers as well as deployers of such systems will then be obliged to perform certain duties and comply with the regulation in order to mitigate risks stemming from risky artificial intelligence.^[42] This approach does justice to the different types and areas of application of artificial intelligence, some of which have very different potential risks for society.

 

The AI Act introduces different forms of governance and regulation including complete prohibitions, the possibility of substantive fines, reporting, record keeping, documentation, transparency and human oversight obligations, but also providing among others the option to establish regulatory artificial intelligence sandboxes.^[43] Proponents of regulatory sandboxes saw this as a great opportunity for the successful implementation of this legal measure on a large scale. Thus, the EU followed other legislators who already established AI-Sandboxes in their own respective jurisdictions, for example Russia,^[44] Brazil, Norway, United Kingdom or Spain.^[45]

 

But how and how well did the EU design these sandboxes? As outlined above the specific operationalization and the actual design of a sandbox are extremely influential on its success.

 

IV. The AI Act as a Case Study

 

If regulatory sandboxes are regarded as a sub-category of experimental regulation, both forms of legislation have found their place in the AI Act in the form of a regulatory AI sandbox, which can be found in Art. 57 ff. AI Act. According to these Articles each Member State shall establish at least one AI regulatory sandbox alone or jointly with other Member States and their competent authorities, cf. Art. 57(1) AI-Act. Accordingly, Member States must either introduce such an AI regulatory sandbox themselves or participate in a sandbox established by another Member State. This applies to the extent and only if participation in the sandbox of the other Member State is comparable to the establishment of its own. In this respect, this should be of particular interest and advantage to smaller member states if their own AI sector is too small to introduce an AI regulatory sandbox. Larger member states are likely to regularly fail this restrictive condition, unless the scale of the desired sandbox is correspondingly large.

To prevent segmentation and fragmentation of regulatory sandbox regimes across the EU, the Commission is obliged under Art. 58(1) of the AI Act to adopt an implementing act in which the modalities for the establishment, development, implementation, operation and monitoring of the AI sandboxes. Art. 58 of the Act lists numerous points that must be observed by the national authorities when establishing and operating sandboxes. It is to be welcomed that the national authorities are given an appropriate amount of leeway to shape the concrete form of the sandboxes without it being too extensive. For example, the authorities can determine the length of the respective sandbox themselves, which makes sense in line with the concept of sandboxes based on individual projects, cf. Art. 58(2)(h) AI-Act.

 

As outlined above, the success of regulatory sandboxes and experimental regulation is also highly dependent on the evaluation process as it is a crucial part of conducting the sandbox and gaining important information for future regulation attempts. Here, the national competent authorities responsible for the establishment and operation of the sandboxes must send annual reports to the AI Office and the European artificial intelligence Board – two organs introduced by the AI Act in order to monitor and guarantee the success of the regulation ­– in accordance with Art. 57(16) AI Act, in which they report on the progress and results of the implementation of these sandboxes, including best practices, incidents, lessons learned and recommendations on their establishment and, where appropriate, application and possible revision of this Regulation. Depending on whether the expected and previously mentioned implementing act of the Commission further specifies these evaluation obligations, the standard of the respective assessments could even be increased accordingly.

 

Furthermore the AI-Act provides several organizational points that should guarantee the successful implementation of European Union AI sandboxes. Pursuant to Art. 57(1) AI-Act the Commission may provide assistance in the form of technical support, advice or the providing of tools for the establishment as well as the operation of such AI regulatory sandboxes. Depending on whether – and if yes on how – the support is actually given, the sandbox framework in the AI Act may turn out as a success or a failure. This of course also depends on whether the support of the Commission is needed in the first place. Since the sandboxes will probably remain national to the greatest extent and according to Art. 57(1) AI-Act might be even conducted on a regional or local level, the centralized expertise of the Commission might turn out to be unnecessary. However, when two or more member states establish and/or operate an AI sandbox together according to Art. 57(1) Para. 1, 2 AI-Act, it may be useful to obtain information from a supranational body like the Commission as it might have more supranational resources in the first place.

 

According to Art. 53(17) AI-Act the Commission must create a comprehensive interface to give stakeholders and interested parties an overview of the sandboxes and, if necessary, contact options, which should make the access to the sandboxes easier. The attempt to amplify supranational cooperation and cross-border innovation is reflected several times within the regulation, cf. Art. 57 Para. 1 or 4 AI Act, which stipulate that the sandboxes should be designed in such a way that competent authorities from other member states can also participate if needed. Also, there is the possibility of a European Union AI regulatory sandbox for the EU institutions themselves, which can be established by the European Data Protection Supervisor.

 

All the above-mentioned points promise to guarantee a successful implementation of regulatory AI-sandboxes on an EU level. However, there are also points in which the EU only partly succeeds in establishing a coherent and effective sandbox environment. To effectively attract applicants to participate in a regulatory sandbox there should be exemptions from the existing regulatory burden. However, there is no mention of this in the AI Act, at least not explicitly. The reason for this is not entirely clear. Here too, the legislator could have easily continued to pursue the risk-based approach and made the derogation from existing rules and regulation dependent on the respective risk level of the respective artificial intelligence technology. There should still be incentives for companies to participate in the sandbox, such as faster distribution of products to the European Union market. Especially with a complex technology such as artificial intelligence, it would have made sense to offer incentives to deviate from the now comprehensive legislation in order to try out new approaches and ideas.^[46]

 

Rather, a genuine "experimentation clause" should have been chosen here, which would have given the supervisory authority a certain amount of leeway to act flexibly in the application of the existing legal framework and to deviate from it accordingly if necessary.^[47] Furthermore, applicants could also be attracted by monetary incentives. Here the providers of artificial intelligence systems that fall under the AI-Act are spared administrative fines as long as they respect the sandbox plan and the terms and conditions for their participation and followed the guidance given by the national competent authority, Art. 57(12) AI-Act.

 

It also has to be noted that AI Sandboxes introduced by the AI-Act do not play a too prominent role in regard to the rest of the regulation. Due to the partly extensively broad wording and categorization of certain artificial intelligence systems, there remains the fear of overregulating the technology;^[48] this could have been easily mitigated or even prevented if the sandbox would have been given a more central role in the legislation as this is exactly one of the main advantages of regulatory sandboxes: balancing regulating and innovation.

 

It can be said that by establishing AI regulatory sandboxes, the AI Act has taken an important and necessary step towards the flexible and innovative regulation of artificial intelligence, perhaps the most important technology of this century. The EU has indeed successfully fulfilled many of the points that should be considered when establishing and designing regulatory sandboxes. However, some other points, in particular the lack of flexibility to deviate from the provisions of the AI Act within the sandbox, were implemented rather inadequately by the EU. This is particularly unfortunate because, due to the importance and significance of the AI Act, a full-fledged regulatory sandbox would have sent an important signal to stakeholders, companies and citizens: namely that the EU is an innovative and progressive legislator. After analyzing the sandboxes in the AI Act, this can only be partially attributed to the EU.

 

Since artificial intelligence would have been an excellent application example for effective sandboxes outside of fintech ­– both in terms of the concept of the technology itself and the importance and potential market capitalization of AI-driven business models – it is particularly unfortunate that the EU has only created a partially promising sandbox here.

 

IV. Conclusion

 

While the two discussed forms of smart legislation – experimental regulation and regulatory sandboxes – offer several advantages, they also have flaws that can be mitigated under the right conditions. These approaches introduce innovation and empiricism to a traditionally bureaucratic and slow legislative process, with the aim of rationalizing lawmaking, especially in technocratic fields. However, politics is not always purely rational and should account for emotions and ideologies, as long as they avoid extremism. While these legislative models can be useful in managing disruptive technologies and national emergencies, their effectiveness depends on careful design by legislators. As seen in the AI Act case study, success is not guaranteed, but with continued use, these approaches are likely to improve, benefiting both society and the legislative process. It was expected that the goal of super-alignment could not have been reached by the AI-Act and its use of experimental regulation. However, the EU did take a big step towards a modern approach of law-making and an alignment as far reaching as possible when it comes to the AI-Act. Whether this approach will be successful in regulating such an important and influential technology as AI remains to be seen.

 

 

---

[1] Pistor, The Code of Capital, passim.

[2] Sabeel Rahman, Artificial Sovereigns: A quasi-constitutional Moment for Tech?, https://lpeproject.org/blog/a-quasi-constitutional-moment-for-tech/.

[3] Bradford, who coined the term in her article, The Brussels Effect, Northwestern University School of Law 2012, Vol. 107, No. 1; see also Siegmann et al., The Brussels Effect of AI Regulations, https://www.governance.ai/research-paper/brussels-effect-ai for further insights on the Brussels Effect in regard to the AI Act. However, it should be noted that the impact of the Brussels effect in the case of the AI Act is questioned by some, as artificial intelligence itself is often already (co-)regulated by other laws that actually focus on data security or intellectual property, for example, cf. Engler, The EU AI Act will have global impact, but a limited Brussels Effect, https://www.brookings.edu/articles/the-eu-ai-act-will-have-global-impact-but-a-limited-brussels-effect/.

[4] Markovits, Are we prisoners of technological fate?, https://lpeproject.org/blog/are-we-prisoners-of-technological-fate/.

[5] Cf. Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 17.

[6] Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 18.

[7] Smuha, Internet Policy Review 2021, Vol. 10, Iss. 3.

[8] Burkhard, Was ist Superalignment und warum ist es wichtig?, https://medium.com/@pratheekburkhard/was-ist-superalignment-und-warum-ist-es-wichtig-52b94fe37e22.

[9] Heaven, Now we know what OpenAI’s superalignment team has been up to, https://www.technologyreview.com/2023/12/14/1085344/openai-super-alignment-rogue-agi-gpt-4/; Burkhard, Was ist Superalignment und warum ist es wichtig?, https://medium.com/@pratheekburkhard/was-ist-superalignment-und-warum-ist-es-wichtig-52b94fe37e22.

[10] Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 5.

[11] Van Gestel/Van Dijck, European Public Law 2011, 539.

[12] Cf. Ranchordas, The whys and woes of experimental legislation, p. 419, 420.

[13] Ranchordas, Sunset Clauses and Experimental Regulations: Blessing or Curse for Legal Certainty?, p. 29.

[14] Ranchordas, Experimental Regulations for AI: Sandboxes for Morals and Mores, p. 94.

[15] Dutil/Williams, Canadian Public Administration 2017, p. 562, 571.

[16] Dutil/Williams, Canadian Public Administration 2017, p. 562, 571.

[17] Soeteman-Hernandez et al., NanoImpact 2021, 10031, p. 6.

[18] Cf. among others Khan, Amazon’s Antitrust Paradox, passim, in which the author outlines how the legal framework of antitrust law isn’t capable of effectively addressing the challenges Amazon’s business model established.

[19] Khan, Amazon’s Antitrust Paradox, passim.

[20] Ahern, European Business Organization Law Review 2021, p. 395, 405; Nabil, Reforming the European Union’s Proposed AI Regulatory Sandbox, https://www.internationalaffairs.org.au/australianoutlook/reforming-the-european-unions-proposed-ai-regulatory-sandbox/#:~:text=More%20specifically%2C%20an%20AI%20sandbox,for%20compliance%20with%20relevant%20laws.

[21] Ranchordas, Experimental Regulations and Regulatory Sandboxes – Law Without Order?, p. 2.

[22] Cf. Ringe, Why we need a regulatory sandbox for AI, https://blogs.law.ox.ac.uk/oblb/blog-post/2023/05/why-we-need-regulatory-sandbox-ai.

[23] https://www.consilium.europa.eu/en/press/press-releases/2020/11/16/regulatory-sandboxes-and-experimentation-clauses-as-tools-for-better-regulation-council-adopts-conclusions/.

[24] Ahern, European Business Organization Law Review 2021, p. 395, 411.

[25] Ringe, Why we need a regulatory sandbox for AI, https://blogs.law.ox.ac.uk/oblb/blog-post/2023/05/why-we-need-regulatory-sandbox-ai.

[26] Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 6.

[27] Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 7.

[28] Omarova, Journal of Financial Regulation 2020, p. 78.

[29] This way the so-called Collingridge Dilemma is avoided (at least to a certain extent), cf. Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 16.

[30] Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 16.

[31] Ringe, Why we need a regulatory sandbox for AI, https://blogs.law.ox.ac.uk/oblb/blog-post/2023/05/why-we-need-regulatory-sandbox-ai.

[32] Ranchordas/Vinci, Regulatory Sandboxes and innovation-friendly Regulation, p. 11.

[33] Friedl/Gil Gasiola, Examining the EU’s Artificial Intelligence Act, https://verfassungsblog.de/examining-the-eus-artificial-intelligence-act/; Smuha, Internet Policy Review 2021, Vol. 10, Iss. 3.

[34] https://www.psa.ac.uk/psa/news/ai-act-it-golden-standard-or-just-another-over-regulation-symphony-brussels.

[35] Cf. Dotan, AI Regulation: A Step Forward or Ethics Washing?, https://www.spiceworks.com/tech/artificial-intelligence/guest-article/ai-regulation-and-ethics/.

[36] See also Art. 9 GDPR, which prohibits the processing of special categories of personal data under certain conditions and through this for instance the use of biometric categorization systems; Ranchordas, MORALS + MACHINES 1/2021, p. 89, 90.

[37] Chen, China sets restrictions on generative AI, but leaves room for innovation, https://thechinaproject.com/2023/07/21/chinas-new-regulations-on-generative-ai-sets-restrictions-but-leaves-room-for-innovation/.

[38] Chen, China sets restrictions on generative AI, but leaves room for innovation, https://thechinaproject.com/2023/07/21/chinas-new-regulations-on-generative-ai-sets-restrictions-but-leaves-room-for-innovation/.

[39] Chen, China sets restrictions on generative AI, but leaves room for innovation, https://thechinaproject.com/2023/07/21/chinas-new-regulations-on-generative-ai-sets-restrictions-but-leaves-room-for-innovation/; Pasquale, The second wave of alghoritmic accountability, https://lpeproject.org/blog/the-second-wave-of-algorithmic-accountability/.

[40] Ranchordas, MORALS + MACHINES 1/2021, p. 90.

[41] Fraser/Villarino, European Journal of Risk Regulation 2023, p. 1, 4; Friedl/Gil Gasiola, Examining the EU’s Artificial Intelligence Act, https://verfassungsblog.de/examining-the-eus-artificial-intelligence-act/.

[42] Friedl/Gil Gasiola, Examining the EU’s Artificial Intelligence Act, https://verfassungsblog.de/examining-the-eus-artificial-intelligence-act/.

[43] Friedl/Gil Gasiola, Examining the EU’s Artificial Intelligence Act, https://verfassungsblog.de/examining-the-eus-artificial-intelligence-act/; MacCarthy/Propp, Machines learn that Brussels writes the rules: The EU’s new AI regulation, https://www.brookings.edu/articles/machines-learn-that-brussels-writes-the-rules-the-eus-new-ai-regulation/; Smuha, Internet Policy Review 2021, Vol. 10, Iss. 3.

[44] Russia even introduced a sandbox, which also covered Artificial Intelligence applications already back in 2021, cf. Ranchordas, MORALS + MACHINES 1/2021, p. 95.

[45] Nabil, Reforming the European Union’s Proposed AI Regulatory Sandbox, https://www.internationalaffairs.org.au/australianoutlook/reforming-the-european-unions-proposed-ai-regulatory-sandbox/#:~:text=More%20specifically%2C%20an%20AI%20sandbox,for%20compliance%20with%20relevant%20laws.

[46] Ringe, Why we need a regulatory sandbox for AI, https://blogs.law.ox.ac.uk/oblb/blog-post/2023/05/why-we-need-regulatory-sandbox-ai.

[47] Ringe, Why we need a regulatory sandbox for AI, https://blogs.law.ox.ac.uk/oblb/blog-post/2023/05/why-we-need-regulatory-sandbox-ai.

[48] Cf. Streitbörger, Kann der was? Ein kritischer Blick auf die letzten Änderungsvorschläge für den AI-Act, https://blog.ai-laws.org/kann-der-was-bewertung-der-jungsten-anderungsvorschlage-fur-den-ai-act-der-eu/.