Roger Clarke's Web-Site© Xamax Consultancy Pty Ltd, 1995-2026 |
|
|||||
| HOME | eBusiness |
Information Infrastructure |
Dataveillance & Privacy |
Identity Matters | Other Topics | |
| What's New |
Waltzing Matilda | Advanced Site-Search | ||||
This is an Annex to a paper entitled
'An
Evaluation of the EU Artificial Intelligence Act
against a Normative
Framework for Regulatory Regimes'
Version of 21 January 2026
? Xamax Consultancy Pty Ltd, 2025-26
Available under an AEShareNet
licence or a Creative
Commons
licence.
This document is at http://rogerclarke.com/EC/RRE-AIA-Annex.html
A Framework for the design and evaluation of regulatory schemes is applied to the European Union's Artificial Intelligence Act (AIA). The AIA identifies five categories of 'AI System', and subjects providers, and to some extent deployers, of four of the categories to some obligations. Each of the four regulatory regimes is considered in light of the Framework, and conclusions are drawn.
A Framework for the design and evaluation of regulatory schemes, which was progressively developed over a 30-year period, appears in the leading technology law journal, Computer Law & Security Review (Clarke 2026a).
Successive versions of the Framework have been applied to the evaluation of regulatory schemes in many contexts. This Annex applies it to the EU's Artificial Intelligence Act (AIA). An exposition and analysis of the AIA is in the paper to which the present document is an Annex (Clarke 2026b).
Section 2 below provides a brief overview of the Framework. Section 3 then applies the Framework, in a series of sub-sections. Section 4 presents the evaluation's conclusions.
Note: This section is not materially different from the overview in section 4 of the paper to which this document is an Annex (Clarke 2026b).
The perspective that the author brings to this matter is that of an information systems professional and researcher much of whose consultancy career has had as its focus strategic and policy aspects of transformative and disruptive information technologies. This has involved assessment of regulatory regimes applicable to many technologies and their applications, as diverse as data matching, drones, wearable cameras, 'big data' analytics, and electronic markets of many kinds, including those delivered by tech platforms as diverse as Airbnb 'homestay' letting and Uber ride-sharing. From this work, conducted over a 30-year period, a framework has been progressively developed, which is consolidated in a companion paper, Clarke (2026a). Table 1 provides an overview of the elements of the Regulatory Design and Evaluation Famework, supported by brief descriptions below.
A regulatory regime can be applied to various categories of objects. In the case of the AIA, the primary focus is artefacts that embody what the AIA refers to as AI technology ("an AI system"), but the regulatory measures apply differentially depending on the particular purposes they are applied to and the risk-level to which the Act assigns them, and the obligations arising from the regulatory regime are assigned to particular enterprise-categories (primarily "providers", but to some extent also "deployers").
The purpose of the Framework is to support the evaluation of the efficacy of regulatory regimes generally. Statutory, code and/or case law is likely to at least influence the process of exercising control over behaviour, and in many cases there will be a substantial body of relevant formal law. The focus of the evaluation framework is not the law, however, but the efficacy of the regulatory regime as a whole, including rather than specifically the law. The term 'efficacy' is used as an overall term to encompass all of the desirable elements, including effectiveness, efficiency, flexibility and adaptability.
The mechanisms are ordered into a hierarchical model of seven layers, with formal law (featuring 'government' and 'compliance') making up the top two, self-regulation (using the catchwords 'governance', 'safeguards' and 'mitigation') the middle three, and systemic governance (comprising infrastructural and natural regulatory features) the foundational two. Three generic entities are distinguished: Regulators, Regulatees and Beneficiaries, with a more detailed model of the players involved presented in Clarke (2026a, Figure 3).
An Evaluation Template is provided, which reflects the set of 16 criteria in Clarke (2026a, Table 2). The evaluation process involves assessing the particular regulatory regime's delivery against each of the criteria, assigning two scores: one a simple ternary indicator 'Yes, Some, No', and the other a subjective score on the scale 0-5 for 6 Process factors and 0-7 for 10 Product and Outcome factors, giving an overall score out of 100. Any endeavour to deliver an objective score would be, by its nature, futile. Rather, the contribution of the framework is to provide structure to the assessment of regime performance against defined criteria. Assessors can refine the analysis, or conduct their own evaluations against the criteria, or against an enhanced or alternative version of the criterion-set. The scoring process is a means of encouraging assessors to focus on the criteria and the extent to which the particular regime does and does not satisfy them, and then to generate an indicative, but inevitably contestable, overall score.
The following section applies the Framework to the EU's Artificial Intelligence Act.
Assessments were performed, by reference to the ERvaluation Framework outlined in the previous section, on each of the three separate regulatory regimes established by the AIA. The provisions relating to the fourth relevant category in the AIA, concerned with GPAI models, after review, were found to not reach a sufficient level to justify review as a regulatory regime.
The subjective scoring against each of the 16 criteria is conducted by reference to the outlines provided in section 3 of Clarke (2026a) and further sources in the refereed literature. Some of those sources were published during the AIA's gestation period 2021-24 and hence need some care in their use because of the considerable changes in the draft statute arising from very active lobbying by powerful stakeholders including Ministers and government agencies at both EU and member-nation levels, industry associations and major AI providers.
Some aspects of the AIA are common to the regimes, in particular the definitions discussed in sections 2.2-2.5 of the main paper (Clarke 2026b), and aspects of the enforcement measures in section 3.5. Some of the evaluation is abstract and generic. That approach was complemented by keeping in focus a number of specific test-applications, listed in Table 2. These were not devised in manner consistent with dispassionate scholarship. They describe real or proposed real-world uses of AI, but they have been conceived as a means of illustrating the effects of the many loopholes designed into the AIA. That approach is consistent with the positioning of this work as a critique, and as a counterbalance to the widespread enthusiasm for both AI and the AIA. It is open to others to repeat the evaluation on the basis of adapted or alternative criteria and alternative test-sets.
An outline of this regime, which many would have anticipated to be the strongest of the four, is in section 3.1 of the main paper, Clarke (2026b). In principle, the eight items in Appendix 1 of the main paper are all prohibited. In practice, there are a great many exclusions, and the exclusions are sufficiently vague that organisations can readily design around the nominal prohibition.
For example:
At an early stage, Veale & Borgesius (2021) identified several serious concerns, including:
" ... a range of problematic loopholes [such that a] cynic might feel the Commission is more interested in prohibitions/rhetorical value than practical effect" (p.99)
" ... the prohibitions concerning manipulative AI systems may have little practical impact [and] the EU legislator has some work to do to make [the social scoring] provision clearly applicable to anything" (p.100)
By the time the Regulation was passed, the shortfalls had increased even further (Wachter 2024, p.679-680):
After a thirty-six-hour negotiation marathon, a compromise was reached ... The final list of prohibited systems leaves much to be desired. This would have been a good opportunity to ban, for instance, biometric categorization systems, 'real-time' and ex-post remote biometric identification in public spaces, predictive policing, and emotion recognition in high-risk areas
In Appendix 1A, features of this regime are outlined and scored. The cluster of criteria associated with Process achieved the barest of Passes, and those associated with Product and Outcomes were each scored about 30%, giving an overall Failing grade of 37%. Even allowing for the vagaries of subjective scoring, given that the AIA's prohibition measures are targeted at what were assessed by the EC to be the highest-risk category of "AI systems", it is difficult to see how this could be regarded as an efficacious regulatory regime.
An outline of this regime is in section 3.2 of the main paper (Clarke 2026b). It applies to AI systems that are a safety product or a safety component of a product, within the meanings of 11 existing EU-wide laws, together with 9 EU Directives that are subject to somewhat varying implementations within individual EU nation-states. The evaluation of the efficacy of that provision requires very substantial familiarity with many laws of many countries, applied in many contexts. It is not attempted here.
The regime also applies to eight categories of "areas [of application of] AI systems referred to in Annex III" (Art.6-2), reproduced in Appendix 2 of the primary paper, and subject to generic and specific exceptions (Arts.8-39) and to an overriding criterion (Art.6.3). The investment needed to establish whether an entity is subject to these provisions, and the effort and cost involved if the impositions apply, are so substantial that it can be confidently expected that a substantial industry segment of advisers well-versed in the minutiae will rapidly emerge to ply their trade, and that the exceptions will be very well exercised.
The ease of avoiding the requirements is illustrated as follows:
A central concern drawn to attention in section 3.2 of the main paper (Clarke 2026b) is the absence of any obligations on deployers of high-risk AI systems to provide explanations for inferences, decisions and actions unfavourable to people's interests, and of any obligations on providers to enable deployers to do so. See Hacker & Passoth (2022) for a typology of explanations. If it were necessary for a deployer of an AI system to present a rationale for a contested inference, decision or action to a review by the deployer, a tribunal or a court, then those AI systems that are materially dysfunctional, illegally biased or fraudulent would be quickly exposed, and the problems addressed. The absence of that necessary pre-condition for accountability, alone, shows the scheme to be seriously inadequate.
Further, in Veale & Borgesius (2021), it was noted that:
... a leaked version of the Draft AI Act required providers to specify organisational measures ... [but] the final Proposal instead emphasises the 'user's discretion ... Statements about the need for 'competence, training and authority' only make the recitals (p.104) For most standalone high-risk systems (and eventually, all such systems), providers can mark the systems as in conformity using only self-assessment" (p.106)
In Appendix 1B, features of this regime are outlined and scored. The cluster of criteria associated with Process was scored as for the previous regime at a bare Pass, but those associated with Product and Outcomes each achieve below 15%, giving an overall grade of a Very Low Fail at 30%. This cannot be regarded as an efficacious regulatory regime.
An outline of this regime is in section 3.3 of the main paper (Clarke 2026b). It imposes very limited, transparency-only obligations on providers and deployers of five categories of "AI systems".
The first instance, relating to systems that "interact with natural persons", probably requires no action at all. The second requirement is that providers of synthetic content mark it as such (with broad exceptions). The third requirement is that deployers of "emotion recognition systems" and "biometric categorisation systems" declare to people subject to them that that's what they are. The fourth requires deployers of 'deep fake' image, audio or video content disclose that the content has been artificially generated or manipulated. The fifth creates a similarly limited obligation in relation to generated text.
Both the coverage and the efficacy of these provisions appear likely to be very low. Considering the remaining Test-Applications in Table 2:
In Appendix 1C, features of this regime are outlined and scored. All three clusters of criteria score dismally low, giving an overall grade of a Very Low Fail at 24%. This is so inadequate that it can barely be regarded as a regulatory regime.
All "AI systems" that escape the prohibited, high-risk and limited-risk categories in the preceding three sections are not subject to any provisions of the AIA. This remaining category is termed 'minimal risk' in the 'High-level summary' (EU 2024). It appears that Minimal-Risk will be by far the largest category.
Of the 12 Test-Applications outlined in Table 2, it appears that 8 escape completely from any obligations under the AIA. Test 11 (diagnosis) is among them, although it may be subject to generic regulation relating to medical devices. Tests 9 and 10 are subject to trivial transparency obligations, without any safeguards, mitigation measures or avenues for recourse in the event of harm being done. The highly-intrusive Test-Application 7, biometrics and pseudo-scientific lie-detection technology (APA 2004) at national borders, is, on the other hand, included within the high-risk category, with exceptions for transparency requirements.
On the one hand, the Test-Applications can be criticised as being contrived to exaggerate the inadequacies of the regime. On the other hand, they demonstrate the ease of avoiding the need to comply with even these limited requirements, and hence the attractiveness to providers and deployers of ignoring or merely paying lip-service to the requirements.
An outline of this regime is in section 3.4 of the main paper (Clarke 2026b). The first requirement, a mere transparency obligation on providers in relation to "general-purpose AI systems, generating synthetic audio, image, video or text content" (Art.50-2) was outlined in section 3.3 above.
Some limited substantive obligations are imposed on all "GPAI model providers", but only in relation to documentation for provision to, and guidance of, downstream providers and deployers (Art.53). Additional substantive obligations are imposed on all "providers of general-purpose AI models with systemic risk" (Art.55), but these are at most minimum requirements of good business practice. In short, it is very difficult to regard these provisions as being even a very light-handed regulatory regime, and hence the score-sheet has not been applied to them.
Unlike the previous three regimes, it could be argued that the maturity-level of the GenAI field is low enough to justify a very preliminary, 'watch and be prepared to act further' approach to the regulation of upstream model providers. On the other hand, Wachter argues that what was nominally justification was actually entirely an exercise in market and institutional power (Wachter 2024, p.694-695):
Regulation of GAI was another big sticking point during the negotiations that almost caused the AIA not to pass. Even though political agreement was reached in October 2023, Germany, Italy, and France started a coordinated effort in November and December 2023 to remove most provisions on GAI. The three nations even threatened to vote against the whole Act if these provisions were left unchanged
Similar extortionary approaches were used by national governments against the European Parliament to achieve very wide exemptions not only for military and national security matters, but also for law enforcement agencies (Bertuzzi 2023). Those agencies are heavily dependent on large, secretive corporate providers of software and services, and those corporations migrate the techniques and capabilities developed for their military, national security and law enforcement clients into the general market. In short, the almost complete absence of any AI regulatory regime in those sectors guarantees that such boundaries as are created by the AIA will be tested, dented, bruised and circumvented.
The evaluation reaches the following conclusions:
Identify the objectives, the object subject to regulation, the regulatory mechanisms that make up the overall regime, the extent of exceptions and exemptions, the key players with particular emphasis on the Regulator, Regulatees and Beneficiaries, the Principles and Rules that apply to Regulatees, and the extent to which the Rules have been articulated through co-regulatory processes.
In col.A, insert Yes, Some or No (coverage); In col.C, insert a subjective evaluation of coverage.
Identify the objectives, the object subject to regulation, the regulatory mechanisms that make up the overall regime, the extent of exceptions and exemptions, the key players with particular emphasis on the Regulator, Regulatees and Beneficiaries, the Principles and Rules that apply to Regulatees, and the extent to which the Rules have been articulated through co-regulatory processes.
In col.A, insert Yes, Some or No (coverage); In col.C, insert a subjective evaluation of coverage.
Identify the objectives, the object subject to regulation, the regulatory mechanisms that make up the overall regime, the extent of exceptions and exemptions, the key players with particular emphasis on the Regulator, Regulatees and Beneficiaries, the Principles and Rules that apply to Regulatees, and the extent to which the Rules have been articulated through co-regulatory processes.
In col.A, insert Yes, Some or No (coverage); In col.C, insert a subjective evaluation of coverage.
Barkane I. (2022) 'Questioning the EU proposal for an Artificial Intelligence Act: The need for prohibitions and a stricter approach to biometric surveillance' Information Polity 27 (2022) 147?162, at https://journals.sagepub.com/doi/pdf/10.3233/IP-211524
Bertuzzi L. (2023) 'European Union squares the circle on the world's first AI rulebook' Euractiv, 9 Dec 2023, at https://www.euractiv.com/section/tech/news/european-union-squares-the-circle-on-the-worlds-first-ai-rulebook/
Clarke R. (2026a) 'Regulatory Regimes for Disruptive IT: A Framework for Their Design and Evaluation' Computer Law & Security Review 60 (April 2026) 106231, PrePrint at http://rogerclarke.com/EC/FRR.html
Clarke R. (2026b) 'An Evaluation of the EU Artificial Intelligence Act against a Normative Framework for Regulatory Regimes' Xamax Consultancy Pty Ltd, 21 January 2026
Hacker P. & Passoth J.-H. (2022) 'Varieties of AI Explanations Under the Law. From the GDPR to the AIA, and Beyond' Chapter in Holzinger A. et al. (eds) 'xxAI - Beyond Explainable AI' Springer Nature, 2022, pp 343?373, at https://link.springer.com/chapter/10.1007/978-3-031-04083-2_17
van Kolfschooten H. (2022) 'EU Regulation of Artificial Intelligence: Challenges for PatientsÅf Rights' Common Market Law, 59,1 (2022) 81-112, at https://www.researchgate.net/profile/Hannah-Van-Kolfschooten/publication/367818376_EU_Regulation_of_Artificial_Intelligence_Challenges_for_Patients'_Rights/links/66e421ccb1606e24c22779b9/EU-Regulation-of-Artificial-Intelligence-Challenges-for-Patients-Rights.pdf
Veale M. & Borgesius F.Z. (2021) 'Demystifying the Draft EU Artificial Intelligence Act' Computer Law Review International 22,4 (2021) 97-112, at https://arxiv.org/pdf/2107.03721
Wachter S. (2024) 'Limitations and Loopholes in the EU AI Act and AI Liability Directives: What This Means for the European Union, the United States, and Beyond' Yale Journal of Law & Technology 26,3 (2024) 671-718, at https://yjolt.org/sites/default/files/wachter_26yalejltech671.pdf
Roger Clarke is Principal of Xamax Consultancy Pty Ltd, Canberra. He is also a Visiting Professorial Fellow associated with UNSW Law & Justice, and a Visiting Professor in Computing in the College of Systems & Society at the Australian National University.
| Personalia |
Photographs Presentations Videos |
Access Statistics |
![]() |
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax. From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It had passed 80 million by the end of 2025. Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer, and her Dr Nurse site |
Xamax Consultancy Pty Ltd ACN: 002 360 456 78 Sidaway St, Chapman ACT 2611 AUSTRALIA Tel: +61 2 6288 6916 |
Created: 24 March 2025 - Last Amended: 21 January 2026 by Roger Clarke
This document is at www.rogerclarke.com/EC/RRE-AIA-Annex.html
Mail to Webmaster - © Xamax Consultancy Pty Ltd, 1995-2026 - Privacy Policy