What will the EU AI Act change in the ML landscape?
Find out how this law will reshape the world of ML, which obligations it will place on companies that build and use artificial intelligence, and what chances and dangers come with it.
In recent months, public interest in artificial intelligence has grown at a record pace, largely thanks to free, user-friendly AI-powered tools like Midjourney or ChatGPT. Because these systems are so accessible and their users so endlessly inventive, people have uncovered many useful and intriguing properties of the machine learning models behind them, for example prompt templates for OpenAI’s ChatGPT model that coax out particular desired responses, yet the same experience has also shown that these tools come with risks. As with any tool, its capabilities can serve good purposes just as well as bad ones.
The potential for misuse, combined with the rising popularity of these systems and the rapid pace of research in the field, has made governments worried about where further progress might lead. To shield citizens from the harmful consequences of AI development, the EU has drafted the AI Act, which aims to set up a legal framework that will steer how artificial intelligence evolves and limit the risks associated with it. Along similar lines, experts published an open letter calling for a six-month halt to AI research so that rules and principles could be established.
The EU is far from the only political body bringing in regulations of this kind. The U.S. has drawn up its own set of rules, the “Blueprint for an AI Bill of Rights”, and the U.K. has its “National AI Strategy.” Each of these countries is trying, in its own manner, to reduce the risks posed by AI and to push for ethical AI development. There is currently a race to regulate AI, and it is likely to end with one regulatory standard accepted by a large share of the world’s countries.
General assumptions
In its rules, the European Union concentrated on reducing the risks faced by users of AI-based systems by imposing specific obligations on the companies that build such systems and those that use them. Its main goal was to make sure AI technologies are safe, transparent, non-discriminatory, traceable and environmentally friendly. It applied the principle of proportional risk, setting out several risk levels for AI systems along with matching rules. The regulations were designed to be technology-agnostic and future-proof.
Affected Businesses
The Act distinguishes four risk levels for AI systems: unaccepted, high, Generative AI, and limited.
Unaccepted Risk
This category covers every AI system used for biometrics in public spaces, for instance facial recognition software (law enforcement being an exception, subject to separate restrictions), social scoring systems, and systems that manipulate human behavior, particularly targeting vulnerable groups like children or people with disabilities. Such AI systems will be prohibited in the EU.
High-Risk
This covers AI systems used in products listed in DIRECTIVE 2001/95/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 3 December 2001 on general product safety, and belonging to one of these areas:
- biometrics of humans
- critical infrastructure
- education
- employment
- basic public services and healthcare
- law enforcement
- migration and border control
- interpreting and enforcing the law
Organizations building AI for these areas will have to register in the EU database. Such systems will undergo human assessment and oversight across their entire life cycle. In addition, anyone using these AI systems must be able to get an explanation of the system’s decisions, enabling them to oversee how it works and spot anomalies or faulty behavior. Consequently, a component implementing explainable AI will become an inherent part of future high-risk AI systems. Users will also have to be informed about what the system can and cannot do.
Generative AI
Generative systems will need to fulfil transparency requirements, in particular:
- making clear that the content was produced by AI
- protecting the model from producing illegal content
- releasing a summary of the training data
Limited Risk
All remaining AI systems fall into the limited-risk category. Artificial intelligence systems of limited risk should satisfy basic transparency requirements so that users can make informed choices. People should know when they are dealing with artificial intelligence.
Opportunities
Globally, we are now in the process of writing law that governs AI. This gives us the chance to shape a regulation that protects society and individual citizens from the harmful effects of AI systems without heavily limiting how artificial intelligence is developed and used in business.
The explainability and transparency requirements will undoubtedly push the research community to focus more on methods for explaining and understanding AI systems. Over time, this will lead to step-by-step improvements in AI models.
Having to protect generative models from producing illegal content will strongly motivate research in this field. I anticipate new methods appearing that will make AI systems safer for the people who use them.
Risks
As I dug into the EU AI Act and what it means for the obligations of companies building artificial intelligence systems, I kept asking myself how the expectations of European officials would map onto technical requirements for those systems. As it turns out, others had the same question. Balint Gyevnar et. all (2023) shared comparable reflections in a paper called “Get Your Act Together: A Comparative View on Transparency in the AI Act and Technology”. Its takeaways are that the regulations in their current form are ambiguous, that officials and engineers understand key terms differently, that the requirements in the regulations do not translate directly into system requirements, and that AI system developers’ proprietary rights may be infringed.
Civil servants and engineers speak different languages. They use the same words, but with different meanings. Terms such as interpretability, explainability, transparency and model output are good examples. These terms must be adjusted and defined precisely so that the requirements can be understood unambiguously. For officials, transparency frequently means transparency geared toward regulatory compliance, whereas for engineers it means understanding how the system itself works, for example through explainable AI methods.
Will officials treat AI systems that recommend content as systems that manipulate people’s behavior? If they do, it will hit the e-commerce sector and streaming platforms hard, as they rely on such solutions to promote their products based on how customers behave. And what about advertising campaigns run by AI?
In some places the regulations are fairly vague and open to interpretation, which poses risks for companies building AI-based solutions.
On top of that, the transparency requirement may infringe proprietary rights, since it demands a description of the motivations and design choices made while building the system, which would amount to disclosing information that lets competitors replicate the solution. This is the most serious danger posed by the regulation. For a business whose competitive edge is its data, a description of the architecture and the datasets used to train the machine learning model is extremely valuable.
What’s next?
Talks with EU Council member states about the final shape of the regulation are about to start. I hope experts will be listened to when the law is amended. If not, we will have to wait for interpretations that spell out the guidelines, making it possible to turn the requirements written in the regulations into technical system requirements and freeing companies from the risk posed by vague legal wording.
Summary
The EU’s regulation in the making tries to rein in the “wild west” of AI development before it causes serious harm. The guidelines aim to protect people using AI systems, particularly in areas where decisions have a major impact on citizens’ lives. I believe the ban on using AI in areas that could threaten citizens’ security and privacy is commendable. The prohibition of social scoring and real-time biometrics in public spaces, for example, will restrict the ability to shape citizens’ behavior from the top down, as happens in the People’s Republic of China.
That said, there are also worries about unclear terminology and the absence of technical requirements for AI systems.
One thing is clear – regulation of artificial intelligence is coming. As engineers, we will need to adjust to it and build systems that are compliant with the law. Judging by the current draft, I do not expect the new rules to bring anything to a standstill. They need some refinement, particularly regarding the description of design decisions and datasets, but they are no existential threat to the industry’s future growth.