AI risk classification decides how much scrutiny each AI system gets before and after it goes live. The natural starting point is a list of models with a grade against each, but under the EU AI Act the risk sits with what a model is used for, so a scheme built around models spends its effort in the wrong places.
Why does AI risk classification depend on the use case?
Take a model that scores transactions and customer behaviour. Used to decide who gets a loan, it falls within Annex III of the AI Act, the Act’s list of high-risk uses, which covers systems that “evaluate the creditworthiness of natural persons or establish their credit score”. Used to flag suspicious payments, the same type of model is expressly carved out, because the same entry excludes “AI systems used for the purpose of detecting financial fraud”.
The underlying model can be much the same in both cases, so a scheme that grades the model once will either over-govern the fraud deployment or under-govern the lending one.
A fraud label does not settle the question either. The Dutch Tax Administration used nationality as an indicator in a system that automatically marked childcare benefit applications as risky, and the Dutch data protection authority fined it €2.75 million for discriminatory and unlawful processing.
The system was framed as fraud screening, but it decided which applications were treated as suspect, and Annex III covers systems used to assess eligibility for public benefits or to reduce and reclaim them.
How does the EU AI Act decide what is high-risk?
Article 6 of the Act gives two routes into the high-risk category. The first covers products, and safety components of products, that fall under the EU safety legislation in Annex I and need an independent conformity assessment. The second covers uses in the eight Annex III areas, which include critical infrastructure, employment, access to essential services, law enforcement and the administration of justice.
The Commission’s draft guidelines on applying Article 6 show how regulators are likely to read the test. Gowling WLG’s analysis of the draft picks out several points that matter for engineering teams:
- Intended purpose drives the outcome: Providers have to set out what each system is for, including every use of a multi-purpose system.
- Disclaimers do not help: A narrow description or a disclaimer cannot keep a system out of high-risk where high-risk use is realistic.
- Connected systems can count as one: Interconnected AI systems may be treated as a single system, so splitting a pipeline into modules will not avoid classification.
- Agent setups are assessed end to end: Multi-agent or orchestrated environments are judged as a whole.
Can a system on the Annex III list be classed as not high-risk?
Article 6(3) offers a narrow route out. It lets a provider conclude that a listed system is not high-risk where it does not pose a significant risk of harm, for example because it only performs a narrow procedural task or a preparatory step ahead of a human assessment. The exception never applies to a system that profiles people, meaning it evaluates or predicts personal aspects such as someone’s finances or work performance.
The Commission had proposed dropping the requirement to register these systems in the EU database, but under the final Digital Omnibus, the registration obligation remains, with a simplified set of information to file. The same regulation pushed back the dates when high-risk obligations apply, as covered in our post on governance tools for AI model lifecycle management.
The provider must also document the assessment before the system goes live and hand it over if a national authority asks, so a not-high-risk conclusion needs an owner who revisits it when the use changes.
What do other frameworks add to risk classification?
The AI Act answers a yes-or-no question, while the systems below the high-risk line range from a spam filter to a pricing engine.
Banking has one of the most established practices of tiering models by risk. The Federal Reserve’s SR 26-2 replaces SR 11-7, the long-standing model risk guidance for US banks, with an approach “tailored to a banking organization’s model risk profile”. The OCC’s announcement of the same change states that “generative AI and agentic AI models are novel and rapidly evolving” and so sit outside the guidance, which leaves organisations to build their own tiers for those systems.
ISO/IEC 42005 covers part of that gap. It sets out how to assess an AI system’s impact on people across its lifecycle and is designed to sit alongside ISO/IEC 42001. It does not hand you tiers, but it gives you a defensible process for gathering the evidence behind a tier decision.
How do you turn a risk classification into controls?
A tier is only useful if something different happens at each level. A workable scheme usually scores a short list of factors:
- Severity of harm: A wrong answer that costs someone a job or a loan weighs more than one that misfiles an email.
- Human review: A system whose output someone checks before it takes effect carries less risk than one whose decisions go straight through.
- Reach: A decision touching thousands of people a day warrants more scrutiny than one affecting a handful.
- Data sensitivity: Personal data raises the stakes, and special category data such as health records raises them further.
- Autonomy: A system that can act on its own needs closer attention, especially an agent that can trigger actions without a person in between.
Each tier then needs controls an auditor can check. At the top, that usually means independent validation before release, continuous monitoring for drift and fairness, sign-off by a named owner and a governance board, and full decision logs with data lineage. Lower tiers can rely on peer review and scheduled checks, with team-lead approval and summary documentation kept on file.
Classification works best when it starts at problem framing, before any training, and is confirmed at validation, so the evidence a high tier demands is captured as the work happens. The lineage side of that is covered in our post on data lineage in machine learning.
The tier also needs revisiting whenever the use changes, as well as when the model is retrained. A transaction-scoring model moved from fraud alerts into lending decisions changes class without a single weight changing, and no retraining trigger would notice.
Monitoring is where tiering pays for itself, because watching every model at full intensity is unaffordable. The warning and critical thresholds in our post on model drift work best set per tier, with the tightest limits on the systems where a wrong answer costs most.
From a model inventory to a use-case inventory
AI risk classification sits at the start of the lifecycle described in our guide to AI model lifecycle management, and every later stage, from validation to retirement, inherits its answer.
Neurotechnology Cloud builds and hosts custom models on dedicated EU infrastructure through its applied AI solutions practice, with monitoring and lineage set to match each system’s risk tier.
If you are working out how to classify the AI systems you already run, get in touch.