What is AI risk?
By Nick Jang, founder of MINERVA Studio ·
The short answer
AI risk is the chance that an AI system causes harm or loss, weighed by how likely it is and how bad it would be. The harm can land on people (an unfair loan decision, a privacy leak), on the organization (a fine, a fraud loss, a bad headline) or on the wider system. The main types are wrong or made-up outputs, bias, privacy, security, lack of transparency, dependence on vendors and misuse, and managing them means listing your AI systems, rating each one and putting controls where the rating says they’re needed.
Risk, applied to AI
NIST’s AI Risk Management Framework uses the ordinary definition of risk: how likely something is to happen, combined with how much it would hurt. What makes AI different isn’t the definition. It’s that AI systems learn from data instead of following rules someone wrote, change behaviour when the data changes, and can produce fluent, confident output that’s simply wrong.
NIST groups the possible harms by who suffers them: people (their rights, safety or money), organizations (operations, reputation, finances) and ecosystems (the financial system, supply chains, the environment). A bank’s chatbot that gives bad advice can hit all three.
The main types of AI risk
| Type | What it looks like |
|---|---|
| Accuracy and reliability | The model is wrong, drifts as data changes, or a generative model invents facts (confabulation). |
| Bias and fairness | Outcomes differ unfairly by age, sex, race or postcode, often learned from historical data. |
| Privacy | Personal data used without a basis, leaked in outputs or kept longer than it should be. |
| Security | Prompt injection, data poisoning, model theft, or attackers using AI against you. |
| Transparency | Nobody can explain why the system decided what it did, to a customer or a regulator. |
| Third-party and concentration | A vendor changes or retires its model, or most of your AI runs on one or two providers. |
| Misuse and autonomy | Staff use tools for things they weren’t approved for, or an agent takes an action nobody signed off. |
| Legal and reputational | A breach of privacy, consumer or anti-discrimination law, and the story that follows. |
For generative AI specifically, NIST’s Generative AI Profile (NIST AI 600-1) lists twelve risks: CBRN information or capabilities; confabulation; dangerous, violent or hateful content; data privacy; environmental impacts; harmful bias and homogenization; human-AI configuration; information integrity; information security; intellectual property; obscene, degrading or abusive content; and value chain and component integration.
What Canadian regulators see
In September 2024 OSFI and the Financial Consumer Agency of Canada published a joint report on AI at federally regulated financial institutions. About 30% used AI in 2019 and about 50% in 2023, and 70% expected to by 2026. The report split the risks in two:
- internal: data governance, model risk and explainability, legal, ethical and reputational risk including bias, third-party risk, and operational and cyber risk;
- external: fast technological change, AI-powered cyber attacks and fraud, competitive pressure, and new credit, market and liquidity risks.
Its sharpest warning was that AI is a transverse risk. It cuts across model, cyber, privacy and vendor risk, and an institution that handles it only inside one of those silos will miss the gaps between them.
How to run an AI risk assessment
- List it. Every AI system in use, built, bought or embedded in a vendor product, including the generative tools staff use.
- Describe it. What it does, what data it uses, who relies on the output and whether it acts on its own.
- Rate it. How much harm it could do if it fails, and how uncertain or opaque it is. The rating sets how much scrutiny it gets.
- Find the specific risks. Walk the categories above against this system, not in general.
- Set controls. Testing, human review, limits on what it can do, monitoring, a fallback when it fails.
- Assign an owner and a review date, and record the decision, including the risks you’re accepting.
Frameworks help with the structure: the NIST AI RMF for the process, ISO/IEC 42001 for a management system, and for Canadian banks OSFI E-23, which from 1 May 2027 treats AI models as models with ratings, validation and monitoring.
MINERVA keeps that record: every AI system with its rating, owner, reviews and decisions, so the assessment is evidence rather than a slide. See AI governance software.
Related questions
What are the biggest risks of AI?
Wrong or invented outputs, unfair bias, privacy leaks, security weaknesses, lack of explainability, dependence on a few vendors and misuse. Which matters most depends on what the AI is used for.
What is an AI risk assessment?
A structured review of an AI system: what it does, what could go wrong, how likely and how harmful that would be, and which controls bring the risk to an acceptable level, with an owner and a review date.
What are the AI risk categories in the EU AI Act?
Unacceptable risk (banned), high risk (strict requirements), limited risk (transparency duties) and minimal risk (no specific obligations).
Is AI risk the same as model risk?
Model risk is part of it. AI risk also covers privacy, security, vendor dependence, misuse and conduct, which is why OSFI and FCAC call AI a transverse risk.
How many banks in Canada use AI?
OSFI and FCAC reported in 2024 that about 50% of federally regulated financial institutions used AI in 2023 and that 70% expected to by 2026.
Sources
- NIST — AI Risk Management Framework (AI RMF 1.0)
- NIST — AI Risk Management Framework: Generative AI Profile (NIST AI 600-1)
- OSFI and FCAC — Risk Report: AI Uses and Risks at Federally Regulated Financial Institutions (24 September 2024)
- EUR-Lex — Regulation (EU) 2024/1689, the AI Act
New answers are posted on MINERVA on LinkedIn. Follow along, or send the question you’d like answered next to minerva@minerva.studio.
A plain-language explanation, not legal advice. The regulator’s own text is the source; read it, and take advice on how it applies to your institution.