What is AI data governance?
By Nick Jang, founder of MINERVA Studio ·
The short answer
AI data governance is the set of rules and records for the data AI systems learn from, look up and produce: where each dataset came from, whether you're allowed to use it for this purpose, whether it's accurate and representative enough, who can access it, how long it's kept, and what happens to the outputs. It extends ordinary data governance in three ways: data used to train or tune a model shapes its behaviour for good, documents fed to a generative model at run time can leak or mislead, and AI makes new data (scores, summaries, synthetic records) that needs governing too. OSFI's E-23, the EU AI Act's Article 10 and ISO/IEC 42001 all set expectations for it.
What it covers
| Area | The question it answers |
|---|---|
| Provenance and lineage | Where did this data come from, and what happened to it on the way? |
| Rights and consent | May we use it to train or run this model, under privacy law, contracts and licences? |
| Quality | Is it accurate, complete, timely and consistently defined? |
| Representativeness and bias | Does it reflect the people and cases the model will meet? Are protected characteristics, or proxies for them, present? |
| Labelling | Who labelled it, by what guide, and how was quality checked? |
| Synthetic and proxy data | Where is it used, how was it generated, and what are its limits? |
| Access and security | Who can read the data, the model and the prompts and outputs? |
| Retrieval content | Which documents can a generative system draw on, and are they current and permitted for every user who asks? |
| Retention and deletion | How long are training data, prompts and outputs kept, and can a person's data be removed? |
What the rules ask
- OSFI E-23 (Canada, from 1 May 2027): data used by a model should be suitable for its intended use: accurate, relevant, representative, traceable and timely, with quality checks, controls over synthetic and proxy data, and bias managed. See OSFI E-23.
- EU AI Act, Article 10: training, validation and test data for high-risk AI systems must be subject to data governance and management practices, be relevant, sufficiently representative and as free of errors as possible, and be examined for possible biases. See the EU AI Act.
- ISO/IEC 42001: its Annex A includes controls on data for AI systems, covering acquisition, quality, provenance and preparation. See ISO 42001.
- Privacy law: in Canada, PIPEDA and provincial laws, with Quebec's Law 25 adding duties for automated decisions. Using personal data to train a model is a use that needs a lawful basis.
A practical framework
- Register each dataset an AI system uses, alongside the model inventory: owner, source, purpose, personal data, rights, refresh date.
- Approve uses, not just datasets. Data cleared for fraud detection isn't automatically cleared for marketing.
- Check quality and representativeness before training and on a schedule after, with results kept.
- Record lineage from source to feature to model version, so a finding can be traced back.
- Govern retrieval sources for generative AI: what's in the index, who can see each document, when it's refreshed.
- Control prompts and outputs: what's logged, who can read the logs, how long they're kept.
- Tie it to the model's validation: data issues become findings with owners, like any other.
MINERVA records a model's data sources and the dates of their checks on the model's own record, and makes data findings block approval like any other high-severity finding. See AI governance software.
Related questions
What is the difference between data governance and AI data governance?
AI data governance extends ordinary data governance to training and tuning data, the documents generative systems retrieve at run time, and the new data AI produces, with extra attention to rights to use data for training, representativeness, bias and lineage to the model version.
What does the EU AI Act require for data?
Article 10 requires training, validation and test data for high-risk AI systems to be governed and managed, relevant, sufficiently representative, as free of errors as possible and examined for possible biases.
Does OSFI E-23 cover data quality?
Yes. E-23 expects the data a model uses to be suitable for its intended use, with quality checks, traceability, representativeness, controls over synthetic and proxy data, and bias managed.
What is a data governance framework for AI?
A set of roles, rules and records covering each dataset an AI system uses: its source, permitted uses, quality, representativeness, access, lineage and retention, tied to the model's validation and approval.
Sources
- OSFI — Guideline E-23, Model Risk Management (2027)
- European Commission AI Act Service Desk — Article 10, Data and data governance
- ISO — ISO/IEC 42001:2023, AI management system
New answers are posted on MINERVA on LinkedIn. Follow along, or send the question you’d like answered next to minerva@minerva.studio.
A plain-language explanation, not legal advice. The regulator’s own text is the source; read it, and take advice on how it applies to your institution.