An AI system's DPIA should begin before a model is connected to live customer, employee or operational data. Once an AI tool is embedded in a workflow, decisions about training data, access rights, human review and vendor accountability can become difficult to reverse. A well-run assessment gives leadership a controlled route from experimentation to deployment, with evidence that privacy risks have been considered and managed.
For organisations operating across the EU, UK and other markets, this is not simply a documentation exercise. AI introduces changing data flows, opaque model behaviour, new suppliers and potentially significant effects on individuals. The DPIA must therefore be designed as an operational control that remains useful after approval, not a static report filed at the end of a project.
When an AI systems DPIA is required
A Data Protection Impact Assessment is required under the GDPR and UK GDPR where processing is likely to result in a high risk to individuals’ rights and freedoms. AI use cases frequently meet that threshold, particularly where they involve profiling, systematic monitoring, sensitive data, vulnerable people, large-scale processing or decisions that may materially affect an individual.
Examples include an AI system that ranks job applicants, detects fraud, predicts customer behaviour, prioritises clinical cases, monitors staff activity or analyses biometric data. Generative AI tools can also create DPIA triggers where prompts, uploaded documents or retrieval sources contain personal data, even where the tool is presented internally as a productivity assistant.
Not every use of AI automatically requires a DPIA. A tightly controlled tool that summarises non-personal operational records may present a very different risk profile from a model that recommends credit decisions. The relevant question is whether the proposed processing, in its real operating context, creates a likely high risk - not whether the technology has been labelled “AI”.
A DPIA should be started early when the intended processing is uncertain. Early assessment exposes questions that procurement, technology and business teams need to resolve: what data will enter the system, what outputs will be relied upon, who can override them, and whether a supplier will reuse data for its own purposes.
Build the assessment around the real system
An effective assessment does not describe AI in abstract terms. It defines the particular system, its users, its data flows and its decision boundaries. This is where many otherwise credible DPIAs lose practical value: they assess the supplier’s product description rather than the organisation’s actual deployment.
Define the use case and decision boundary
State the business purpose in terms that can be tested. “Improving efficiency” is too broad. “Drafting responses to low-risk supplier questionnaires for review by procurement staff” establishes a clearer boundary, including the fact that a person remains responsible for the final response.
The assessment should identify whether the system generates content, makes recommendations, classifies people, predicts outcomes, detects patterns or supports decisions. It should also record what the organisation will not permit the system to do. These exclusions are often as valuable as the intended use case because they inform configuration, training and access controls.
Where AI informs decisions about individuals, document the level of human involvement honestly. A reviewer who routinely accepts a score without sufficient context may not provide meaningful oversight. Decision owners need enough authority, information and time to challenge an output, escalate concerns and depart from the model’s recommendation.
Map data from input to output
AI data mapping must go beyond the initial dataset. It should cover data collected through prompts, uploaded files, application programming interfaces, retrieval-augmented generation sources, feedback loops, logs, telemetry, model tuning and output retention.
This mapping should establish whether personal data is used to train, fine-tune or evaluate a model; whether a provider retains inputs; and whether data is transferred outside the UK, EU or other relevant jurisdictions. It should distinguish controller and processor responsibilities in the operational arrangement, while recognising that contractual labels alone do not determine how data is actually handled.
Special category data, criminal offence data and children’s data require particular scrutiny. So do datasets that appear anonymous but may be re-identified when combined with other information or queried through a model.
Assess risks that conventional DPIAs can miss
Traditional privacy risks remain relevant, including excessive collection, unlawful disclosure, weak retention controls and inadequate security. AI adds risks that need a more specific analysis of how a model behaves over time and how people may rely on it.
Four areas commonly require focused assessment:
- Purpose drift: A system introduced to support staff may later be used to evaluate performance, target customers or make eligibility decisions without a new assessment of the changed purpose.
- Inaccurate or biased outputs: Outputs may be incomplete, fabricated, disproportionately harmful to certain groups or based on historical patterns that are unsuitable for the current decision.
- Loss of transparency and contestability: Individuals and internal teams may be unable to understand what information influenced an outcome or how to challenge a result effectively.
- Data leakage and access misuse: Prompts, outputs and connected knowledge bases may expose personal data to unauthorised users, other tenants, providers or downstream systems.
Risk assessment should connect each risk to the people affected, the potential severity of harm, and the likelihood that existing controls will fail. Generic statements that a vendor is “secure” or that staff will “use AI responsibly” are not sufficient controls. The DPIA needs clear ownership, measurable safeguards and a route for testing whether they work.
Turn findings into deployable controls
The most useful DPIAs create a deployment plan. Each identified risk should have a specific control, an accountable owner, evidence of implementation and a review point. This provides project teams with a usable operating model rather than a late-stage compliance obstacle.
Controls will depend on the use case, but may include role-based access, data minimisation rules for prompts and sources, approved-use guidance, output validation, confidence thresholds, prohibited decision types, retention limits, audit logging and incident escalation procedures. High-impact use cases may need staged deployment, sample testing, performance monitoring across relevant groups and enhanced management review.
Vendor due diligence should be treated as an ongoing control, not a procurement checkpoint. The organisation should understand the provider’s data use terms, subprocessors, hosting locations, security arrangements, model update process, deletion capability and contractual support for data subject rights. If a supplier changes the model, introduces a new feature or alters how it uses customer data, the original risk assessment may no longer reflect the live service.
The DPIA should also define reassessment triggers. These may include a material change in purpose, a new data source, use of special category data, a shift from support to automated decision-making, a significant model update, a security incident or evidence of materially inaccurate outputs.
Align the DPIA with AI governance requirements
A DPIA is not a substitute for EU AI Act risk classification, conformity assessment obligations or, where applicable, a fundamental rights impact assessment. Equally, an AI governance programme does not replace GDPR accountability obligations. Organisations need both disciplines to work from the same system inventory and evidence base.
A practical AI system register can connect these requirements. For each system, it should record the owner, purpose, supplier, jurisdictions, affected groups, data categories, risk classification, DPIA status, human oversight design, applicable controls and review dates. This gives legal, privacy, security, procurement and business leaders a common view of the organisation’s AI estate.
For international organisations, consistency matters, but a single global template should not conceal local differences. The UK GDPR, EU GDPR, sector requirements, employment rules and data localisation expectations can affect the risk analysis and control design. A central framework should therefore permit local review where the deployment context or affected population changes.
Make DPIA governance a shared operating responsibility
AI systems rarely sit within one function. Legal teams may interpret applicable requirements, privacy teams may lead the DPIA and accountability process, while technical operations teams configure access, logging, integration and security controls. Business owners remain accountable for using the system within its approved purpose.
This is why a three-team model - Legal, Privacy and Technical Operations - is more effective than treating the DPIA as a form owned by a single individual. It brings regulatory interpretation, data protection discipline and implementation capability into the same delivery process. Formiti supports this model across global operations, helping organisations convert assessments into controls that can be maintained across changing systems and jurisdictions.
Senior leaders should receive concise, decision-ready reporting: which systems are live, which are awaiting assessment, what residual risks have been accepted, where supplier dependencies exist and when the next review is due. This gives boards and executives visibility without requiring them to work through technical assessment detail.
The value of an AI systems DPIA is realised when it changes how a system is designed, approved and monitored. If the assessment can show what the organisation knows, what it has decided, who owns the controls and when it will review them, it becomes a credible foundation for responsible AI deployment rather than paperwork produced after the fact.