Patient Data Consent Is Broken. AI Is About to Make It Worse

Executive summary

Healthcare consent in the markets we work in is largely a paper artefact of the pre-AI era. A patient at admission signs language that authorises treatment, audit, and research consistent with applicable law. That language is statutorily defensible. It is also analytically thin. It does not describe whether the patient's data may be used to train a commercial foundation model, whether the resulting model may be sold to other institutions, or what happens to the model parameters if the patient later asks for their data back.

Regulators have not been silent. GDPR Article 9 treats health data as a special category and Article 89 carves out a research-purposes derogation ^[1]. The ICO has published structured guidance on AI and data protection ^[2]. The UK Data Protection Act 2018 supplements UK GDPR with health and social care conditions for processing ^[3]. NHS England operates the Data Security and Protection Toolkit and a national data opt-out that is in effect a layer of consent at national scale ^[4][5]. Nigeria moved from an NDPR-only regime to a primary statute, the NDPA 2023, with NITDA's AI strategy adding direction ^[7][8][9]. South Africa enforces POPIA, with health information as a defined special category ^[10]. The US relies on HIPAA, supplemented by state laws including Washington's My Health My Data Act and California's CCPA / CPRA, which now reach health data that HIPAA does not cover ^[11][12][13]. The FDA's 2024 final guidance on Predetermined Change Control Plans and the MHRA Software and AI as a Medical Device roadmap address how AI systems change after release ^[14][15]. The WHO has published ethics guidance for AI in health, with an extension in 2024 specifically on large multi-modal models ^[16][17].

What is missing, in our reading, is not the statutory frame. It is the connection between that frame and the consent paperwork that hospitals actually use. Consent language assumes a research project with edges. AI training has no edges in the same way. Models are reused, fine-tuned, sold, and deployed in places the patient never visited. We propose a three-layer consent model as a published recommendation, drawing on Effy Vayena's dynamic consent work, Wellcome's Understanding Patient Data evidence on commercial access, and the Ada Lovelace Institute's algorithmic impact assessment case study with the NHS ^[18][20][21]. We are publishing the proposal so that regulators, ethicists, and providers can attack it before any of us treat it as settled.

01 /Why the existing consent paperwork does not fit AI

Healthcare consent has always done two jobs at once. It documents a clinical decision the patient and clinician have made together, and it creates a defensible record for the institution. The form is therefore necessarily compressed. It does not describe everything that happens to the data. It cannot. The ambient understanding has been that the unwritten parts are governed by general law (data protection, professional regulation, common-law confidentiality) and by institutional governance.

That works when the unwritten parts are stable and predictable. AI training breaks that stability in three ways.

First, the data flows further. A model trained on data from a single institution may be fine-tuned with data from another, deployed across a network, and embedded in a third party's product. The patient signed a form at one site. The data, in compressed parametric form, is now somewhere else.

Second, the purpose is harder to bound. A study has a protocol, a sponsor, an ethics approval, and an end date. A foundation model has a training corpus, a downstream task that may not yet exist, and a planned-update cycle. The FDA's 2024 PCCP guidance acknowledges this directly by allowing manufacturers to pre-specify the kinds of changes they may make to an AI device after release ^[14]. The MHRA's roadmap takes a similar line for the UK ^[15]. Neither document is a consent framework. They are a model-side response to model-side change. The consent side has not had its equivalent.

Third, the asymmetry between training data and deployment context is genuinely new. A model trained on data from urban tertiary care can be deployed at a primary care clinic in another country. The patient population at the deployment site may look nothing like the training set. The patients at the deployment site never consented to the training set existing in the first place, but they are now affected by a model whose parameters embody it.

The patients who signed the original consent form, in many cases, do not know any of this is happening. That is not because they are being deceived. It is because the consent form was not written to ask the question.

02 /The UK and EU regulatory frame in practice

GDPR Article 9 makes health data a special category and prohibits processing unless one of the listed conditions applies ^[1]. Most clinical processing relies on Article 9(2)(h) (necessary for medical diagnosis or care) or Article 9(2)(i) (public interest in public health). Secondary use for AI training tends to lean on Article 9(2)(j), processing necessary for archiving in the public interest, scientific or historical research, or statistical purposes, with the safeguards required by Article 89.

Article 89 expects safeguards: technical and organisational measures, including data minimisation and pseudonymisation. It does not, on its face, mandate AI-specific patient-facing notice. It assumes the standard transparency obligations under Articles 13 and 14 are doing that work. They were not written for AI training.

The ICO has tried to plug the gap with its AI guidance ^[2]. The guidance covers lawful basis selection, fairness and accuracy, transparency, and data protection by design. It is good guidance. It does not, and could not, fix the underlying issue that the consent forms in clinical use predate the technology. The Data Protection Act 2018 supplements UK GDPR with conditions for health and social care processing ^[3], and NHS England's Data Security and Protection Toolkit operationalises the security and information governance baselines for NHS organisations and their contractors ^[4]. NHS England also runs the national data opt-out, which allows patients to opt out of confidential patient information being used for research and planning beyond direct care ^[5]. The national data opt-out is, in effect, a national-scale third layer of consent. It works, in the sense that it exists and is operationalised. It is also coarse. It does not distinguish between research, commercial AI training, and other uses.

The 2017 ICO finding against the Royal Free London NHS Foundation Trust is the most cited UK example of an AI consent failure ^[6]. The trust transferred 1.6 million patient records to DeepMind for the development of Streams, a clinical alerting app. The ICO concluded the transfer did not have a valid lawful basis. The case is often described as a transparency failure. We read it differently. It was a consent-design failure. The patients had not been told, the form they had signed had not been written to cover the kind of transfer that actually happened, and the institutional governance had not caught the mismatch.

The Topol Review, published by NHS Health Education England in 2019, framed the digital and AI readiness question for the NHS workforce and named patient engagement as part of the answer ^[19]. The Wellcome / Ipsos MORI Understanding Patient Data work, particularly the One-Way Mirror report on commercial access, found that UK public attitudes are more supportive of NHS-internal use than of commercial flows, and that transparency materially affects acceptance ^[20]. The Ada Lovelace Institute's 2022 algorithmic impact assessment case study with the NHS AI Lab proposed an assessment process for procuring AI in healthcare that builds in stakeholder engagement and consent considerations ^[21]. None of these is a regulatory instrument. All of them are signals that the consent question has been visible in UK policy circles for some time without a corresponding fix in the consent forms patients sign.

03 /Nigeria, South Africa, and the African policy frame

Nigeria moved from a regulator-issued framework to a primary statute when the Data Protection Act 2023 came into force ^[7]. The NDPR 2019 sits alongside the Act and continues to provide the operational framework, including special-category-data requirements ^[8]. The Nigeria Data Protection Commission is the primary regulator. Health data falls within the special category list. The NDPA 2023 introduced statutory penalties and clearer accountability requirements that did not exist in the same form under NDPR alone.

NITDA's National AI Strategy, published in August 2024, sets ethical and operational direction without statutory weight ^[9]. It names data governance and health among its priority areas. The strategy is a useful articulation of where the policy intent is heading. It does not yet have a corresponding consent regime for AI training in clinical settings. Hospitals operating in Nigeria are, in practice, falling back on NDPA / NDPR plus institutional ethics committee processes. That stack works for traditional research. It does not specifically address foundation-model training.

South Africa's POPIA names health information as a special category and requires a stricter justification for processing ^[10]. The Information Regulator has issued guidance on responsible-party obligations and is a relatively active enforcer compared to some of its peers. POPIA's research provisions allow processing for historical, statistical, or research purposes subject to safeguards, similar in shape to GDPR Article 89. The same gap therefore applies. The legal basis for AI training is defensible, but the patient-facing consent does not specifically describe AI training as a distinct purpose.

The wider African policy picture, including the AU Continental AI Strategy adopted by the African Union in 2024, sets aspirational direction. The continental strategy acknowledges health AI and data sovereignty as priorities but operates above the level of consent paperwork. National regulators, ministries of health, and individual institutions are the layer where consent design actually happens. That layer, in our reading, has not yet caught up with the technology.

04 /The US picture: HIPAA, state laws, and the FDA

HIPAA's Privacy Rule covers most of what happens to protected health information held by covered entities and business associates ^[11]. Treatment, payment, and healthcare operations are permitted without authorisation. Research generally requires either patient authorisation or an IRB / privacy board waiver. De-identified data, under the Safe Harbor or expert determination methods, falls outside HIPAA's coverage entirely.

This is where the gap opens. Many AI training pipelines run on de-identified or limited datasets, which means they are not subject to HIPAA in the same way as the underlying clinical record. Patients are typically informed of the de-identification possibility through the Notice of Privacy Practices, which is a long document few patients read carefully. A model trained on a de-identified dataset and then deployed back into clinical care creates a feedback loop that HIPAA was not designed to govern.

State laws have begun to fill that space. Washington's My Health My Data Act, passed in 2023, applies to consumer health data outside HIPAA's reach, including data from wellness apps, mental-health platforms, and reproductive-health services ^[12]. It requires affirmative consent for collection of consumer health data, separate consent for sharing, and gives consumers a right to deletion. Enforcement runs through the state's Consumer Protection Act, which carries meaningful damages exposure. California's CCPA / CPRA framework treats sensitive personal information, including health information, as a category requiring a higher tier of disclosure and opt-out ^[13]. Other states, including Connecticut and Texas, have moved in similar directions.

The FDA's 2024 final guidance on Predetermined Change Control Plans for AI-enabled device software is the clearest signal yet that the regulator is treating AI as a lifecycle problem rather than a point-in-time approval ^[14]. PCCP allows manufacturers to specify in advance the kinds of changes they may make to an AI device after release without triggering a new submission, provided the changes stay within an approved envelope. This is a sensible response to the operational reality of AI. It is also a regulatory mechanism that does not have a consent equivalent. The patient who interacts with an AI device under a PCCP is interacting with a system whose parameters may have shifted since the model was trained on their data, and the consent paperwork they signed at the originating institution does not describe that shift.

05 /Where WHO and the ethics literature land

The WHO 2021 guidance, Ethics and governance of artificial intelligence for health, set out six core principles ^[16]. Among them: protect autonomy, ensure transparency, and foster responsibility and accountability. The guidance was deliberately principles-based. It is widely cited by ministries of health when designing national AI policy. Its 2024 update on large multi-modal models extends the framework to generative and foundation-model use cases ^[17]. The 2024 update is more direct about the data lifecycle question and explicitly names the issue of training-data provenance.

The academic literature has been ahead of the regulators on these questions. Effy Vayena's work, including her 2018 paper with Alessandro Blasimme and I. Glenn Cohen on machine learning in medicine, named consent, fairness, and accountability as the central ethical challenges and has been cited consistently in subsequent regulatory thinking ^[18]. The dynamic consent literature, of which Vayena is a notable contributor, has argued for years that consent should be revisitable and granular rather than one-shot and broad. The technology to support dynamic consent has existed in research-cohort settings (UK Biobank-adjacent work, for instance) for some time. Its translation into routine clinical settings has been slow.

Cohen's broader body of work on AI and health law, including his commentary on FDA pathway design and on liability allocation, has been a useful frame for the policy conversation in the US ^[18]. His point, repeated across multiple venues, is that the regulatory infrastructure for medical devices was not designed for software that learns. That argument has been substantially adopted by the FDA in its PCCP framework ^[14]. It has not yet been adopted with equal force on the consent side.

The Ada Lovelace Institute's algorithmic impact assessment case study with the NHS AI Lab in 2022 ^[21], the Wellcome / Understanding Patient Data evidence base on UK public attitudes to commercial access ^[20], and the Future of Privacy Forum's industry guidance on health and wellness AI ^[22] each push in the same direction: more granular notice, separable consent for distinct uses, and a clearer line between in-institution improvement and commercial flows. None of them, individually, has been picked up as a regulatory instrument. Together they describe a fairly stable consensus in the policy community about where things should head.

06 /Three regulatory gaps that matter

Across the regimes we have read, three gaps recur.

Provenance disclosure. Patients should be able to know, in plain language, what categories of data their care provider has used to train models that affect their care. The current state in most jurisdictions is that this information is not collected in a form that could be disclosed even if a patient asked. Hospitals do not always have a clean inventory of which datasets they have shared with which AI developers under what terms. The DPIA process under GDPR is the closest existing instrument and it is internal-facing rather than patient-facing.

Cross-institutional flow. There is no clear regime in any jurisdiction we have read for what happens when a model trained at institution A is deployed at institution B. The patient at B has not consented to A's data being used to inform their care, even though the model's parameters embody that data. The current implicit answer is that, because the data has been transformed into model weights, no further consent is needed. We do not think that answer is adequate. We are not arguing it is wrong as a legal matter. We are arguing it is a position that has not been tested and that deserves to be made explicit.

Withdrawal of consent. If a patient withdraws AI-training consent today, what happens to model parameters that already reflect their historical data? Strictly speaking, the data has been used; the model has been trained; the gradients have been applied. Reversing that contribution is technically possible (machine unlearning research has been moving fast in this area) but is not standard practice. Most consent frameworks have no answer to the question of what withdrawal actually means once the data has been incorporated into a model. The honest answer in many cases is that withdrawal currently means future data will not be used, but past contributions cannot be cleanly removed. That is a defensible position. It is also a position that should be communicated to patients at the point of consent, not discovered after withdrawal.

These gaps are not theoretical. The Royal Free / DeepMind finding ^[6] turned on something close to provenance disclosure. The Wellcome work on commercial access ^[20] turned on cross-institutional flow. The dynamic consent literature ^[18] turned on withdrawal. Each gap has been visible in case law, public attitude research, or academic ethics. None is yet routinely closed in front-line consent paperwork.

07 /A proposed layered consent model

We propose, as a published recommendation, a three-layer consent model. The layers are designed to be independently opt-out-able and to map onto how patients actually think about their data.

Layer one is treatment consent. This is unchanged from current practice. A patient in front of a clinician consents to the clinical decisions being made and to the data flows that those decisions require. The legal basis under GDPR is typically Article 9(2)(h). Under HIPAA it is treatment under the Privacy Rule. Under POPIA it is the equivalent processing condition for legitimate interests in healthcare. Under NDPA 2023 it is the corresponding processing condition for medical care.

Layer two is audit and quality improvement consent. This too is largely unchanged. The patient is informed that their data may be used for clinical audit, governance, and internal quality improvement. This corresponds to healthcare operations under HIPAA, the public-interest research provisions under GDPR Article 9(2)(j), and the equivalents elsewhere. The governance bodies that already exist (clinical audit committees, information governance teams) continue to do this work.

Layer three is AI-specific consent, and this is where the proposal does its work. We suggest this layer be subdivided into three named uses: in-institution model improvement (data used to train models that the institution itself deploys for its own patients), cross-institutional model training (data shared with partner institutions or with developers building models intended for deployment beyond the originating institution), and commercial AI products (data used to train models that are sold or licensed). Each of the three sub-categories can be opted out of independently.

Layer three is paired with two operational commitments. First, an annual patient-facing summary that tells the patient, in plain language, what categories of data the institution has used in each of the three sub-categories. Second, a clear statement at the point of consent about what withdrawal means in each sub-category, including the honest acknowledgement that withdrawal of training consent applies to future data and that historical contributions to already-trained models may not be cleanly removable.

We think the model is implementable. We do not claim to have implemented it. We are publishing it as a proposal, with the explicit invitation to be told why it is wrong, what it misses, and where it would fail in operation. We expect to be wrong about parts of it. That is the price of being specific enough to be useful. The shape of the proposal is consistent with the dynamic consent direction in the academic literature ^[18], with the operational pattern of the NHS national data opt-out ^[5], and with the algorithmic-impact-assessment thinking from Ada Lovelace ^[21]. None of those informs the proposal in a way that lets us claim novelty. What we are doing is making the synthesis explicit and arguing it should become the default.

Coderex advises hospital systems, ministries of health, and AI developers on how to redesign clinical consent for the AI era and on how to operationalise it without disrupting front-line clinical workflow. The model in this paper is one input. Implementation specifics will vary by jurisdiction, by institutional governance maturity, and by what an honest reading of patient flow at admission can actually carry.

Expect the first major regulator finding on consent for AI training to land in the UK or US within the next 18 months, before any of the African regimes have had the chance to litigate the question through their less-developed enforcement infrastructure. Expect the academic dynamic-consent literature to be picked up by at least one national health service in a published policy by 2028. Expect the gap between FDA-style lifecycle regulation and the consent paperwork patients actually sign to become the dominant policy story in healthcare AI through 2027. The institutions that get ahead of this gap publicly will be in a meaningfully stronger position than those that wait for the regulator finding.

Methodology

This whitepaper is a desk review of public regulatory texts, regulator guidance, peer-reviewed literature, and published civil-society research. We have read each cited source. We have not generated any composite scoring or original empirical data for this paper. The layered consent model in section 7 is presented as a published recommendation, not as a deployed system, and is not based on any specific Coderex client engagement.

Three caveats are worth naming explicitly. First, regulatory positions move. The Nigeria DPC, the UK ICO, the FDA, and the MHRA all publish updates with non-trivial frequency. Where we have cited a specific document, the publication date is recorded in the bibliography and the position may have shifted since. Second, our framing draws disproportionately on UK, US, Nigerian, and South African regimes because those are the markets where Coderex's data governance practice operates most heavily. We have not, for instance, surveyed Brazil's LGPD, India's DPDP Act, or the EU AI Act's specific health-system implications in equivalent depth. Third, the layered consent model is a proposal, and proposals are easier to write than to operationalise. The implementation reality, including patient flow at admission, system integration, and the cost of an annual summary, would shape any actual deployment.