Doctors to refer NHS to data watchdog over feeding patient data to AI

LONDON — Doctors will urge the U.K.’s data watchdog to investigate whether NHS England broke the law in training its generative artificial intelligence model Foresight on patient records.

Doctors are drafting a formal complaint to the Information Commissioner’s Office alleging that NHS England breached data protection rules by plugging general practitioner patient data solely intended for Covid-19 research into a more all-purpose AI model without consent, according to a person familiar with the situation, granted anonymity to speak candidly.

NHS England confirmed in a statement to POLITICO on 30 May that it had paused Foresight’s ongoing data processing and launched an internal audit into the project after representatives from the British Medical Association (BMA) and Royal College of General Practitioners (RCGP) raised concerns.

Touted as a “world-first pilot project” capable of helping early interventions by pinpointing high-risk patient groups, Foresight was trained on the de-identified data of 57 million people in England, including NHS England’s Data for Pandemic Planning and Research (GDPPR). 

NHS England’s own guidelines state that applications for accessing GP data must undergo additional review by a Professional Advisory Group (PAG), which comprises members of the BMA, RCGP, and Caldicott Guardian Arjun Dhillon.

The PAG approved the British Heart Foundation consortium’s use of GDPPR specifically for Covid-19 research under a pandemic-era emergency directive — but was never consulted about large language model training. 

“The methodology appears to be new, contentious, and potentially with wide repercussions. It appears unlikely that a proposal would have been supported without additional, extraordinary agreements to permit it,” GP leaders wrote to Ming Tang, chief data and analytics officer at NHS England on 16 May, according to an email seen by POLITICO. 

“The self-declared scope of this project appears inconsistent with the legal basis under which these data were to be used,” they added in the email to Tang.

ICO demand

Another fault line has opened up between doctors and NHS England over whether the Information Commissioner’s Office should be called in to conduct the audit externally.

The Joint GP IT Committee — which represents GPs across the UK in discussions related to the use and management of GP data — asked NHS England to refer itself to the data watchdog over the issue following a meeting last Thursday.

But Michael Chapman, director of research and clinical trials at NHS Digital, wrote to seven members of the committee that evening saying the review would instead be carried out in-house by the organization’s data protection officer “to establish the facts ahead of any approach to the [Information Commissioner’s Office],” citing “standard procedure when data protection concerns are raised,” according to an email seen by POLITICO. 

That’s done little to placate doctors’ representatives, who are wary of allowing NHS England to mark its own homework given the gravity of the issue.

In addition to demanding that the BMA be called as a witness to the Information Commissioner’s investigations, the Joint GP IT Committee also wants “explicit governance” over uses of AI and an undertaking that future emergency measures permitting the use of GP data contain a sunset clause if doctors haven’t been consulted, the person quoted above said.

It comes as influential figures inside the governing Labour Party push to liberalize the use of NHS data. This week former Prime Minister Tony Blair made a surprise appearance at the SXSW festival in London alongside Tech Secretary Peter Kyle, where he said it was “absurd” that NHS data wasn’t routinely available for innovation. 

Kyle also called for the U.K. to make better use of NHS data: “This government, this country, our state has the most extraordinarily powerful and rich data set, [more] than any other country in the world. Now, if we can use that data wisely and safely, then we can have the kinds of leaps and bounds forward for the scientific development for the commercialization of new techniques, new services, new medicines, and the understanding of humankind and social sciences that no other country in the world can do.” 

An NHS spokesperson said: “Maintaining patient privacy is central to this project and we are grateful to the Joint GP IT Committee for raising its concerns and meeting with us to discuss the strict governance and controls in place to ensure patients’ data remains secure.” 

An ICO spokesperson said: “We are aware of the Foresight AI model and regularly engage with NHS England on data protection issues to ensure the public body continually strives to meet its data protection obligations and mitigate potential risks.”

The Department of Health and Social Care declined to comment. The Department for Science, Innovation and Technology did not respond to a request for comment.