DeepMind’s recent foray into providing software as a service to UK hospitals has reached the end of its run.
The Google -owned AI division has just announced it will be stepping back from providing a clinicial alerts and task management healthcare app to focus on research — handing off the team doing the day to day delivery of the Streams to its parent, Google.
Announcing the move in a blog post entitled ‘Scaling Streams with Google’, DeepMind’s co-founders write: “Our vision is for Streams to now become an AI-powered assistant for nurses and doctors everywhere — combining the best algorithms with intuitive design, all backed up by rigorous evidence. The team working within Google, alongside brilliant colleagues from across the organisation, will help make this vision a reality.”
DeepMind’s 2015 plunge into the health apps space always looked like a curious departure for an AI specialist because — despite the above quote — the Streams app does not use any AI.
Rather it uses a National Health Service algorithm. The design of the app was also outsourced to a UK based app studio.
Yet DeepMind began its foray into health with grand ambitions about applying AI to patient data, quietly inking an expansive data-sharing arrangement plus memorandum of understanding with an NHS Trust to get access to millions of patients’ full (and fully identifiable) medical records, as we reported at the time.
It also made a 2015 ethics application with the NHS’ Health Research Authority to apply AI to the patient data. Though it later said it quickly realized that clinicians’ “most urgent problems” were rather more fundamental than a pressing need to rush into experiments with AI. (And DeepMind has always maintained that the patient data it obtained under its arrangement with the Royal Free NHS Trust, with whom the Streams app was co-developed, was never used for AI.)
The Streams project ran into major controversy in May 2016 when fuller details emerged about the scope and terms of the data-sharing underpinning the app — and questions started being asked about data governance due process, legal bases for data-sharing and Google’s role and potential interest in people’s medical records.
After a year, the initial data-sharing arrangement between DeepMind and the Royal Free was scraped and replaced with a tighter contract.
Then last year the UK’s data protection watchdog ruled the first arrangement had breached UK law — with the information commissioner saying patients “would not have reasonably expected” their sensitive medical records to be used for developing an app.
Although by then the Streams app had already been deployed into Royal Free hospitals. And DeepMind had inked a few more deals with NHS Trusts to use the app.
It also emerged that DeepMind was providing Streams to Trusts essentially free of charge for the first five years. And a panel of external reviewers engaged by DeepMind with the aim of boosting trust warned in their annual review earlier this year of the risk of it “exert[ing] excessive monopoly power” as a result of a data access and streaming infrastructure that it bundles with the Streams app.
The whole episode opened a pandora’s box of data governance, privacy and trust issues — which DeepMind now appears to be dumping directly onto Google which will now be fully in the frame as the health app provider (and patient data handler) behind Streams.
“The Streams team will remain in London, under the leadership of former NHS surgeon and researcher Dr Dominic King,” write the DeepMind co-founders now. “We’re fully committed to all our NHS partners, and to delivering on our current projects and more. We’ll be working closely with them as we plan for the team’s transition, and information governance and safety remain our top priorities. Patient data remains under our partners’ strict control, and all decisions about its use will continue to lie with them.”
They add that DeepMind’s role from here on in will be focused on research, rather than software as a service, saying: “As a research organisation, DeepMind will continue to work on fundamental health research with partners in academia, the NHS and beyond. When we have promising results that could have impact at scale, we’ll work closely with the Streams and translational research teams at Google on how to implement research ideas into clinical settings.”
So provision of any health AIs that DeepMind develops in the future will be left to Google to deploy and scale.
As will the task of winning patient trust — which may well prove the biggest challenge here.
The trust issue was also flagged by DeepMind’s independent reviewers last year, when they wrote in their annual report: “As far as we can ascertain, DMH [DeepMind Health] does not share its data with Google, yet the public perception that this might be the case, now or in the future, will be difficult to overcome and has the potential to delay or undermine work that could be of great potential benefit to patients.”
It’s not clear whether Google will engage a panel of independent reviewers to oversee its provision of Streams going forward, as DeepMind had. We’ve asked the company to confirm its intention vis-a-vis oversight.
COMMENTS