Dive Brief:
- A STAT review of internal IBM documents suggests the company’s Watson supercomputer wrongly advised doctors on how to treat patients’ cancers.
- The documents — slides presented by then-IBM Watson Health deputy chief health officer Andrew Norden in June and July of last year — include “multiple examples of unsafe and incorrect treatment recommendations” and raise “serious questions about the process for building content and the underlying technology.”
- IBM has insisted that Watson for Oncology’s advice relies on real patient data, but that reportedly wasn't so. No patients died as a result of the inaccurate advice, according to STAT.
Dive Insight:
The documents include comments from a physician at Jupiter Hospital in Florida, who told IBM officers: “This product is a piece of s—. We bought it for marketing and with hopes that you would achieve the vision. We can’t use it for most cases.”
Several IBM employees also reported to Norden that Watson for Oncology was “very limited,” according to STAT.
In a statement, IBM defended Watson for Oncology, saying it has trained oncologists to treat 13 cancers and supported care for more than 84,000 patients at 230 hospitals around the world.
“We have learned and improved Watson Health based on continuous feedback from clients, new scientific evidence and new cancers and treatment alternatives. This includes 11 software releases for even better functionality during the past year, including national guidelines for cancers ranging from colon to liver cancer,” the statement reads.
This is not the first troubling account of the program. A 2017 Forbes report found that Watson Health’s partnership with MD Anderson Cancer Center was not meeting up to its goals. The report said MD Anderson was actively seeking bids from other contractors that might replace IBM in future projects.
And a previous STAT investigation found Watson Health has struggled to learn about different cancer types, with a relatively small number of hospitals adopting its oncology offerings.
Watson Health has been showing other signs of stress recently. Last month, Watson Health officials cited soft product demand and changes to the Affordable Care Act for a decision to scale back its hospital business. The decision marks an abrupt turn for a company that has invested heavily to win contracts with hospitals moving to value-based care. The business develops tools to help providers manage their pay-for-performance contracts.
IBM also recently confirmed that it had laid off an undisclosed number of workers at Watson Health, but maintained its cognitive solutions unit would remain strong.
The reports of inaccurate advice could slow growth of machine learning and artificial intelligence-assisted efforts in medicine, or at least deepen the conviction many hold that humans need to make the ultimate decisions regarding patient care.
During HIMSS 2018, an expert panel suggested that AI’s potential may be overhyped, but that the technology is ready for prime time. What’s needed is physician trust that AI is reliable and worthy of being adopted.
