In 2011, IBM’s Watson took the world by storm when it won the television game show Jeopardy!, showcasing the power of artificial intelligence (AI). Emboldened by this success, IBM sought to extend Watson’s capabilities beyond trivia to address real-world challenges.
Healthcare, with its complex data and critical decision-making needs, became a primary focus. Among its flagship initiatives was Watson for Oncology, a system designed to assist doctors in diagnosing and treating cancer through AI-driven insights.
Cancer treatment represents one of the most intricate and rapidly evolving domains in medicine. With over 18 million new cases diagnosed globally each year, oncologists face an overwhelming amount of medical literature, treatment protocols, and emerging research. Watson for Oncology aimed to address this challenge by analyzing vast amounts of data to recommend evidence-based treatment plans, all in a matter of seconds.
IBM marketed Watson for Oncology as a revolutionary tool that could bridge the gap between cutting-edge research and clinical practice. Its promise was to assist oncologists in identifying personalized treatment options for patients, thereby improving outcomes and reducing variability in care.
However, this ambitious vision quickly collided with the complex realities of cancer care, resulting in widespread criticism and eventual failure.
Background
At the start of the project it had five lofty objectives;
1) Streamlining Clinical Decision-Making: Watson for Oncology aimed to provide oncologists with AI-generated insights, synthesizing vast amounts of data into actionable treatment recommendations.
2) Bridging Knowledge Gaps: With the rapid pace of medical advancements, Watson sought to keep clinicians updated on the latest evidence, clinical trials, and treatment protocols.
3) Improving Patient Outcomes: The system was designed to support personalized care by tailoring treatment recommendations to each patient’s unique genetic and clinical profile.
4) Expanding Access to Expertise: IBM envisioned Watson as a tool to democratize high-quality oncology care, particularly in resource-limited settings where access to specialists is constrained.
5) Establishing Market Leadership: Beyond healthcare, IBM sought to position Watson as a leader in AI applications, demonstrating the transformative potential of cognitive computing.
The project was supported by partnerships with leading institutions like Memorial Sloan Kettering Cancer Center (MSKCC) to train Watson for Oncology.
The partnership aimed to imbue Watson with the expertise of MSKCC’s oncologists by feeding it clinical guidelines, peer-reviewed literature, and patient case histories. The AI would then analyze patient records and suggest ranked treatment options based on the latest evidence. It was envisioned as a tool that could augment oncologists' expertise, particularly in under-resourced settings.
IBM invested heavily in the project, pouring billions into Watson Health, which encompassed Watson for Oncology. The company acquired several firms specializing in healthcare data and analytics, including Truven Health Analytics and Merge Healthcare. These acquisitions were meant to enhance Watson’s capabilities by providing access to large datasets and advanced imaging tools.
Initial trials and pilots were conducted in countries like India and China, where disparities in healthcare resources presented an opportunity for Watson to make a meaningful impact. However, reports soon emerged that the AI’s recommendations were often inconsistent with local clinical practices. For example, Watson’s reliance on U.S.-centric guidelines made it difficult to implement in regions with differing treatment standards or drug availability.
By 2018, skepticism was growing. High-profile reports detailed instances where Watson provided inappropriate or even unsafe recommendations. These challenges, coupled with declining revenues for IBM Watson Health, culminated in the program’s discontinuation in 2023.
This case study examines how a project with such potential faltered, offering lessons for future ventures at the intersection of AI and healthcare.
Is your project headed for trouble? Find out! Just answer the 27 questions of my Project Trouble Assessment, which will take you less than 10 minutes, and you will know.
If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.
Timeline of Events
2011–2012: Watson's Post-Jeopardy! Evolution
Following its Jeopardy! success, IBM began exploring commercial applications for Watson, identifying healthcare as a priority. In 2012, IBM partnered with Memorial Sloan Kettering to develop Watson for Oncology, marking the start of an ambitious initiative.
2013: Initial Development and Training
Watson’s training began with curated data from MSKCC, including clinical guidelines and research publications. Early feedback highlighted challenges in teaching the system to interpret ambiguous or contradictory medical information.
2014: Pilot Testing at Memorial Sloan Kettering
MSKCC oncologists started testing Watson on hypothetical patient cases. Early results revealed gaps in the system’s knowledge and its tendency to offer impractical or unsafe recommendations, raising concerns about its readiness.
2015: Launch and Early Adoption
IBM officially launched Watson for Oncology with aggressive marketing campaigns. Hospitals in countries like Thailand, India, and South Korea signed adoption agreements, drawn by the promise of bringing world-class cancer care to underserved regions.
2016: Growing Skepticism Among Oncologists
Reports emerged of dissatisfaction among oncologists using Watson. Many found the system’s recommendations simplistic, biased toward MSKCC practices, and poorly adapted to local guidelines.
2017: Critical Media Coverage
Investigative reports revealed that some of Watson’s recommendations were based on hypothetical scenarios rather than real-world data. These revelations damaged IBM’s credibility and raised ethical questions about its marketing claims.
2018: Customer Contracts Cancelled
Major clients, including MD Anderson Cancer Center, ended their contracts with IBM, citing high costs and underwhelming results. IBM began scaling back its marketing efforts for Watson for Oncology.
2019: Internal Restructuring at IBM Watson Health
Facing declining revenues, IBM restructured its Watson Health division. Resources were redirected to other AI projects, and development on Watson for Oncology slowed significantly.
2021: Watson Health Division Sold
IBM announced the sale of its Watson Health assets to a private equity firm, effectively marking the end of its ambitions in AI-driven cancer care.
2023: Retrospective Studies Highlighting System Flaws
Postmortem analyses identified systemic issues, including poor data quality, inadequate clinical validation, and unrealistic timelines, as key factors in the project’s failure.
What Went Wrong?
Overreliance on Limited Training Data: Watson’s knowledge base was heavily influenced by MSKCC’s practices, leading to recommendations that often failed to align with local guidelines or real-world cases. This lack of diversity in training data undermined the system’s global applicability.
Unrealistic Marketing Claims: IBM’s aggressive marketing exaggerated Watson’s capabilities, creating unrealistic expectations among customers. When the system failed to deliver, trust eroded quickly.
Inadequate Physician Involvement: Oncologists reported that Watson’s interface was not user-friendly and often disrupted their workflow. Limited engagement with end-users during development contributed to these usability issues.
Lack of Adaptability to Local Contexts: Watson struggled to accommodate variations in healthcare systems, resource availability, and cultural practices. This rigidity limited its usefulness in diverse settings.
Ethical and Transparency Concerns: IBM’s use of hypothetical cases and selective data to demonstrate Watson’s capabilities raised ethical red flags. Customers felt misled by the lack of transparency.
How IBM Could Have Done Things Differently?
Broader and More Diverse Training Data: IBM could have partnered with multiple institutions worldwide to train Watson on a broader dataset, ensuring recommendations were evidence-based and applicable in varied contexts.
Iterative Development with Physician Feedback: By involving more oncologists in the design and testing process, IBM could have identified and resolved usability issues early on, ensuring the system met clinical needs.
Transparent Communication of Capabilities: IBM should have been more transparent about Watson’s limitations, focusing on incremental benefits rather than overhyping its transformative potential.
Emphasis on Local Adaptability: Developing a system that could integrate local guidelines and resource constraints would have made Watson more practical for global deployment.
Strengthened Ethical Oversight: IBM could have established an independent advisory board to review marketing claims, data usage, and clinical validation processes, building trust with stakeholders.
Closing Thoughts
The failure of IBM Watson for Oncology offers valuable lessons for AI projects in healthcare and beyond. It highlights the importance of realistic expectations, rigorous validation, and end-user involvement in developing and deploying AI solutions.
While IBM’s vision was ambitious, its execution fell short, underscoring the challenges of applying AI in complex, high-stakes domains. Moving forward, the healthcare industry must balance optimism about AI’s potential with a commitment to patient safety and ethical responsibility.
Is your project headed for trouble? Find out! Just answer the 27 questions of my Project Trouble Assessment, which will take you less than 10 minutes, and you will know.
If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.
Sources
> IBM official press releases (2011–2021).
> Investigative reports from Stat News and The Wall Street Journal on Watson for Oncology’s challenges.
> Interviews with Memorial Sloan Kettering oncologists published in medical journals.
> Retrospective analyses in The Lancet Digital Health and JAMA Oncology.
> Public statements by IBM executives, including John Kelly III (SVP, IBM Research).
