Why AI can’t (yet) decide your cancer treatment

Artificial intelligence is making its way into oncology, but before AI-driven Clinical Decision Support Systems (CDSS) can transform treatment planning, there’s a fundamental problem to solve—data readiness. A new study by researchers from University Hospital Münster and the German Research Center for Artificial Intelligence (DFKI) examines whether existing medical data is good enough for AI to make meaningful treatment recommendations for skin cancer patients. Their findings highlight the persistent challenges of unstructured medical records, missing critical patient data, and the need for better documentation practices to ensure AI can truly assist in clinical decision-making.

AI needs better data to assist oncologists

Clinical decision-making in oncology is complex. Physicians often rely on limited trial data and personal experience to make treatment choices, especially in advanced-stage cases where standard protocols may not apply. AI has the potential to fill this gap by identifying patterns across thousands of cases—something human memory and intuition simply can’t scale to. However, for AI to provide reliable recommendations, it requires structured, high-quality data—and that’s where the real challenge begins.

Medical records are messy. A significant portion of critical patient information is buried in free-text doctor’s notes, scattered across multiple hospital systems, or entirely undocumented. This study assessed whether current skin cancer treatment data is AI-ready and found that key medical and social factors influencing treatment decisions are either unstructured, inconsistently recorded, or missing altogether.

AI readiness in skin cancer treatment

Researchers evaluated data from five skin cancer patients at the Skin Tumor Center of University Hospital Münster, analyzing how well clinical records could support an AI-driven CDSS. Their methodology involved:

• Extracting medical records from hospital information systems, cancer registries, and laboratory data.
• Assessing data quality using AI-readiness frameworks.
• Holding expert workshops with oncologists to determine which data points are most critical for decision-making.

Key findings:

• Only 13 out of 41 essential data points were available in structured format—the rest were either buried in free-text or completely absent.
• Social and personal factors that influence treatment, such as quality of life, willingness to travel, and patient preferences, were not recorded in any structured way.
• Even within structured data, inconsistencies were rampant—oncologists used different abbreviations, languages, and terminology, making AI-based information extraction highly unreliable.
• Decision-making rationales were often undocumented, meaning AI models would struggle to understand why certain treatments were chosen in past cases.

What if AI did not just write—but edited itself?

Why AI can’t yet assist oncologists effectively

The study confirms what AI researchers have been saying for years—garbage in, garbage out. Without clean, well-structured medical data, AI-driven decision support systems can’t provide accurate or useful recommendations.

This is especially critical in analogy-based AI models like Case-Based Reasoning (CBR), which compares new cases to past cases to recommend treatments. If past case records are incomplete, missing crucial patient history, or filled with inconsistencies, the AI’s recommendations will be flawed from the start.

A major roadblock is the dominance of free-text documentation in electronic medical records (EMRs). Physicians write notes in narrative form, which is useful for humans but difficult for AI to process reliably. While natural language processing (NLP) and large language models (LLMs) could help extract key insights, they require extensive training on high-quality datasets, which simply don’t exist in many hospitals.

Recommendations from the study:

To make AI useful in skin cancer treatment planning, hospitals need to:

1. Standardize medical documentation – Physicians should enter key decision-making factors in structured fields, not just in free-text notes.
2. Improve data quality and consistency – Using uniform medical terminology and structured formats will make AI-driven analysis more reliable.
3. Integrate social and personal factors – Patient preferences, quality of life, and treatment accessibility should be part of the data recorded.
4. Leverage AI for better data entry – AI-powered documentation assistants could help doctors automatically structure their notes without adding extra workload.

Featured image credit: Kerem Gülen/Imagen 3