Conceptual AI-generated art: a missing piece represents essential information absent from an otherwise convincing answer. Not a medical scan.
An answer can contain nothing false and still be unsafe to act on.
That is the distinction I want physicians to keep in mind when evaluating artificial intelligence (AI). A response may correctly describe a diagnosis, summarize a treatment, or identify several relevant facts. The problem may be the contraindication it never checks, the uncertainty it quietly erases, or the next step it leaves unsaid.
Reading for mistakes is not the same as reading for what is missing.
A recent interview with Ethan Goh in ASCO AI in Oncology, published under license from the American Society of Clinical Oncology, makes this distinction central: strong performance on a medical knowledge examination does not establish competence in a different task, and exam-based evaluation can miss errors of omission.[1]
I see that as an argument for better evidence, not less enthusiasm. These tools deserve serious evaluation precisely because they may become useful parts of clinical work. But the relevant question is not simply whether a model knows medicine. It is whether this system helps this clinical team perform this particular task safely.
Start with three questions
Before getting lost in a leaderboard, I would ask:
What exactly is the tool supposed to do? Draft a patient explanation, extract treatment dates, suggest a differential, and prioritize a worklist are different claims.
What must it never leave out? Identify the missing fact, qualification, or action that could change the decision.
How will the clinician verify it? The answer needs an inspectable basis, visible uncertainty, and a workable path for correction.
Those questions are not a complete validation protocol. They are a practical way to tell whether the evidence being offered is even about the decision in front of us.
A high examination score may be relevant to a medical reference tool. It cannot, by itself, tell us whether a chart-screening system preserves missing information or whether a busy physician notices the warning at the bottom of its output.
An exam score measures something important, but narrower
Benchmarks are standardized sets of tasks used to compare performance. They make evaluation more reproducible and help expose weaknesses. The mistake is not using them. It is asking a score to support a claim the test was not designed to establish.
A 2025 analysis by Siun Kim and Hyung-Jin Yoon examined 702 clinical evaluations of 85 language models across 168 studies and found only a moderate association between medical question-answering benchmark scores and clinical performance.[2]
That does not make examination performance meaningless. It means the relationship is incomplete. Nor should the phrase clinical evaluations be read as a collection of prospective trials proving patient benefit. The study examines reported task performance, not a universal measure of safe deployment.
Consider the difference between selecting an answer from a complete vignette and reconstructing a treatment history from a folder of notes. In the latter task, the system must determine which document matters, reconcile dates, distinguish planned treatment from completed treatment, and preserve unresolved conflicts. Knowing the right medical principle is only one part of that job.
I would treat evidence as a progression: knowledge, task performance, performance in the actual workflow, and then clinical benefit. Success at one stage gives a reason to investigate the next. It does not let us skip it.
The missing piece is a safety problem
An error of commission adds something wrong. An error of omission leaves out something necessary. A review that checks only the statements present in an answer can miss the second category entirely.
The NOHARM study, short for Numerous Options Harm Assessment for Risk in Medicine, makes that concern concrete. In the July 2026 version of this preprint, omissions accounted for more than 80% of severe errors across the evaluated systems on specialist-consultation tasks.[3]
The denominator matters. This does not mean that more than 80% of answers were harmful. The study evaluated potentially harmful recommendations and omissions against expert annotations. It did not measure an observed rate of patient injury in routine care.[3]
Its practical lesson is still important: checking factual correctness is not enough. We also need a task-specific account of what a safe and useful response must include.
For a patient explanation, that might include the uncertainty that changes how a result should be understood. For a management suggestion, it might include a prerequisite assessment. For record extraction, it might be an explicit statement that a decisive fact was not found.
Completeness does not mean maximal length. An answer padded with every conceivable caveat can be difficult to use. I want the essential information for the decision, with secondary detail available without burying the point.
In oncology, unknown is not no
Consider a hypothetical clinical-trial screening workflow. This is an illustration, not a report of a tool I have tested or a study I have conducted.
Suppose a protocol requires a specific biomarker result. The records available to the screening system do not contain the report. The patient may not have been tested, the result may be pending, or the report may be somewhere the system cannot access.
If the tool turns that absence into a negative biomarker result, it has changed the meaning of the record. If its interface then removes the patient from the review queue, a missing document has become a consequential decision.
For this hypothetical workflow, I would want at least three distinct states:
The available evidence establishes that the criterion is met.
The available evidence establishes that the criterion is not met.
The criterion cannot yet be assessed from the available evidence.
Conflicting information should also remain visible rather than being forced into a clean answer. These are screening states, not a substitute for final eligibility determination.
The same principle applies to a treatment date, a prior exposure, or a required assessment. A system should distinguish what the record establishes from what it has inferred and what it still needs.
I would prefer a visible request for review to an unsupported conclusion that silently closes the question.
A citation is a starting point, not the end of verification
For clinical use, provenance means being able to trace an output to the evidence behind it.
That has two different forms. A medical recommendation may need a guideline or study supporting the claim. A patient-specific fact needs the relevant passage in that patient’s record. A literature citation cannot establish that a particular patient received a drug on a particular date.
For a consequential extracted fact, I would want the original wording, document location, and relevant date beside the normalized value. If two notes disagree, I would want to see the disagreement. If a conclusion depends on a protocol version, I would want that version identified.
The point is not to decorate an answer with references. It is to make verification possible without asking the physician to reconstruct the entire task from scratch.
There is another layer: who decided what a good answer should contain?
PanCanBench, a pancreatic oncology benchmark described in a March 2026 preprint, used 282 authentic patient questions and question-specific evaluation criteria. The authors reported that synthetic AI-generated rubrics inflated absolute scores by an average of 17.9 points compared with their expert-rubric approach.[4]
That is a finding from this benchmark, not proof that every automated rubric is unreliable. It is a reason to scrutinize the scoring method as carefully as the model. A polished score is only as meaningful as the standard behind it.
Evaluate the deployed system, including the physician
The model is not necessarily the whole product.
Imagine a hybrid system: one component retrieves documents, a language model extracts facts, fixed software rules apply criteria, and an interface presents a worklist for human review. Each step creates a different opportunity for error.
The model could extract the correct number while the software uses the wrong unit. The rules could work correctly on the wrong document version. The final display could hide uncertainty that was preserved internally.
Testing the model alone would not establish the reliability of that complete chain. I would want component testing to locate failures and end-to-end testing to show what actually reaches the user.
Human assessment is not a new addition to this field. Singhal and colleagues’ 2023 Nature paper on medical language models evaluated dimensions beyond multiple-choice accuracy, including possible harm, completeness, bias, and helpfulness.[5]
But evaluating an answer with human raters is different from evaluating clinicians using a tool. In the NOHARM preprint’s randomized study of 101 generalist physicians, AI assistance improved performance compared with conventional resources, yet assisted physicians frequently omitted valuable recommendations and scored below many systems operating alone.[3]
That does not establish that autonomous practice is preferable. It shows why adding a physician cannot simply be assumed to produce the best combination of human and machine capabilities.
For a local evaluation, I would compare usual review with assisted review using the actual interface. Can physicians reject unsupported suggestions, recover missing information, and recognize when the system should not be used? Does it save time after verification and correction, not just before them?
Human oversight should be a tested part of the workflow, not a reassuring phrase attached to it.
A physician’s checklist before the next demonstration
Here is the more detailed checklist I would bring to a vendor meeting or an internal pilot review. These are my practical recommendations, not a validated scoring instrument.
Ask for a task-specific claim. Define the input, intended user, output, and downstream decision. Broad claims about medical intelligence are not a substitute.
Ask to see consequential misses. Request examples of omitted required elements, unsupported conclusions, and incorrect handling of missing data. Averages should not obscure the failures that matter most.
Ask how the answer key was built. Relevant clinicians should define essential content and adjudicate disagreement. The test should allow defensible alternatives rather than pretending every complex case has one perfect answer.
Ask for difficult records, not just clean demonstrations. Include incomplete, conflicting, duplicated, and outdated information. Test whether uncertainty survives from the input through the final display.
Ask what a negative result does. Does it merely inform review, or can it suppress a patient, referral, or option? The consequences should determine how demanding the evidence needs to be.
Ask for denominators and uncertainty. How many relevant opportunities for error were tested? Were important case types represented? A small set with no observed failures does not establish that failures will be rare in practice.
Ask how the team performed. Count missed items, unnecessary review, correction time, and appropriate overrides. Faster generation alone is not the same as a better workflow.
Ask what happens after an update. Record the tested configuration and define when changes to the model, prompt, evidence sources, rules, or interface require retesting. Assign responsibility for monitoring, stopping use, and recovery.
Not every low-risk drafting aid needs the evidence package of a treatment decision system. The intensity of evaluation should follow the intended use and the consequences of failure. But even a modest tool needs a clear boundary around what it is allowed to do.
Better benchmarks should make us more optimistic
The evaluation landscape is moving beyond a single examination score. The Medical AI Superintelligence Test, or MAST, presents assessments across clinical reasoning, safety, and medical images.[6]
Such work can help us ask more precise questions. It does not certify a local workflow, and its title should not be mistaken for proof of autonomous clinical competence.
My optimism is strongest when a tool has a narrow job, shows its evidence, preserves uncertainty, and makes the clinician’s work easier to inspect. That is a more useful ambition than demanding either blind trust or perfect performance at every medical task.
The next step is not to stop measuring what these systems know. It is to measure what they leave out, how those gaps travel through software and human decisions, and whether the complete workflow improves care.
A good clinical answer must be more than correct in what it says. It must be sufficiently complete for what we intend to do next.
Educational commentary, not patient-specific medical advice. The trial-screening example is hypothetical. Sources checked September 8, 2026; findings from the cited NOHARM and PanCanBench preprints should be interpreted as provisional.
Sources
[1] https://ascoai.org/articles/2026/09/why-more-and-better-benchmarks-are-needed-for-clinical-ai | Why More, and Better, Benchmarks Are Needed for Clinical AI | ASCO AI in Oncology
[2] https://aclanthology.org/2025.bionlp-1.24 | Questioning Our Questions: How Well Do Medical QA Benchmarks Evaluate Clinical Capabilities of Language Models? - ACL Anthology
[3] https://arxiv.org/abs/2512.01241 | [2512.01241] First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations
[4] https://arxiv.org/abs/2603.01343v1 | [2603.01343v1] PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology
[5] https://www.nature.com/articles/s41586-023-06291-2 | Large language models encode clinical knowledge | Nature
[6] https://www.arise-ai.org/mast | MAST: Medical AI Superintelligence Test & Benchmarks | ARISE