Cluster-randomised trial of an LLM decision-support tool in Kenyan primary care finds no reduction in treatment failure
In a pragmatic trial at 16 Penda Health clinics in Kenya, clinical officers randomised to a GPT-4o-based decision-support tool in the medical record saw 14-day treatment failure in 2.2% of 4,693 patients against 2.0% of 4,654 without it, an adjusted odds ratio of 0.77 (95% CI 0.55 to 1.08); the tool was safe but did not significantly reduce treatment failure.
- Model
- AI Consult (GPT-4o)
- Field
- Medicine
- Date
- 2026-06-26
- Human collaborators
- Ambrose Agweyu, Robert Korom, Xiaoxuan Liu, Bilal A. Mateen
What was found
AI Consult is built into the cloud medical record Penda Health uses and reviews the clinician's notes, diagnosis and treatment at decision points. 103 clinical officers, the diploma-trained clinicians who staff much of Kenyan primary care, were randomised: 52 with the tool and 51 without. 9,691 patients were enrolled between April and July 2025. The primary outcome was an expert-adjudicated composite of treatment failure within 14 days. No serious adverse event was judged related to the tool, and independent review of adverse events found no safety signal. The trial was funded by the Gates Foundation and registered with the Pan-African Clinical Trials Registry (202502499779176). An earlier 2025 quality-improvement study at 15 Penda clinics, run with OpenAI, reported 16% fewer diagnostic errors and 13% fewer treatment errors across 39,849 visits by clinicians who had the tool, as rated by independent physicians; that measured errors in documented care, a different and softer outcome than this trial's.
Novelty check
Europe PMC, (large language model OR LLM OR GPT*) AND randomised, with primary care, clinical decision support or clinic in the title, 2023 to 25 June 2026: earlier randomised trials gave clinicians an LLM on written cases (Goh et al., JAMA Network Open, 2024), tested patient-facing chatbots (Nature Health, January 2026), or measured process outcomes. This search found no earlier randomised trial of clinician-facing LLM decision support with a patient outcome as the primary endpoint.
Caveats and known objections
A negative primary result, recorded as such. One clinic network in one country. Treatment failure is a hard but rare outcome, about 2%, and the trial may have been too small to detect a modest benefit, which the authors say is probably the most any benefit would be. The unadjusted rates point slightly the other way from the adjusted odds ratio. Penda Health staff, who run the clinics and the tool, are among the authors. The earlier OpenAI study's error reductions are a different result on a softer endpoint and are not confirmed by this trial.
Nobody outside the lab has checked this yet.
Reading the primary source closely enough to say whether it supports the claim counts as a check, and you are credited on the entry.
Or on GitHub: submit a check challenge the grade send a correction or send a pull request
Flag this for triage
Signals order the review queue and nothing else. They are never published, and they never move a grade: that takes a citation.
Entry history (1 event)
- AddedEntered the registry graded Peer reviewed and AI-assisted.
Entries are never deleted. A grade that does not hold up is downgraded on the record, with the reason beside it.
Graded peer reviewed for verification and ai-assisted for autonomy. What these mean.
Cite this entry
whataifound.org. (2026). Cluster-randomised trial of an LLM decision-support tool in Kenyan primary care finds no reduction in treatment failure. whataifound.org: A Registry of AI Scientific and Mathematical Discoveries. https://whataifound.org/finding/2026-06-26-penda-ai-consult-cluster-rctBibTeX
@misc{whataifound-kemriwellcometrustresearchprogramme-2026-rct,
title = {Cluster-randomised trial of an LLM decision-support tool in Kenyan primary care finds no reduction in treatment failure},
author = {{whataifound.org}},
year = {2026},
howpublished = {whataifound.org: A Registry of AI Scientific and Mathematical Discoveries},
note = {Result by KEMRI-Wellcome Trust Research Programme / Penda Health / PATH. Verification: Peer reviewed. Autonomy: AI-assisted.},
url = {https://whataifound.org/finding/2026-06-26-penda-ai-consult-cluster-rct}
}