Pivotal trial of an autonomous AI diagnostic for diabetic retinopathy (IDx-DR)
In a prospective trial at 10 US primary care offices, an AI system that grades retinal photographs with no clinician reading them detected more-than-mild diabetic retinopathy with 87.2% sensitivity and 90.7% specificity against a specialist reading-centre reference, exceeding every prespecified endpoint; the results were the basis for the first FDA authorization of an autonomous AI diagnostic.
- Model
- IDx-DR
- Field
- Medicine
- Date
- 2018-08-28
- Human collaborators
- Michael D. Abràmoff, James C. Folk
What was found
900 people with diabetes and no known retinopathy were enrolled, of whom 819 could be fully analysed. Existing primary care staff, after standardised training, took the photographs with instant image-quality feedback from the system, which then returned a disease-level result, positive or negative for more-than-mild retinopathy or diabetic macular oedema, with no clinician interpreting the images. The reference standard was widefield stereoscopic photography and macular OCT graded by the Wisconsin Fundus Photograph Reading Center. Prespecified superiority endpoints were sensitivity above 85% and specificity above 82.5%; imageability was 96.1%. The FDA granted De Novo authorization in April 2018, before the paper appeared. Later versions were cleared by 510(k), including IDx-DR v2.3 in 2022.
Novelty check
Europe PMC, titles (artificial intelligence OR deep learning OR autonomous) AND diabetic retinopathy AND (prospective OR pivotal, or clinical-trial publication type), before 28 August 2018: no hits. Retrospective deep-learning grading of retinal photographs had been published (Gulshan et al., JAMA, 2016). What was new was a prospective, preregistered trial in primary care with the AI as the only reader, and the authorization that followed.
Caveats and known objections
Funded by IDx, whose founder, Michael Abràmoff, led the study; the reference grading was done by an independent reading centre. This entry found no published real-world accuracy study of the system by an unaffiliated group. Sensitivity of 87% means roughly one referable case in eight was missed against the reference standard, at the 24% prevalence of this trial population. Autonomy is graded autonomous for the diagnostic read only: who was screened, and what happened after a referral, followed a pathway clinicians designed.
Trial registration
That a study was registered, with the outcomes it declared and the date it declared them, so a published endpoint can be checked against the one planned. It does not certify the results, their analysis, or any clinical benefit; registry records are submitted by the sponsor and are not peer reviewed.
Regulatory authorization
DEN180001
That the FDA authorized the device for marketing in the US for a stated indication, on the evidence the manufacturer submitted. Most AI devices reach the market by 510(k), which requires substantial equivalence to an earlier device rather than new clinical evidence, so an authorization is not evidence of clinical benefit.
De Novo classification, 11 April 2018. Digital Diagnostics now holds the later 510(k) clearances, K203629 (2021) and K213037 (IDx-DR v2.3, 2022).
Nobody outside the lab has checked this yet.
Reading the primary source closely enough to say whether it supports the claim counts as a check, and you are credited on the entry.
Or on GitHub: submit a check challenge the grade send a correction or send a pull request
Flag this for triage
Signals order the review queue and nothing else. They are never published, and they never move a grade: that takes a citation.
Entry history (1 event)
- AddedEntered the registry graded Peer reviewed and Autonomous.
Entries are never deleted. A grade that does not hold up is downgraded on the record, with the reason beside it.
Graded peer reviewed for verification and autonomous for autonomy. What these mean.
Cite this entry
whataifound.org. (2018). Pivotal trial of an autonomous AI diagnostic for diabetic retinopathy (IDx-DR). whataifound.org: A Registry of AI Scientific and Mathematical Discoveries. https://whataifound.org/finding/2018-08-28-idx-dr-autonomous-diagnosticBibTeX
@misc{whataifound-idx-2018-diagnostic,
title = {Pivotal trial of an autonomous AI diagnostic for diabetic retinopathy (IDx-DR)},
author = {{whataifound.org}},
year = {2018},
howpublished = {whataifound.org: A Registry of AI Scientific and Mathematical Discoveries},
note = {Result by IDx / University of Iowa. Verification: Peer reviewed. Autonomy: Autonomous.},
url = {https://whataifound.org/finding/2018-08-28-idx-dr-autonomous-diagnostic}
}