Algorithm for Diagnosing Diabetic Retinopathy Could Halve Human Graders

An artificial intelligence (AI) enabled algorithm for automated diabetic retinopathy (DR) detection demonstrated safe levels of sensitivity with specificity that could halve the workload for human graders, according to findings published in the British Journal of Ophthalmology.

Researchers conducted a prospective study to evaluate the performance of an AI algorithm to triage retinal images from the English Diabetic Eye Screening Programme (DESP) into test-positive/technical failure vs test-negative, using human grading following a standard national protocol as the reference standard. The study involved the manual grading of retinal images from 30,405 consecutive screening episodes from 3 English DESPs, followed by an automated process with machine learning enabled software, EyeArt v2.1® (Eyenuk Inc.). Researchers determined screening performance (sensitivity, specificity) and diagnostic accuracy (95% CIs) using human grades as the reference standard.

The study found 95.7% mean sensitivity of EyeArt (95%CI, 94.8% to 96.5%) for referable retinopathy, which comprises sensitivities of 98.3% (95%CI, 97.3% to 98.9%) for mild-to-moderate nonproliferative retinopathy  with referable maculopathy and 100% (95%CI, 98.7% to 100%) for moderate-to-severe non-proliferative retinopathy and 100% (95% CI, 97.9% to 100%) for proliferative disease. In 68% of cases, with a specificity of 54.0%, Eyert agreed with the human grade of no retinopathy, when combined with non-referable retinopathy.

The researchers explain that, though there was a small difference in the EyeArt software’s performance across the 3 centres included in the study, this was to be expected given patient- and centre-specific characteristics, and overall, EyeArt was effective in detecting meaningful retinopathy.

“It is noteworthy that detection rates were high and likelihood ratios for referable retinopathy were stable across centres,” the study explains. “The sensitivity of EyeArt is high for detection of sight-threatening retinopathy and exceeds any published AI algorithm to date. Importantly, the specificity is sufficient to make an even greater cost-saving to the [National Health Service] than previously described.”

The study notes that diabetes is a health challenge for every country, with an estimation of 625 million people to have been diagnosed globally by 2045. Thus, the use of AI for DR screening is a new and promising possibility, given that costs and quality of eye care will become more important as this number rises.

Reference

Heydon P, Egan C, Bolter L, et al. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30,000 patients. Br J Ophthalmol. 2021;105(5):723-728. doi:10.1136/bjophthalmol-2020-316594