Posterboard#: B0141 Abstract Number: 2762 - B0141 AuthorBlock: Aaron S. Coyner1, Ryan Swan1, James M. Brown2, Jayashree Kalpathy-Cramer2, Sang Jin Kim1, J. Peter Campbell1, Karyn Jonas3, R.V. Paul Chan3, Susan Ostmo1, Michael F. Chiang1 1Oregon Health & Science University, Portland, Oregon, United States;2MGH/Harvard Medical School, Boston, Massachusetts, United States;3University of Illinois College of Medicine, Chicago, Illinois, United States; DisclosureBlock: Aaron S. Coyner, None; Ryan Swan, None; James M. Brown, None; Jayashree Kalpathy-Cramer, None; Sang Jin Kim, None; J. Peter Campbell, None; Karyn Jonas, None; R.V. Paul Chan, None; Susan Ostmo, None; Michael F. Chiang, Clarity Medical Systems Code S (Non-remunerative) , Novartis Code C (Consultant) , NIH R01 EY019474 Code F (Financial Support) , NIH P30EY010572 Code F (Financial Support) , NSF SCH-1622679 Code F (Financial Support) Purpose Accurate image-based ophthalmic diagnosis relies on clarity of fundus images. This has important implications for ophthalmic diagnosis, and for emerging methods such as telemedicine and computer-based image analysis. The purpose of this study was to implement a convolutional neural network (CNN) for automatically assessing the quality of fundus images in retinopathy of prematurity (ROP). Methods 5,204 wide-angle fundus images were collected from preterm infants during routine ROP screening examinations. Images were assessed for quality via ability to diagnose ROP accurately, and labeled “acceptable” or “not acceptable.” Acceptable images included only those that were deemed acceptable for ROP diagnosis by the majority of image graders. 2,550 images were used for training, 1,000 for validation, and 1,624 for testing. Additionally, 30 images of varying quality were selected for review by six individual expert graders who performed pairwise comparisons between images to rank them from worst to best quality. An overall expert consensus rank was developed from individual rankings. A CNN was trained using dropout and early stopping to avoid overfitting. For the set of 30 images, rank was established using the predicted probabilities of each image belonging to the acceptable image class. Results CNN training was halted when validation accuracy for identifying acceptable vs. not acceptable images reached 95%. Test set accuracy was 94.5%, with area under the receiver operating curve (AUROC) equal to 0.972, and area under the precision-recall curve (AUPR) equal to 0.975. Individual expert ranks were highly correlated with one another (correlation coefficient [CC] 0.89-0.94) and with the consensus rank (CC 0.94-0.98). The CNN image rank correlated with individual expert ranks (CC 0.75- 0.82) and the consensus rank (CC 0.77). Conclusions A CNN can accurately distinguish acceptable vs. not acceptable quality retinal fundus photos acquired from preterm infants during routine ROP examinations. The CNN ranks images similarly to expert graders, which suggests that similar image features may be used for quality assessment. Future directions will explore the features used for quality assessment by the CNN.