Parkinson’s Disease is the second most common neurodegenerative disease in the United States, with no single, definite diagnostic test currently available. Retinal image analysis represents a promising route to ameliorate these issues: the retina being part of the brain motivates the hypothesis that changes in the brain caused by neurodegenerative diseases may have retinal biomarkers identifiable from ophthalmic images. In this work, we collected a dataset of Optical Coherence Tomography images for both Parkinson’s Disease cases (349 patients, 60,132 images) and three control cohorts with a 1:5 image ratio (total images for controls: 300,660): a random group (2,388 subjects), a gender matched group (1,785 subjects), and a gender and age matched group (2,988 subjects). We trained two deep learning models (ResNet50 and RETFound) to predict which images are taken from patients with Parkinson’s, obtaining Area under the ROC curve values of 0.74, 0.62, and 0.53 for RETFound on the random, unmatched cohort, on the gender matched cohort, and on the age and gender matched cohort, respectively. We hypothesized that age could be used by our trained models as a proxy for PD prediction, and we further verified it by computing the Spearman correlation coefficient between age and the models’ probability of PD, reporting 0.68, 0.64, and 0.15 for the previously mentioned model and cohorts. Our initial analysis highlighted the challenges of predicting PD from retinal images, as age represents a strong confounder that can be used as a proxy for the diagnostic task of interest. In the future, we will explore alternative model architectures, training paradigms, and additional data sources in terms of modalities (e.g., clinical notes or EHR data) and institutions (via Federated Learning).