Age-related macular degeneration (AMD) is a retinal disorder affecting the elderly, and society’s aging population means that the disease is becoming increasingly prevalent. The vision in patients with early AMD is usually unaffected or nearly normal but central vision may be weakened or even lost if timely treatment is not performed. Therefore, early diagnosis is particularly important to prevent the further exacerbation of AMD. This paper proposed a novel automatic detection method of AMD from optical coherence tomography (OCT) images based on deep learning and a local outlier factor (LOF) algorithm. A ResNet-50 model with L2-constrained softmax loss was retrained to extract features from OCT images and the LOF algorithm was used as the classifier. The proposed method was trained on the UCSD dataset and tested on both the UCSD dataset and Duke dataset, with an accuracy of 99.87% and 97.56%, respectively. Even though the model was only trained on the UCSD dataset, it obtained good detection accuracy when tested on another dataset. Comparison with other methods also indicates the efficiency of the proposed method in detecting AMD.