We propose a new method for training convolutional neural networks (CNNs) and use it to classify glaucoma from fundus images. This method integrates reinforcement learning along with supervised learning and uses it for transfer learning. The training method uses hill climbing techniques via two different climber types, namely, “random movement” and “random detection,” integrated with a supervised learning model through a stochastic gradient descent with momentum model. The model was trained and tested using the Drishti-GS and RIM-ONE-r2 datasets having glaucomatous and normal fundus images. The performance for prediction was tested by transfer learning on five CNN architectures, namely, GoogLeNet, DenseNet-201, NASNet, VGG-19, and Inception-Resnet v2. A five-fold classification was used for evaluating the performance, and high sensitivities while maintaining high accuracies were achieved. Of the models tested, the DenseNet-201 architecture performed the best in terms of sensitivity and area under the curve. This method of training allows transfer learning on small datasets and can be applied for tele-ophthalmology applications including training with local datasets.