Reading a doctor’s handwritten prescription is a challenge that most patients and some pharmacists face; an issue that, in some cases, lead to negative consequences due to wrong deciphering of the prescription. Part of the reason why doctor’s prescriptions are so difficult to decipher is that doctors make use of Latin abbreviations and medical terminology that most people don’t understand. This paper demonstrates how Artificial Neural Networks (ANN) is used to develop a system that can recognize handwritten English medical prescriptions. Using the Deep Convolution Recurrent Neural Network to train this supervised system, input images are segmented and processed to detect characters and classify them into the 64 different predefined characters. The results show that the proposed system yields good recognition rates and an accuracy of %98.