Deep learning has revolutionized the field of neural network models, offering limitless applications in various do- mains. This study focuses on Transfer Learning (TL), a technique leveraging pre-trained deep learning models trained on large datasets for image classification tasks. Specifically, this research explores the effectiveness of various transfer learning models in three medical image datasets: Alzheimer’s MRI images, Chest CT-Scan images, and Chest X-ray images. The main objective of this study is to assess and compare the performance of various TL models, including MobileNetV2, ResNet50, Xception, and InceptionV3, on the three medical image datasets. Additionally, a customized Convolutional Neural Network (CNN) model is developed to compare its performance against the pre-trained TL models. Each model was trained and evaluated on the three medical image datasets. The performance of the TL models was compared in terms of accuracy and training time. The results of this study revealed that ResNet50 consistently outperforms other TL models, demonstrating accurate predictions at the expense of longer training times. MobileNetV2 and InceptionV3 exhibit the fastest training times across all datasets, but they demonstrate poorer performance in certain datasets. The developed CNN model performs poorly in terms of accuracy and tends to overfit, indicating that creating a CNN model for medical image classification is not feasible in this study. The findings of this study offer valuable insights into the performance of TL models in medical image datasets. Researchers can utilize this information to make informed decisions when selecting TL models for medical imaging applications. Understanding the strengths and weaknesses of different TL models enhances the potential for accurate and efficient medical image classification. The insights gained from this study contribute to researchers’ understanding of selecting transfer learning models for medical imaging applications, aiding in the advancement of medical image analysis and diagnosis.