The widespread use of neural networks and their increasing complexity necessitate effective training algorithms to optimize their performance. While second-order methods like Scaled Conjugate Gradient (SCG) offer potential benefits by utilizing curvature infor- mation, standard SCG scales poorly with large datasets typical in modern deep learning. This thesis tackles the problem of adapting SCG for training deep neural networks on large datasets. We investigate SCG's behavior on benchmark tasks, identifying its strengths and limitations. Based on this analysis, we propose Mini-Batch SCG (MBSCG) and two training techniques, Reused-Batch and Batch-Overlap, designed to enhance scalability and convergence. Comparative experiments against Adam on MNIST, CIFAR-100, and SST-2 demonstrate the viability of our approach. Furthermore, interpretability studies reveal that SCG-based methods can induce distinct learned representations compared to Adam.