Abstract Deep Learning approach dragged the full attention of researcher in medical images due to their superior literature reported success and promising directions. Concluding the most discriminative set of features greatly represents a valuable guide for achieving the satisfaction performance of a medical diagnosing system. Despite, many methods proposed for such objective, the restricted Boltzmann machines outperform as they learn features directly from data, however they lack the optimal classification performance due to data complexity. Additionally, the contractive au-toencoder offers regularized term that explicitly increases the robustness of features representation and enhancement in overall performance. This paper proposes a novel deep learning framework for diagnosing female brain disorder from fMRI scans. The configuration combines the contractive autoencoder and the discriminative restricted Boltz-mann machine (DRBM) as we seek an improvement for the classification of fMRI. The demonstrated effectiveness of the contractive autoencoder supports fitting the probability distribution model of the DRBM and transfer learning to a deeper level. Our experimental indicates that the proposed model is able to detect female brain disorder with an accuracy of 88.17% and improved literature reported results on common issues.