Mitochondria act as hubs of numerous metabolic pathways. Mitochondrial dysfunctions contribute to altering the redox balance and predispose to aging and metabolic alterations. The sirtuin family is composed of seven members and three of them, SIRT3-5, are housed in mitochondria. They catalyze NAD+-dependent deacylation and the ADP-ribosylation of mitochondrial proteins, thereby modulating gene expression and activities of enzymes involved in oxidative metabolism and stress responses. In this context, mitochondrial sirtuins (mtSIRTs) act in synergistic or antagonistic manners to protect from aging and aging-related metabolic abnormalities. In this review, we focus on the role of mtSIRTs in the biological competence of reproductive cells, organs, and embryos. Most studies are focused on SIRT3 in female reproduction, providing evidence that SIRT3 improves the competence of oocytes in humans and animal models. Moreover, SIRT3 protects oocytes, early embryos, and ovaries against stress conditions. The relationship between derangement of SIRT3 signaling and the imbalance of ROS and antioxidant defenses in testes has also been demonstrated. Very little is known about SIRT4 and SIRT5 functions in the reproductive system. The final goal of this work is to understand whether sirtuin-based signaling may be taken into account as potential targets for therapeutic applications in female and male infertility.