The corpus cavernosum is the most important structure for penile erection, and its dysfunction causes many physiological and psychological problems. However, its cellular heterogeneity and signalling networks at the molecular level are poorly understood because of limited access to samples. Here, we profile 64,993 human cavernosal single-cell transcriptomes from three males with normal erection and five organic erectile dysfunction patients. Cell communication analysis reveals that cavernosal fibroblasts are central to the paracrine signalling network and regulate microenvironmental homeostasis. Combing with immunohistochemical staining, we reveal the heterogeneity and describe a detailed spatial distribution map for each fibroblast, smooth muscle and endothelial subcluster in the corpus cavernosum. Furthermore, comparative analysis and related functional experiments identify candidate regulatory signalling pathways in the pathological process. Our study provides an insight into the human corpus cavernosum microenvironment and a reference for potential erectile dysfunction therapies.