Heavy metals contained in coal gangue can be released into the surrounding environment through various pathways during long-term accumulation, posing potential threats to human health. To effectively control and mitigate the health risks of heavy metals in the soil of coal gangue accumulation areas, this study focused on the Panyi coal gangue accumulation area in Huainan. The study involved collecting soil samples to determine the concentrations of Cd, Zn, Cu, Fe, Mn, Cr, Ni, and Pb and employed the positive matrix factorization model for quantitative analysis of the contributions from different pollution sources of soil heavy metals. This approach, coupled with the results of the source analysis and a health risk assessment model, evaluates the risks posed by specific sources and further identifies the spatial distribution characteristics of health risk contributions from these sources. The results showed that the average concentrations of soil heavy metals Cd and Zn were 4.65 and 2.16 times their background values, respectively, with the average values of other heavy metals all below background levels. Among them, Zn was the most influenced by human activities. Source analysis indicated that the sources of soil heavy metals in the study area were influenced by coal gangue accumulation pollution, traffic activity pollution, natural parent material soil formation, and agricultural activity pollution sources, with contribution rates of 27.5%, 16.4%, 30.4%, and 25.6%, respectively. The non-carcinogenic risk of soil heavy metals to children requires special attention. Based on the specific source-health risk assessment model analysis, agricultural activities contributed the most to the non-carcinogenic risk to children （59.3%）, identifying them as a priority source for control measures. Given the spatial distribution of health risks from various sources, recommendations include enhancing the resourceful use of coal gangue, collecting and treating leachate from coal gangue, and implementing environmental management strategies to reduce pesticide and fertilizer use.