This study investigates the influence of CSF relaxin (RLN) on the association between smoking and sleep quality, considering previous findings linking smoking and RLN with psychiatric conditions.In a case-control study of 168 Chinese adult males (70 smokers, 98 non-smokers), levels of relaxin in cerebrospinal fluid (CSF) were measured. Sleep quality was assessed using the Pittsburgh Sleep Quality Index (PSQI), comprising seven scales. Logistic regression and mediation models analyzed the relationships between nicotine dependence, PSQI scores, and CSF relaxin. Logistic regression examined the interaction of nicotine dependence and relaxin gene on PSQI subdimension scores.Smokers exhibited more severe sleep problems in PSQI total score and four PSQI subdimension scores (p < 0.05). CSF relaxin levels were significantly higher in smokers (20.7 ± 7.0 vs 16.3 ± 6.5, p < 0.001) and correlated closely with PSQI total score (r = 0.275, p < 0.001). Logistic regression found that CSF relaxin associated with PSQI subdimension scores, particularly in sleep disturbance (OR = 3.07 (1.61-5.99), adjusted p < 0.01). Mediation analysis indicated relationship between nicotine dependence and PSQI total score, with CSF relaxin as a mediator, and the indirect effect accounted for 25% of the total effect (Indirect effect = 0.124 (0.021-0.223), Total effect = 0.494 (0.193-0.807)). Additionally, polymorphisms in gene of relaxin and its receptors were closely tied to smoking behaviors and sleep quality (p < 0.05).CSF relaxin levels were significantly elevated in smokers and closely associated with PSQI subdimension scores, particularly with the sleep disturbance subdimension score. Moreover, CSF relaxin mediated the relationship between nicotine dependence and sleep quality. Polymorphisms (RLN3 rs12327666, rs1982632, and rs7249702, RLN3R1 rs35399, and RLN3R2 rs11264422) also played a role in smoking behaviors or sleep quality.