Abstract Background This study investigated sex differences in the associations between Alzheimer’s disease (AD) biomarkers, cognitive performance, and decline in memory clinic settings. Methods 249 participants (females/males:123/126), who underwent tau-PET, amyloid-PET, structural MRI, and plasma glial fibrillary acidic protein (GFAP) measurement were included from Geneva and Lausanne Memory Clinics. Mann-Whitney U tests investigated sex differences in clinical and biomarker data. Linear regression models estimated the moderating effect of sex on the relationship between biomarkers and cognitive performance and decline. Sex differences in cognitive decline were further evaluated using longitudinal linear mixed-effect models with three-way interaction effects. Results Women and men present similar clinical features, amyloid, and neurodegeneration. Women had higher tau load and plasma levels of GFAP than men (p < 0.05). Tau associations with amyloid (standardized β = 0.54,p < 0.001), neurodegeneration (standardized β=-0.44,p < 0.001), and cognition (standardized β=-0.48,p < 0.001) were moderated by a significant interaction with sex. Specifically, the association between amyloid and tau was stronger among women than men (standardized β=-0.19,p = 0.047), whereas the associations between tau and cognition and between tau and neurodegeneration were stronger among men than in women (standardized β=-0.76,p = 0.001 and standardized β=-0.56,p = 0.044). Women exhibited faster cognitive decline than men in the presence of severe cortical thinning (p < 0.001). Conclusion Women showed higher tau load and stronger association between amyloid and tau than men. In individuals with high tau burden, men exhibited greater neurodegeneration and cognitive impairment than women. These findings support that sex differences may impact tau deposition through an upstream interplay with amyloid, leading to downstream effects on neurodegeneration and cognitive outcomes.