Abstract Background Few studies have investigated the relationship between neighborhood vulnerability and health-related quality of life (HRQOL) in the childhood cancer population. This study evaluated the impact of neighborhood vulnerability on HRQOL among adult survivors of childhood cancer. Methods This cross-sectional study included 4393 adult survivors of childhood cancer from the St Jude Lifetime Cohort Study. At the baseline (2007-2020), HRQOL was assessed using the SF36v2’s physical and mental components summaries (PCS and MCS). Neighborhood vulnerability was assessed using the overall, domain, and indicator-specific scores of the Social Vulnerability Index (SVI) and Minority Health SVI (MHSVI). Multivariable logistic regression was used to evaluate associations of neighborhood vulnerability (quartiles: Q1–Q4) with impaired HRQOL (1SD below the norm), adjusting for diagnosis, demographics, personal socioeconomic status (SES), lifestyle, and chronic health condition burden. Interactions of SVI and MHSVI with personal SES on impaired HRQOL were analyzed. Results Among survivors, 51.9% were male, averaging 30.3 years of age at evaluation and 21.5 years since diagnosis. Comparing neighborhoods with higher vs lower vulnerability (Q4 vs Q1), overall (odds ratio [OR] = 1.60, 95% confidence interval [CI] = 1.19 to 2.16) and domain-specific vulnerability (socioeconomic: OR = 1.59, 95% CI = 1.18 to 2.15; household composition: OR = 1.54, 95% CI = 1.16 to 2.06; housing and transportation: OR = 1.33, 95% CI = 1.00 to 1.76; medical vulnerability: OR = 1.60, 95% CI = 1.22 to 2.09) were significantly associated with impaired PCS, but not MCS. Residing in neighborhoods lacking urgent care clinics was significantly associated with impaired PCS (OR = 1.39, 95% CI = 1.08 to 1.78). Having lower vs higher personal education and living in higher vulnerability neighborhoods were associated with more impaired PCS (Pinteraction = .021). Conclusions Specific aspects of neighborhood vulnerability increase the risk for impaired physical HRQOL. Addressing these neighborhood factors is essential to enhance the HRQOL of survivors.