Psychiatric outcomes are commonly observed in individuals with repeated concussions, though their underlying mechanism is unknown. One potential mechanism linking concussion with psychiatric symptoms is inflammation-induced activation of the kynurenine pathway, which is thought to play a role in the pathogenesis of mood disorders. Here, we investigated the association of prior concussion with multiple psychiatric-related outcomes in otherwise healthy male and female collegiate-aged athletes (N = 212) with varying histories of concussion recruited from the community. Specially, we tested the hypotheses that concussion history is associated with worse psychiatric symptoms, limbic system structural abnormalities (hippocampal volume, white matter microstructure assessed using neurite orientation dispersion and density imaging; NODDI), and elevations in kynurenine pathway (KP) metabolites (e.g., Quinolinic acid; QuinA). Given known sex-effects on concussion risk and recovery, psychiatric outcomes, and the kynurenine pathway, the moderating effect of sex was considered for all analyses. More concussions were associated with greater depression, anxiety, and anhedonia symptoms in female athletes (ps ≤ 0.005) and greater depression symptoms in male athletes (p = 0.011). More concussions were associated with smaller bilateral hippocampal tail (ps < 0.010) and left hippocampal body (p < 0.001) volumes across male and female athletes. Prior concussion was also associated with elevations in the orientation dispersion index (ODI) and lower intracellular volume fraction in several white matter tracts including the in uncinate fasciculus, cingulum-gyrus, and forceps major and minor, with evidence of female-specific associations in select regions. Regarding serum KP metabolites, more concussions were associated with elevated QuinA in females and lower tryptophan in males (ps ≤ 0.010). Finally, serum levels of QuinA were associated with elevated ODI (male and female athletes) and worse anxiety symptoms (females only), while higher ODI in female athletes and smaller hippocampal volumes in male athletes were associated with more severe anxiety and depression symptoms (ps ≤ 0.05). These data suggest that cumulative concussion is associated with psychiatric symptoms and limbic system structure in healthy athletes, with increased susceptibility to these effects in female athletes. Moreover, the associations of outcomes with serum KP metabolites highlight the KP as one potential molecular pathway underlying these observations.