Environmental challenges are part of daily life for any individual. In fact, stress appears to be increasingly present in our modern, and demanding, industrialized society. Virtually every aspect of our body and brain can be influenced by stress and although its effects are partly mediated by powerful corticosteroid hormones that target the nervous system, relatively little is known about when, and how, the effects of stress shift from being beneficial and protective to becoming deleterious. Decades of stress research have provided valuable insights into whether stress can directly induce dysfunction and/or pathological alterations, which elements of stress exposure are responsible, and which structural substrates are involved. Using a broad definition of pathology, we here review the "neuropathology of stress" and focus on structural consequences of stress exposure for different regions of the rodent, primate and human brain. We discuss cytoarchitectural, neuropathological and structural plasticity measures as well as more recent neuroimaging techniques that allow direct monitoring of the spatiotemporal effects of stress and the role of different CNS structures in the regulation of the hypothalamic-pituitary-adrenal axis in human brain. We focus on the hypothalamus, hippocampus, amygdala, nucleus accumbens, prefrontal and orbitofrontal cortex, key brain regions that not only modulate emotions and cognition but also the response to stress itself, and discuss disorders like depression, post-traumatic stress disorder, Cushing syndrome and dementia.