Functional neuroimaging studies have indicated the involvement of separate brain areas in three distinct attention systems: alerting, orienting, and executive control (EC). However, the structural correlates underlying attention remains unexplored. Here, we utilized graph theory to examine the neuroanatomical substrates of the three attention systems measured by attention network test (ANT) in 65 healthy subjects. White matter connectivity, assessed with diffusion tensor imaging deterministic tractography was modeled as a structural network comprising 90 nodes defined by the automated anatomical labeling (AAL) template. Linear regression analyses were conducted to explore the relationship between topological parameters and the three attentional effects. We found a significant positive correlation between EC function and global efficiency of the whole brain network. At the regional level, node-specific correlations were discovered between regional efficiency and all three ANT components, including dorsolateral superior frontal gyrus, thalamus and parahippocampal gyrus for EC, thalamus and inferior parietal gyrus for alerting, and paracentral lobule and inferior occipital gyrus for orienting. Our findings highlight the fundamental architecture of interregional structural connectivity involved in attention and could provide new insights into the anatomical basis underlying human behavior.