Identifying relationships between structural and functional networks is crucial for understanding the large-scale organization of the human brain. The potential contribution of emerging techniques like functional near-infrared spectroscopy to investigate the structure-functional relationship has yet to be explored. In our study, using simultaneous Electroencephalography (EEG) and Functional near-infrared spectroscopy (fNIRS) recordings from 18 subjects, we characterize global and local structure-function coupling using source-reconstructed EEG and fNIRS signals in both resting state and motor imagery tasks, as this relationship during task periods remains underexplored. Employing the mathematical framework of graph signal processing, we investigate how this relationship varies across electrical and hemodynamic networks and different brain states. Results show that fNIRS structure-function coupling resembles slower-frequency EEG coupling at rest, with variations across brain states and oscillations. Locally, the relationship is heterogeneous, with greater coupling in the sensory cortex and increased decoupling in the association cortex, following the unimodal to transmodal gradient. Discrepancies between EEG and fNIRS are noted, particularly in the frontoparietal network. Cross-band representations of neural activity revealed lower correspondence between electrical and hemodynamic activity in the transmodal cortex, irrespective of brain state while showing specificity for the somatomotor network during a motor imagery task. Overall, these findings initiate a multimodal comprehension of structure-function relationship and brain organization when using affordable functional brain imaging.