Spatial variation of material structures is a principal mechanism for creating and controlling spatially varying material properties in nature and engineering. While the spatially varying homogenized properties can be represented by scalar and vector fields on the macroscopic scale, explicit microscopic structures of constituent phases are required to facilitate the visualization, analysis, and manufacturing of functionally graded material (FGM). The challenge of FGM structure modeling lies in the integration of these two scales. We propose to represent and control material properties of FGM at macroscale using the notion of material descriptors, which include common geometric, statistical, and topological measures, such as volume fraction, correlation functions, and Minkowski functionals. At microscale, the material structures are modeled as Markov random fields (MRFs): we formulate the problem of design and (re)construction of FGM structure as a process of selecting neighborhoods from a reference FGM, based on target material descriptors fields. The effectiveness of the proposed method in generating a spatially varying structure of FGM with target properties is demonstrated by two examples: design of a graded bone structure and generating functionally graded lattice structures with target volume fraction fields.