Quantitative and qualitative fuzzy measures have been proposed to hesitant fuzzy sets (HFSs) from different points. However, few of the existing HFSs fuzzy measures refer to the grey relational analysis (GRA) theory. Actually, the GRA theory is very useful in the fuzzy measure domain, which has been developed for such the intuitionistic fuzzy sets. Therefore, in this paper, we apply the GRA theory to the HFSs and interval-valued hesitant fuzzy sets (IVHFS) domain. We propose the HFSs grey relational degree, HFSs slope grey relational degree, HFSs synthetic grey relational degree and IVHFSs grey relational degree, which describe the closeness, the variation tendency and both the closeness and variation tendency of HFSs and closeness of IVHFSs, respectively, greatly enriching the fuzzy measures of HFSs. Furthermore, with the help of the TOPSIS method, we develop the grey relational based MADM methodology to solve the HFSs and IVHFSs MADM problems. Finally, combined with two practical MADM examples about energy policy selection with HFSs information and emergency management evaluation with IVHFSs information, we obtain the most desirable decision results, and compared with the previous methods, the validity, effectiveness and accuracy of the proposed grey relational degree for HFSs and IVHFSs are demonstrated in detail.