Strain gradient crystal plasticity models have been quite successful in describing size-dependent deformation. Various formulations have been suggested in the literature involving approaches that vary considerably in principle. In this respect it is important to associate certain classes of material models to specific deformation mechanisms and to investigate possible inherent restrictions. To this end, the scaling relation between dislocation substructures and the order (first order, second order) of strain gradients in the corresponding continuum formulation are discussed. Conclusions for isotropic and kinematic hardening on slip system level and their related orders in the strain gradients are drawn.