Based on the scale-invariance approach to finite plasticity, a two-back stress model is proposed to describe phenomenologically the evolution of substructure or texture development. Each back stress obeys its own evolution and rotational rules. Different types of hardening rules are obtained on the basis of crystal slip and a scale invariance argument to account for different types of substructure evolution. An anisotropic yield surface is used to consider directional hardening/softening and distortion of the yield surface. The extension to include viscoplastic effects is straightforward within the scale invariance approach. The model is applied to analyze experimental data for uniaxial loading, torsion of cylindrical specimens with fixed ends, and unconstrained torsion.