Twisted and orbifold formulations of lattice ${\cal N}=4$ super Yang-Mills theory which possess an exact supersymmetry require a $U(N)=SU(N)\otimes U(1)$ gauge group. In the naive continuum limit, the $U(1)$ modes trivially decouple and play no role in the theory. However, at non-zero lattice spacing they couple to the $SU(N)$ modes and can drive instabilities in the lattice theory. For example, it is well known that the lattice $U(1)$ theory undergoes a phase transition at strong coupling to a chirally broken phase. An improved action that suppresses the fluctuations in the $U(1)$ sector was proposed in arXiv:1505.03135 . Here, we explore a more aggressive approach to the problem by adding a term to the action which can entirely suppress the $U(1)$ mode. The penalty is that the new term breaks the $\mathcal{Q}$-exact lattice supersymmetry. However, we argue that the term is $1/N^2$ suppressed and the existence of a supersymmetric fixed point in the planar limit ensures that any SUSY-violating terms induced in the action possess couplings that also vanish in this limit. We present numerical results on supersymmetric Ward identities consistent with this conclusion.