With modern scanning probe microscopes, it is possible to manipulate surface structures even at the atomic level. However, manipulation of nanoscale objects such as clusters is often more relevant and also more challenging due to the complicated interactions between the surface, cluster and apparatus. We demonstrate the manipulation of nanometer scale gold clusters on the NaCl(001) surface with a non-contact atomic force microscope, and show that the movement of clusters is in certain cases constrained to specific crystallographic directions. First principles calculations explain this kinetic anisotropy as the result of the cluster attaching to surface defects: cation vacancies allow the clusters to bond in such a way that they only move in one direction. Constraining the movement of clusters could be exploited in the construction of nanostructures or nanomechanical devices, and the manipulation signatures may also be used for identifying cluster-defect complexes.