The tensor product representation of quantum states leads to a promising variational approach to study quantum phase and quantum phase transitions, especially topological ordered phases which are impossible to handle with conventional methods due to their long range entanglement. However, an important issue arises when we use tensor product states (TPS) as variational states to find the ground state of a Hamiltonian: can arbitrary variations in the tensors that represent ground state of a Hamiltonian be induced by local perturbations to the Hamiltonian? Starting from a tensor product state which is the exact ground state of a Hamiltonian with $\mathbb{Z}_2$ topological order, we show that, surprisingly, not all variations of the tensors correspond to the variation of the ground state caused by local perturbations of the Hamiltonian. Even in the absence of any symmetry requirement of the perturbed Hamiltonian, one necessary condition for the variations of the tensors to be physical is that they respect certain $\mathbb{Z}_2$ symmetry. We support this claim by calculating explicitly the change in topological entanglement entropy with different variations in the tensors. This finding will provide important guidance to numerical variational study of topological phase and phase transitions. It is also a crucial step in using TPS to study universal properties of a quantum phase and its topological order.

10 pages, 6 figures