In received signal strength fingerprints based indoor localization systems, the radio map built by labeled wireless fingerprints is easily outdated over time, while re-calibrating the overall radio map is time consuming. To avoid the tedious task, we propose to employ manifold alignment to label the current radio map from outdated radio map, with the constraint of the Hidden Markov Model trained by trajectories of the received signal strength readings. Manifold alignment can align the low-dimensional manifold structures of two different data sets and transfer knowledge across them. Transition matrix generated by Hidden Markov Model is used to constrain the alignment of manifolds. The proposed algorithms are tested in a real world ZigBee environment. Experiment results show that our method outperforms state-of-the-art transfer learning algorithms.