Psychological adaptation to thermal environments might play an important role in the human thermal comfort. However, effects of psychological thermal adaptation on thermal comfort have been rarely studied. To investigate building occupants' psychological adaptation to indoor climates in hot summer and cold winter, a long-term tracking survey within a year in a naturally ventilated building was conducted on a group of occupants from a college class in Nanjing situated in the Hot Summer and Cold Winter zone. Indoor and outdoor environment parameters were measured, and at the same time occupants' clothing insulation, physical activity level, and thermal sensation were investigated using questionnaires. Results indicate building occupants adapted to their hot environments in summer and cold environments in winter by psychological adaptation. Occupants' neutral skin temperature was elevated in summer while lowered in winter due to psychological adaptive processes. The thermal sensitivity to skin/operative temperature change is influenced by psychological adaptation, and the higher the psychological adaptation, the lower the sensitivity to skin/operative temperature change. Furthermore, psychological adaptation is affected by both the indoor climate and outdoor climate to which occupants are exposed.