Traditionally, the habitable zone is defined as the region around a star in which liquid water can be stable on a planetary surface. At first these assessments considered planets on circular orbits. More recent investigations into nonzero orbital eccentricities found that the limits of the habitable zone should reflect the orbit-averaged flux a planet receives. However, all these studies assumed the potentially habitable planet is isolated. If additional planets are in the system, gravitational interactions between planets can cause eccentricity oscillations on timescales of 103 - 106 years. Furthermore, the known multi-planet systems (generally consisting of giant planets) appear to undergo large amplitude eccentricity oscillations. Such large variations produce variations in orbit-averaged flux, and hence may impact habitability. We show that plausible architectures of rocky planet systems can indeed lead to orbits with large eccentricity cycles. Moreover, some planets could cross either the inner or outer habitable zone boundaries due to these oscillations. Even more complicated behavior is possible in the habitable zones of M stars, where tidal effects are also important. We present a few interesting cases in which orbital architecture produces dramatic time-varying insolation, and hence impacts habitability.