We show that temperature compensation based on differential thermal expansion between sapphire and fused silica can be used to create a Fabry-Perot cavity with an exceptionally low coefficient of thermal expansion at low temperatures. We describe the design of such a cavity that utilizes shaped fused silica mirrors and a sapphire spacer. The geometry of the fused silica mirror was designed using a finite element model to have a small platform, giving a frequency temperature turning point of 16.6 K. The measured turning point was 16.2 K and the curvature was 6 x 10(-10) K(-2), both of which were consistent with the model.