The introduction of an optical resonator can enable efficient and precise interaction between a photon and a solid-state emitter. It facilitates the study of strong light-matter interaction, polaritonic physics and presents a powerful interface for quantum communication and computing. A pivotal aspect in the progress of light-matter interaction with solid-state systems is the challenge of combining the requirements of cryogenic temperature and high mechanical stability against vibrations while maintaining sufficient degrees of freedom for in-situ tunability. Here, we present a fiber-based open Fabry-Pérot cavity in a closed-cycle cryostat exhibiting ultra-high mechanical stability while providing wide-range tunability in all three spatial directions. We characterize the setup and demonstrate the operation with the root-mean-square cavity length fluctuation of less than $90$ pm at temperature of $6.5$ K and integration bandwidth of $100$ kHz. Finally, we benchmark the cavity performance by demonstrating the strong-coupling formation of exciton-polaritons in monolayer WSe$_2$ with a cooperativity of $1.6$. This set of results manifests the open-cavity in a closed-cycle cryostat as a versatile and powerful platform for low-temperature cavity QED experiments.

10 pages, 8 figures