Microscopic variations in material stiffness play a vital role in cellular scale biomechanics, but are difficult to measure in a natural 3D environment. Brillouin microscopy is a promising technology for such applications, providing non-contact label-free measurement of longitudinal modulus at microscopic resolution. Here we develop heterodyne detection to measure Brillouin scattering signals in a confocal microscope setup, providing sensitive detection with excellent frequency resolution and robust operation in the presence of stray light. The functionality of the microscope is characterized and validated, and the imaging capability demonstrated by imaging structure within both a fibrin fiber network and live cells.