Quantum inertial sensors test general relativity, measure fundamental constants, and probe dark matter and dark energy in the laboratory with outstanding accuracy. Their precision relies heavily on carefully choreographed quantum control of the atomic states with a collection of lasers, microwaves, and electric and magnetic fields. Making this technology available outside of the laboratory would unlock many applications, such as geophysics, geodesy, and inertial navigation. However, this requires an apparatus of reduced size, weight, and power use and increased robustness, modularity, and ease-of-use. Here, we describe the design and implementation of an in-vacuum electromagnet able to create the magnetic fields necessary for various quantum control operations, such as magneto-optical trapping or magnetic levitation to assist evaporative cooling. Placing the electromagnet inside the vacuum chamber has significant advantages, such as fast switching times that are not limited by induced current inside the vacuum chamber metal and reduced size, weight, and power usage. However, dissipating the heat produced typically requires complex designs that include bulky metal heatsinks or cooling using water or cryogens. Our design implements heatpipes in a compact, low-vibration, and robust apparatus, which uses a phase transition in the working fluid to achieve thermal conductivity that is more than 100 times larger than that of typical bulk metal. We show that the setup can conduct more than 50 W of thermal power in a configuration that provides ample optical access and is compatible with the ultrahigh vacuum requirements of atomic and molecular experiments.