Diamond anvil cells allow the behavior of materials to be studied at pressures up to hundreds of gigapascals in a small and convenient instrument. However, physical access to the sample is impossible once it is pressurized. We show that optical tweezers can be used to hold and manipulate particles in such a cell, confining micron-sized transparent beads in the focus of a laser beam. Here, we use a modified optical tweezers geometry, allowing us to trap through an objective lens with a higher working distance, overcoming the constraints imposed by the limited angular acceptance of the anvil cell. We demonstrate the effectiveness of the technique by measuring water's viscosity at pressures of up to 1.3 GPa. In contrast to previous viscosity measurements in anvil cells, our technique measures absolute viscosity and does not require scaling to the accepted value at atmospheric pressure. This method could also measure the frequency dependence of viscosity as well as being sensitive to anisotropy in the medium's viscosity.