We present a new method to realize a vector magnetometer that has been optically referenced to an external coordinate system. The vector magnetometer is made up of three Hall sensors attached to the orthogonal faces of a glass cube. When placed in a homogeneous magnetic field, rotating the vector magnetometer around two axes allows the orientation of each sensor to be determined with respect to the glass cube. The orientation of the glass cube is detected by an autocollimator and optically referenced to an external coordinate system. A direction uncertainty below $130~\mu $ rad (which is traceable to angle standards) is achieved.