We have developed a four-channel optically pumped atomic magnetometer for magnetoencephalography (MEG) that incorporates a passive diffractive optical element (DOE). The DOE allows us to achieve a long, 18-mm gradiometer baseline in a compact footprint on the head. Using gradiometry, the sensitivities of the channels are < 5 fT/Hz1/2, and the 3-dB bandwidths are approximately 90 Hz, which are both sufficient to perform MEG. Additionally, the channels are highly uniform, which offers the possibility of employing standard MEG post-processing techniques. This module will serve as a building block of an array for magnetic source localization.