Fourier-domain mode-locked (FDML) optoelectronic oscillators (OEOs) are regarded as a promising candidate to generate linearly chirped microwave waveforms (LCMWs) with large time-bandwidth products. Nevertheless, up to date, the mode locking mechanism in FDML-OEOs is still not clear enough. Here, a comprehensive theoretical analysis is made to reveal the mode locking mechanism in FDML-OEOs. In particular, the phase relationship among numerous oscillation modes under stable oscillation is obtained. In addition, the FDML oscillation process originated from either noise or single-mode oscillation and is numerically simulated based on the model. Therefore, the initial oscillation process is comprehensively analyzed in the time domain, the Fourier domain, and the fractional Fourier domain, which provides a deep insight into the FDML oscillation process. Finally, the initial oscillation process of a FDML-OEO is captured in the experiment. The corresponding analysis is carried out to reveal the real mode locking mechanism, where the experimental results fit in with the theoretical and numerical results. This work provides a new approach for in-depth analysis of FDML-OEOs.