Abstract Micro-expression analysis is an interesting and challenging task in computer vision. It has inspired a series of possible applications in many areas such as police–criminal interrogation and important business negotiation. One of the most crucial step in a micro-expression recognition system is the extraction of well discriminating features. In this paper, we propose a new feature based on the fusion of motion boundary histograms (FMBH). This feature is generated by combining both the horizontal and the vertical components of the differential of optical flow as inspired from the motion boundary histograms (MBH). The proposed feature is then validated and evaluated through the leave-one-subject-out (LOSO) protocol for micro-expression recognition. Moreover, the proposed method is compared to state-of-the-art methods on four well-known databases CASME, CASME II, SMIC and CAS(ME)2. Comparative experimental results demonstrate that the proposed FMBH feature descriptor yields promising performance.