AbstractFacial expression recognition (FER) is a significant research task in the computer vision field. In this paper, we present a novel network FaceCaps for facial expression recognition with the following novel characteristics: an embedding structure based on a Capsule network which encodes relative spatial relationships between features; incorporates the feature polymerization property of FaceNet, thus offering a more efficient approach to discriminate complex facial expressions; a target reconstruction loss as a better regularization term for Capsule networks. Experimental results on both lab‐controlled datasets (CK+) and real‐world databases (RAF‐DB and SFEW 2.0) demonstrate that the method significantly outperforms the state‐of‐the‐art.