The ability to extract the discriminative features remains a fundamental task of object detection, especially for small objects. Many mainstream object detection models, use the feature pyramids structure, a kind of fusion approaches, to predict objects of different scales. This traditional fusion strategy aims to merge different feature maps by linear operation, which does not allow the model to learn the complementary relationship between spatial information and semantic information. To address this problem, we develop a non-linear embedded network (NlENet) to achieve multi-scale fusion, which can learn the potential complementary relationship through end-to-end autonomous learning and get a more accurate performance. There are three main blocks in this proposed network, residual convolution unit (RCU), multi-resolution fusion and chained residual pooling. Due to the flexibility of the NlENet, we can embed it into many mainstream detection frameworks with few modification. We confirm that our fusion network can extract richer and more accurate features and achieve a better object detection performance on the COCO2017 dataset.