Abstract This work explores the use of deep convolutional neural networks for high resolution remote sensing imagery segmentation. Encoder-decoder frameworks are popular in semantic image segmentation. However, encoder-decoder models face two main problems. The one is structural stereotype which is receptive fields imbalance rooted in this kind of frameworks. The other is insufficient learning that deeper neural networks tend to encounter the notorious problem of vanishing gradients. Structural stereotype leads to unfair learning and inhomogeneous reasoning. We are the first to reveal the problem and propose ensemble training and inference strategies to suppress the adverse consequences of structural stereotype as far as possible. To alleviate the problem of insufficient learning, we propose a novel residual architecture for encoder-decoder models. The proposed method yields state-of-the-art performances on the ISPRS 2D semantic labeling contest benchmark.