Semantic segmentation networks focus on the scene parsing of an unrestricted open scene. The typical segmentation architectures are stacks consisting of convolutional layers, which are used to extract semantic features. The feature map dimension is sharply changed at sampling units for most of networks, which ensure effective propagation of the gradient in deep nets. In this article, we proposed a state-of-the-art network model named Fully Convolutional Pyramidal Networks (FC-PRNet), which employs pyramidal residual structure to change the feature map dimension at all convolutional layers. This design is an effective way of improving generalization ability and optimizing parameters, and FC-PRNet could achieve excellent capability of semantic extraction. We used urban scene benchmark CamVid and KITTI dataset to test our network, the experimental results show that FC-PRNet achieves better results without any pre-training or post-treatment module. Moreover, due to smart construction of pyramidal residual structures, FC-PRNet has less parameters than other existing networks trained on these datasets.