The segmentation of diversified roads and buildings from high-resolution aerial images is essential for various applications, such as urban planning, disaster assessment, traffic congestion management, and up-to-date road maps. However, a major challenge during object segmentation is the segmentation of small-sized, diverse shaped roads, and buildings in dominant background scenarios. We introduce O-SegNet- the robust encoder and decoder architecture for objects segmentation from high-resolution aerial imagery data to address this challenge. The proposed O-SegNet architecture contains Guided-Attention (GA) blocks in the encoder and decoder to focus on salient features by representing the spatial dependencies between features of different scales. Further, GA blocks guide the successive stages of encoder and decoder by interrelating the pixels of the same class. To emphasize more on relevant context, the attention mechanism is provided between encoder and decoder after aggregating the global context via an 8 Level Pyramid Pooling Network (PPN). The qualitative and quantitative results of the proposed and existing semantic segmentation architectures are evaluated by utilizing the dataset provided by Kaiser et al. Further, we show that the proposed O-SegNet architecture outperforms state-of-the-art techniques by accurately preserving the road connectivity and structure of buildings.