We propose an image coding scheme that compresses image into semantically scalable bitstream using deep neural networks. This scheme is expected to support intelligent analysis when the bitstream is partially decoded, as well as high-fidelity reconstruction of image when the bitstream is completely decoded. We implement such a semantically scalable image coding scheme based on semantic map. In the proposed scheme, the original image is firstly semantically segmented and the semantic map is compressed as the base layer. Then, the original image is segmented into several individual objects according to the semantic map, and each object is coded separately. A recurrent neural network-based encoder is used to compress these objects at several quality levels. At the decoder side, the semantic map can be directly applied for intelligent analysis. A generative adversarial network is used to synthesize a rough image using the semantic map. If user is interested in a certain object, more bits can be transmitted to enhance the quality of the object. Experimental results show that the proposed method achieves comparable compression performance with JPEG2000 at high bit rates, while facilitates intelligent analysis at low bit rates.