Medical image description is an important problem in content-based medical image retrieval. Hierarchical medical image semantic features description model is proposed according to the main sources to get semantic features currently. Based on it, medical image description model combining low-level features and semantic features is proposed. Experimental results show that additional text taking as part of semantic features improves the image retrieval precision and recall.