Social image search becomes an active research field in recent years due to the rapid development in big data processing technologies. In the retrieval systems, text description/tags play a key role to bridge the semantic gap between low-level features and higher-level concepts, and so guarantee the reliable search. However, in practice manual tags are usually noisy and incomplete, resulting in a limited performance of image retrieval. To tackle this problem, we propose a probabilistic topic model to formalize the correlation of tags with visual features via the latent semantic topics. Our proposed approach allows us to effectively annotate and refine tags based on a Monte Carlo Markov Chain algorithm for approximate inference. Moreover, we present a measuring scheme using the refined tags and extracted topics for ranking the images. The experimental results from two large benchmark datasets show that our approach provides promising accuracy.