More and more abundant Web images on the Internet make clients difficult seek the information they really need so that how to quickly and accurately retrieve their interested Web images is one of the most challenging tasks. The kernel idea of the model is that the text keyword features, visual content features, link information and other information of Web images are utilized together to reduce the semantic gaps of them in the Web image search processes. The automatic image annotation model is presented here, which effectively combines the generation model with the discriminant classification method. The Web image retrieval model in the paper has been designed on knowledge inference to seamlessly integrate both the Web image text features and the semantic features of Web images. The experiments have demonstrated that the established system here makes Web image retrieval more accurate and more rapid than the exciting ones.