Document summarization addresses the problem of presenting the information in a compact form to the readers. Different approaches to summarize documents have been proposed and evaluated in literature. Common research problems in multi-document summarization are Redundancy and Extraction of sentences; that are important and semantically linked with other sentences. With the combination of agglomerative hierarchical clustering and Latent Semantic Analysis (LSA); which measures semantic similarity between sentences and reduces dimensions by preserving only highly weighted vectors, we propose a novel multi document summarization approach. Latent Dirichlet Allocation Model is used to identify important topic terms in the resultant summary. We have used Recall Oriented Understudy for Gisting Evaluation (ROUGE) metric to evaluate our system against other state-of-the art techniques using Document Understanding Conference (DUC) dataset 2004. Experimental results show that there is substantial performance improvement using our system and it makes better summary as compared to the other state-of-art techniques.