Every natural language has a large set of words, which, when these are used in a piece of text, may vary in sense denotation. It has been noted that for ages that context, where these words are found to be used, can play an explicit and active role to influence the words to deviate from the original sense to generate new senses. And these new senses are usually contextualized or context based. The newly acquired senses often vary from the original senses of words usually derived from their origin or etymology. This phenomenon of words in a natural language has several long-standing problems relating to understanding and cognition of word meanings, using word meanings in machine learning as well as presenting word meanings in the dictionary. In this chapter, we shall describe the ‘corpus-based approach’ to deal with the phenomenon of sense understanding of words. We shall try to discuss how the information extracted from a corpus can help us attribute meaning of words to their unique distributional information and contextual environments. While distributional information refers to the frequency distribution of the senses of the words, contextualized environments refer to the setting of occurrence of the words in some particular textual situation. We shall also try to show how we can extract necessary information from the contexts of the words used in a corpus to pick up necessary cues for understanding the actual contextualized senses. To substantiate our arguments, we shall try to draw supporting data, information, and evidence from the Bangla corpus of written texts developed in the TDIL project.