Corpora tagged with part-of-speech and phrase structure information have been used for both exploratory data analysis as well as unsupervised learning of language models. These corpora have proved invaluable resources for research activities such as training part-of-speech taggers, disambiguating word-senses, detecting noun-phrases, inducing selectional restrictions, extracting argument structure and inducing probabilistic grammars.