Many studies emphasize the importance of genetic diversity and the need for an appropriate tuning of selection pressure in genetic programming. Additional important aspects are the performance and effects of the genetic operators (crossover and mutation) on the transfer and stabilization of inherited information blocks during the run of the algorithm. In this context, different ideas about the usage of lineage and genealogical information for improving genetic programming have taken shape in the last decade. Our work builds on those ideas by introducing an evolution tracking framework for assembling genealogical and inheritance graphs of populations. The proposed approach allows detailed investigation of phenomena related to building blocks, size evolution, ancestry and diversity. We introduce the notion of genetic fragments to represent subtrees that are affected by reproductive operators (mutation and crossover) and present a methodology for tracking such fragments using flexible similarity measures. A fragment matching algorithm was designed to work on both structural and semantic levels, allowing us to gain insight into the exploratory and exploitative behavior of the evolutionary process. The visualization part which is the subject of this paper integrates with the framework and provides an easy way of exploring the population history. The paper focuses on a case study in which we investigate the evolution of a solution to a symbolic regression benchmark problem.