Chimpanzee and gorilla chromosomes differ from human chromosomes by the presence of large blocks of subterminal heterochromatin thought to be composed primarily of arrays of tandem satellite sequence. We explore their sequence composition and organization and show a complex organization composed of specific sets of segmental duplications that have hyperexpanded in concert with the formation of subterminal satellites. These regions are highly copy number polymorphic between and within species, and copy number differences involving hundreds of copies can be accurately estimated by assaying read-depth of next-generation sequencing data sets. Phylogenetic and comparative genomic analyses suggest that the structures have arisen largely independently in the two lineages with the exception of a few seed sequences present in the common ancestor of humans and African apes. We propose a model where an ancestral human-chimpanzee pericentric inversion and the ancestral chromosome 2 fusion both predisposed and protected the chimpanzee and human genomes, respectively, to the formation of subtelomeric heterochromatin. Our findings highlight the complex interplay between duplicated sequences and chromosomal rearrangements that rapidly alter the cytogenetic landscape in a short period of evolutionary time.