Genetic diversity is declining globally, a trend that may particularly affect exploited populations that must adapt to rapid environmental change and other threats. Estimated genomic changes in effective population size mirrored known whaling history and shifts in technology. In the Southern Ocean, a comparison of genomes from historical and contemporary populations indicated that the contemporary genomes have less diversity and an elevated realized mutation load for moderately deleterious mutations, likely due to the effects of whaling. Our results demonstrate that the relatively recent, brief, and marked depletion of humpback whale populations by whaling likely had subtle but discernible, negative, and lasting effects on the whales’ genomes. Thus, even as some humpback whale populations are now recovering to pre-exploitation numbers, they likely do so with a diminished adaptive capacity in the face of future conditions and threats.