The process of second harmonic generation (SHG) has a unique property of forming a sharp optical contrast between noncentrosymmetric crystalline materials and other types of material, which is a highly valuable asset for contrast microscopy. The coherent signal obtained through SHG also allows for the recording of holograms at high spatial and temporal resolution, enabling whole-field four-dimensional microscopy for highly dynamic microsystems and nanosystems. Here we describe a new holographic principle, harmonic holography (H(2)), which records holograms between independently generated second harmonic signals and reference. We experimentally demonstrate this technique with digital holographic recording of second harmonic signals upconverted from an ensemble of second harmonic generating nanocrystal clusters under femtosecond laser excitation. Our results show that harmonic holography is uniquely suited for ultrafast four-dimensional contrast microscopy.