Previous studies assessing the involvement of the face-sensitive N170 component of the event-related potential (ERP) in the processing of face identity have shown controversial results when assessing N170 amplitude in repetition suppression (RS) designs. On the other hand, N170 adaptation is robustly associated with the inter-stimulus interval (ISI) between immediate face repetitions. Interestingly, interactions of face identity and ISI could provide valuable information on early encoding of face identity, but have not been investigated so far. We employed a repetition suppression paradigm using identical and non-identical repetitions as well as parametrically varied ISIs between 500 msec and 2,000 msec in 27 healthy subjects to investigate N170 adaptation effects. Both face identity and varying ISIs significantly influenced N170 adaptation effects, albeit with small effects sizes. Most importantly, however, face identity and ISIs strongly interacted with rapid N170 amplitude recovery in non-identical trials, but sustained N170 adaptation in identical trials. We excluded low-level sensory contributions to the N170 adaptation effect by analyzing the P1 component and by running an additional experiment employing different stimulus sizes. This specific result strongly argues in favor of neuronal sensitivity to face identity, which is primarily mirrored in the N170 temporal decay function that essentially differentiates identical and non-identical face trials. In general, taking advantage of the temporal dimension of adaptation processes, i.e., their decay over time, provides additional dissections of neuronal function into feature-specific selectivity versus non-selectivity.