Cognitive deficits in old-age depression vary as a function of multiple factors; one rarely examined factor is long-term psychiatric history. We investigated effects of psychiatric history on cognitive performance in old-age depression and in remitted persons. In the population-based Swedish National Study on Aging and Care in Kungsholmen study, older persons (≥60 years) without dementia were tested with a cognitive battery and matched to the Swedish National Inpatient Register (starting 1969). Participants were grouped according to current depression status and psychiatric history and compared to healthy controls (n = 96). Group differences were observed for processing speed, attention, executive functions, and verbal fluency. Persons with depression and psychiatric inpatient history (n = 20) and late-onset depression (n = 49) performed at the lowest levels, whereas cognitive performance in persons with self-reported recurrent unipolar depression (n = 52) was intermediate. Remitted persons with inpatient history of unipolar depression (n = 38) exhibited no cognitive deficits. Heart disease burden, physical inactivity, and cumulative inpatient days modulated the observed group differences in cognitive performance. Among currently depressed persons, those with inpatient history, and late onset performed at the lowest levels. Importantly, remitted persons showed no cognitive deficits, possibly reflecting the extended time since the last admission (m = 15.6 years). Thus, the present data suggest that cognitive deficits in unipolar depression may be more state- than trait-related. Information on profiles of cognitive performance, psychiatric history, and health behaviors may be useful in tailoring individualized treatment.