Brown adipose tissue aging and the concomitant loss of thermogenic capacity have been linked to an inability to maintain normal energy homeostasis in late life. Similarly, the ability of white fat to convert into brite/beige adipose tissue declines. This may ultimately exacerbate the progression of age-related metabolic pathologies, such as insulin resistance and obesity. The depletion of all types of brown adipocytes during aging is well-established and has been described in rodent models as well as humans. We here review the available literature on the potential mechanisms leading to cell-autonomous and microenvironment-related aspects of brown adipocyte dysfunction. Among these, cellular senescence, mitochondrial impairment, and deteriorating changes to the local and endocrine microenvironments have been proposed. An important goal of aging research is to develop approaches that may not only extend life expectancy but also prolong health-span. These efforts may also be aimed at maintaining metabolic health throughout life by targeting brown adipocyte function.