(1) Background: Aspergillus produces high morbidity and mortality, especially in at-risk populations. In Spain, the evolution of mortality in recent years due to this fungus is not well established. The aim of this study was to estimate the case fatality rate of aspergillosis in inpatients from 1997 to 2017 in Spain. (2) Methodology: A retrospective descriptive study was conducted with records of inpatients admitted to the National Health System with a diagnosis of aspergillosis. (3) Principal findings: Of 32,960 aspergillosis inpatients, 24.5% of deaths were registered, and 71% of the patients who died were men. The percentage of deaths increased progressively with age. The case fatality rate progressively decreased over the period, from 25.4 and 27.8% in 1997–1998 to values of 20.6 and 20.8% in 2016 and 2017. Influenza and pneumonia occurrence/association significantly increased case fatality rates in all cases. (4) Conclusions: Our study shows that lethality significantly decreased in the last two decades despite the increase in cases. This highlights the fact that patients with solid and/or hematological cancer do not have a much higher mortality rate than the group of patients with pneumonia or influenza alone, these two factors being the ones that cause the highest CFRs. We also need studies that analyze the causes of mortality to decrease it and studies that evaluate the impact of COVID-19.