Background: Pathogens are often assumed to evolve towards reduced virulence, but counterexamples abound. Faced with a new pathogen, such as SARS-CoV-2, it is crucial to be able to forecast the case fatality rate (CFR) and the overall disease burden. Considerable effort has been invested towards developing a mathematical framework for predicting virulence evolution. Although many approaches accurately recapitulate complex outcomes, most rely on an assumed trade-off between CFR and infection rate. It is often impractical to empirically validate this constraint for human pathogens. Methods: A compartment model with parameters tuning the degree to which symptomatic individuals are isolated and the duration of immunity is constructed and evaluated at both short timescales and at equilibrium. Results: We reveal kinetic constraints whereby variation of multiple parameters in concert leads to decreased CFR and increased pathogen fitness, whereas independent variation of the parameters decreases pathogen fitness. Smallpox, SARS-CoV-2, and influenza are analyzed as diverse representatives of human respiratory viruses. We show that highly virulent viruses, such as smallpox, are often constrained by the host behavior, whereas moderately virulent viruses, such as SARS-CoV-2, appear to be typically constrained by the relationship between the duration of immunity and CFR. Conclusions: Evolution of human respiratory epidemics appears to be often kinetically constrained and a reduction in CFR should not be assumed. These results agree with previous work demonstrating an increase in virulence for smallpox and further predict that SARS-CoV-2 is likely to continue presenting a substantial disease burden. Herd immunity against SARS-CoV-2 and viruses with similar life history traits might be unachievable without vaccination. However, partial isolation of symptomatic individuals can have a major effect on the epidemic dynamics not only by reducing the number of fatalities in the short term but also by changing the evolutionary trajectory of moderate CFR viruses towards reduced CFR.