Immunomodulatory therapy has been extensively studied in randomized clinical trials for the treatment of patients hospitalized for COVID-19 with inconsistent findings. Guideline committees, reviewing the same clinical trial data, have generated different recommendations for immunomodulatory therapy.We hypothesize that trial design differences, specifically whether the study utilized an open-label or placebo-controlled design, accounted for the inconsistent mortality effects reported in clinical trials of immunomodulator therapies for COVID-19.We reviewed COVID-19 treatment guidelines (World Health Organization [WHO], Infectious Diseases Society of America [IDSA] and The National Institutes of Health [NIH]) and identified the meta-analyses associated with glucocorticoids, IL-6 inhibitors, JAK kinase inhibitors, and complement C5a inhibitors that were available to the guideline authors at the time recommendations were either made or updated.We identified a meta-analysis for each of the immunomodulator classes that are included in current COVID-19 treatment guidelines: glucocorticoids [WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group; Shankar-Hari M, Vale CL, Godolphin PJ, Fisher D, Higgins JPT, et al. Association between administration of IL-6 antagonists and mortality among patients hospitalized for COVID-19: A meta-analysis. JAMA. 2021;326:499-518] (cited 419), IL-6 antagonists [WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group; Shankar-Hari M, Vale CL, Godolphin PJ, Fisher D, Higgins JPT, et al. Association between administration of IL-6 antagonists and mortality among patients hospitalized for COVID-19: A meta-analysis. JAMA. 2021;326:499-518] (cited 419), JAK inhibitors [Kramer A, Prinz C, Fichtner F, Fischer AL, Thieme V, Grundeis F, et al. Janus kinase inhibitors for the treatment of COVID-19. Cochrane Database Syst Rev. 2022;6:CD015209] (cited 34), and complement C5a inhibitors [Tsai CL, Lai CC, Chen CY, Lee HS. The efficacy and safety of complement C5a inhibitors for patients with severe COVID-19: A systematic review and meta-analysis. Expert Rev Anti Infect Ther. 2023;21:77-86] (cited 1). Using the same randomized clinical trials, we evaluated the four meta-analyses accounting for trial design: placebo-controlled or open-label. Glucocorticoids (Risk Ratio [RR] 0.91 [95% CI, 0.49-1.69]), IL-6 inhibitors sarilumab (RR 1.17 [95% CI, 0.96-01.43]), and tocilizumab (RR 0.95 [95% CI, 0.76-1.19]) did not reduce mortality in placebo-controlled trials, whereas baricitinib did confer a large survival benefit (RR 0.65 [95% CI, 0.52-0.81]). The complement C5a inhibitor, vilobelimab, also reduced mortality in a single placebo-controlled trial (RR 0.76 [95% CI, 0.57-1.0]).Placebo-controlled trial evidence indicates that baricitinib should be the first choice immunomodulator for patients hospitalized for COVID-19 who require any form of oxygen support-low- or high-flow oxygen, non-invasive or invasive ventilation. Vilobelimab warrants study in a large placebo-controlled trial. Treatment guidelines for future pandemics should prioritize the results of placebo-controlled trials.