SARS-CoV-2 infection results in varying disease severity, ranging from asymptomatic infection to severe illness. A detailed understanding of the immune response to SARS-CoV-2 is critical to unravel the causative factors underlying differences in disease severity and to develop optimal vaccines against new SARS-CoV-2 variants. We combined single-cell RNA and T cell receptor sequencing with CITE-seq antibodies to characterize the CD8+ T cell response to SARS-CoV-2 infection at high resolution and compared responses between mild and severe COVID-19. We observed a population of exhausted CD8+ T cells in severe SARS-CoV-2 infection and identified a population of NK-like, terminally differentiated CD8+ effector T cells characterized by expression of FCGR3A (encoding CD16). Further characterization of NK-like CD8+ T cells revealed heterogeneity among CD16+ NK-like CD8+ T cells and profound differences in cytotoxicity, exhaustion, and NK-like differentiation between mild and severe disease conditions. We propose a model in which differences in the surrounding inflammatory milieu lead to crucial differences in NK-like differentiation of CD8+ effector T cells, ultimately resulting in the appearance of NK-like CD8+ T cell populations of different functionality and pathogenicity. Our in-depth characterization of the CD8+ T cell-mediated response to SARS-CoV-2 infection provides a basis for further investigation of the importance of NK-like CD8+ T cells in COVID-19 severity.