This study seeks to understand the effect of the COVID-19 pandemic on a group of university ESL learners and their teachers in the U.S. While some schools quickly closed to protect students and teachers, others transitioned to remote teaching and learning. In addition to challenges associated with moving instruction to an online format, many teachers and students have had to grapple with a variety of additional issues regarding health, employment, finances, and other concerns. With the aim of better understanding the effects of the pandemic on students and teachers, researchers prepared a survey that addressed stress levels, learning and teaching English, and remote instruction. Participants indicated how important learning English was before and after the pandemic and responded to an open-ended question soliciting their perspectives on what was most difficult during the pandemic. The researchers looked at their responses, demographic information, and proficiency gains to learn more about the impact of the COVID-19 pandemic on students and teachers. The results of this study show that the effects of the pandemic increased stress for both students and teachers across an array of contexts. Although still relatively important, the priority for teaching and learning decreased for both the practitioners and their students during the pandemic due to a variety of new stressors in their lives. The transition to online instruction was more challenging for the students than the teachers. Results also show that during the pandemic students experienced less language development for speaking than for writing.