Abstract Language comprehension is often affected in individuals with post-stroke aphasia. However, deficits in auditory comprehension are not fully correlated with deficits in reading comprehension and the mechanisms underlying this dissociation remain unclear. This distinction is important for understanding language mechanisms, predicting long-term impairments and future development of treatment interventions. Using comprehensive auditory and reading measures from a large cohort of individuals with aphasia, we evaluated the relationship between aphasia type and reading comprehension impairments, the relationship between auditory versus reading comprehension deficits and the crucial neuroanatomy supporting the dissociation between post-stroke reading and auditory deficits. Scores from the Western Aphasia Battery—Revised from 70 participants with aphasia after a left-hemisphere stroke were utilized to evaluate both reading and auditory comprehension of linguistically equivalent stimuli. Repeated-measures and univariate ANOVA were used to assess the relationship between auditory comprehension and aphasia types and correlations were employed to test the relationship between reading and auditory comprehension deficits. Lesion-symptom mapping was used to determine the dissociation of crucial brain structures supporting reading comprehension deficits controlling for auditory deficits and vice versa. Participants with Broca’s or global aphasia had the worst performance on reading comprehension. Auditory comprehension explained 26% of the variance in reading comprehension for sentence completion and 44% for following sequential commands. Controlling for auditory comprehension, worse reading comprehension performance was independently associated with damage to the inferior temporal gyrus, fusiform gyrus, posterior inferior temporal gyrus, inferior occipital gyrus, lingual gyrus and posterior thalamic radiation. Auditory and reading comprehension are only partly correlated in aphasia. Reading is an integral part of daily life and directly associated with quality of life and functional outcomes. This study demonstrated that reading performance is directly related to lesioned areas in the boundaries between visual association regions and ventral stream language areas. This behavioural and neuroanatomical dissociation provides information about the neurobiology of language and mechanisms for potential future treatment interventions.