Abstract Though phonemic fluency tasks are traditionally indexed by the number of correct responses, the underlying disorder may shape the specific choice of words—both correct and erroneous. We report the first comprehensive qualitative analysis of incorrect and correct words generated on the phonemic (‘S’) fluency test, in a large sample of patients (n = 239) with focal, unilateral frontal or posterior lesions and healthy controls (n = 136). We conducted detailed qualitative analyses of the single words generated in the phonemic fluency task using categorical descriptions for different types of errors, low-frequency words and clustering/switching. We further analysed patients’ and healthy controls’ entire sequences of words by employing stochastic block modelling of Generative Pretrained Transformer 3–based deep language representations. We conducted predictive modelling to investigate whether deep language representations of word sequences improved the accuracy of detecting the presence of frontal lesions using the phonemic fluency test. Our qualitative analyses of the single words generated revealed several novel findings. For the different types of errors analysed, we found a non-lateralized frontal effect for profanities, left frontal effects for proper nouns and permutations and a left posterior effect for perseverations. For correct words, we found a left frontal effect for low-frequency words. Our novel large language model–based approach found five distinct communities whose varied word selection patterns reflected characteristic demographic and clinical features. Predictive modelling showed that a model based on Generative Pretrained Transformer 3–derived word sequence representations predicted the presence of frontal lesions with greater fidelity than models of native features. Our study reveals a characteristic pattern of phonemic fluency responses produced by patients with frontal lesions. These findings demonstrate the significant inferential and diagnostic value of characterizing qualitative features of phonemic fluency performance with large language models and stochastic block modelling.