This paper is concerned with the role of phonatory quality in signalling affect. An overview of perception experiments is presented, which used synthetic stimuli with different phonatory qualities and f 0 contours in order to explore the mapping of voice quality to affect as well as the way in which voice quality combines with f 0. Results highlight the need for these phonetic parameters to be considered together. To identify the phonatory correlates of affect, we also need to understand the substrate of voice source variation, due to the linguistic content of utterances (prosodic and segmental) as well as to speaker specific characteristics. Illustrations of the former type of variation are presented, based on source parameterisation of inverse filtered data. A holistic analytic approach is advocated, which incorporates the main phonetic dimensions (voice quality, f 0 and temporal parameters) and which integrates the affective dimension with the more linguistic dimension of prosody.