This paper considers the potentially important role played by non-verbal communication in constraining pragmatic processing. Attention is paid to claims about the role of emotion in memory encoding and recall, its role in the formulation of plans and goals, and the creation of a shared emotional sense through various interpersonal processes. It is argued that ignoring these factors can lead to pragmatic theories which overestimate the processing demands facing the conversationalist, and that this overestimation will be problematic for any systems which seek to build on these theories in the future.