AbstractCommunicating with multiple addressees poses a problem for speakers: Each addressee necessarily comes to the conversation with a different perspective—different knowledge, different beliefs, and a distinct physical context. Despite the ubiquity of multiparty conversation in everyday life, little is known about the processes by which speakers design language in multiparty conversation. While prior evidence demonstrates that speakers design utterances to accommodate addressee knowledge in multiparty conversation, it is unknown if and how speakers encode and combine different types of perspective information. Here we test whether speakers encode the perspective of multiple addressees, and then simultaneously consider their knowledge and physical context during referential design in a three‐party conversation. Analyses of referential form—expression length, disfluency, and elaboration rate—in an interactive multiparty conversation demonstrate that speakers do take into consideration both addressee knowledge and physical context when designing utterances, consistent with a knowledge‐scene integration view. These findings point to an audience design process that takes as input multiple types of representations about the perspectives of multiple addressees, and that bases the informational content of the to‐be‐designed utterance on a combination of the perspectives of the intended addressees.