Facial expression is usually synthesized or predicted on the basis of a given emotion. The prototypical expressions for basic emotions (happiness, sadness, surprise, disgust, anger, and fear) as postulated by discrete emotion psychologists are rather consistently produced and interpreted among different cultures, and can be used as icons to represent a basic emotion. However, these facial expressions are actually rarely observable in every day affective episodes, because of the complexity and multi-functional nature of facial expression and a high inter-individual variance. This article sketches the different functions of facial expression and presents and alternative way to predict, synthesize, and understand facial behavior. We present an empirical setting, which allows us to collect behavioral data in human-computer interactions, and show some examples from empirical studies as well as a computational model for facial expression on the basis of appraisal theory.