Computer analysis of human face images includes detection of faces, identification of people, and understanding of expression. Among these three tasks, facial expression has been the least studied, and most of the past work on facial expression tried to recognize a small set of emotions, such as joy, disgust, and surprise. This practice may follow from the work of Darwin, who proposed that emotions have corresponding prototypic facial expressions. In everyday life, however, such prototypic expressions occur relatively infrequently; instead, emotion is communicated more often by subtle changes in one or a few discrete features. FACS-code Action Units, defined by Ekman, are one such representation accepted in the psychology community. In collaboration with psychologists, we have been developing a system for automatically recognizing facial action units. This talk will present the current version of the system. The system uses a 3D Active Appearance Model to align a face image and transform it to a person-specific canonical coordinate frame. This transformation can remove appearance changes due to changes of head pose and relative illumination direction. In this transformed image frame, we perform detailed analysis of both facial motion and facial appearance changes, results of which are fed to an action-unit recogniser.