In human computer interaction, the ultimate goal is to have effortless and natural communication. In the research literature significant effort has been directed toward understanding the functional aspects of the communication. However, it is well known that the functional aspect is insufficient for natural interactions. Indeed, the emotional or affective aspect has been shown in the psychology literature to be at least if not more important. As emotional beings, we interact most comfortably with other emotional beings. We give an overview of our current research toward automatic recognition of human emotions.