Empathy is a key component of interpersonal interactions that is often neglected by modern communication technologies. This paper presents the theoretical basis, prototype, and preliminary user testing for an application of affective computing in mediating live human-to-human interactions. The proposed system uses facial expression recognition to identify the emotional state of a user's conversation partner. It algorithmically generates emotional music to match the expressive state of the partner and plays the music to the user in a non-disruptive manner. Preliminary user studies indicate that the prototype system can generate music that users reliably associate with the emotions of anger, happiness, fear, and sadness and that the presence of emotional music augments the emotional response generated by visual cues.