Sonification Of Facial Actions For Musical Expression

descriptionPublicationkeyboard_double_arrow_right Conference object , Article , Preprint , Other literature type 01 Jun 2005Embargo end date: 01 Jan 2020Publisher:ZenodoJournal:CoRR, volume abs/2010.03223

Authors: Mathias Funk; Kazuhiro Kuwabara; Michael J. Lyons 0001;

doi: 10.5281/zenodo.1176750 , 10.5281/zenodo.1176749 , 10.48550/arxiv.2010.03223

arXiv: 2010.03223

Sonification Of Facial Actions For Musical Expression

- Summary
- Subjects
- Metrics

Abstract

The central role of the face in social interaction and non-verbal communication suggests we explore facial action as a means of musical expression. This paper presents the design, implementation, and preliminary studies of a novel system utilizing face detection and optic flow algorithms to associate facial movements with sound synthesis in a topographically specific fashion. We report on our experience with various gesture-to-sound mappings and applications, and describe our preliminary experiments at musical performance using the system.

Proceedings of the International Conference on New Interfaces for Musical Expression, 2005 (NIME-05)

Keywords

FOS: Computer and information sciences, Sound (cs.SD), Audio and Speech Processing (eess.AS), Computer Vision and Pattern Recognition (cs.CV), Computer Science - Human-Computer Interaction, Computer Science - Computer Vision and Pattern Recognition, FOS: Electrical engineering, electronic engineering, information engineering, H.5.5, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing, Human-Computer Interaction (cs.HC)

Impact byBIP!

	selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).	1
	popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.	Average
	influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).	Average
	impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.	Average