
This paper presents two key contributions to the real-time classification of Instrumental Playing Techniques (IPTs) in the context of NIME and human-machine interactive systems: the EG-IPT dataset and the ipt~ Max/MSP object. The EG-IPT dataset, specifically designed for electric guitar, encompasses a broad range of IPTs captured across six distinct audio sources (five microphones and one direct input) and three pickup configurations. This diversity in recording conditions provides a robust foundation for training accurate models. We evaluate the dataset by employing a Convolutional Neural Network-based classifier (CNN), achieving state-of-the-art performance across a wide array of IPT classes, thereby validating the dataset's efficacy. The ipt~ object is a new Max/MSP external enabling real-time classification of IPTs via pre-trained CNN models. While in this paper it's demonstrated with the EG-IPT dataset, the ipt~ object is adaptable to models trained on various instruments. By integrating EG-IPT and ipt~, we introduce a novel, end-to-end workflow that spans from data collection, model training to real-time classification and human-computer interaction. This workflow exemplifies the entanglement of diverse components (data acquisition, machine learning, real-time processing, and interactive control) within a unified system, advancing the potential for dynamic, real-time music performance and human-computer interaction in the context of NIME.
[INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI], [INFO.INFO-SY] Computer Science [cs]/Systems and Control [cs.SY], Music Classification, NIME, [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], [INFO] Computer Science [cs], Real-Time, [INFO.INFO-SD] Computer Science [cs]/Sound [cs.SD], Electric Guitar, [INFO.INFO-ET] Computer Science [cs]/Emerging Technologies [cs.ET], Instrumental Playing Techniques, [INFO.INFO-IR] Computer Science [cs]/Information Retrieval [cs.IR], Music AI, [INFO.INFO-HC] Computer Science [cs]/Human-Computer Interaction [cs.HC], Python, Max/MSP
[INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI], [INFO.INFO-SY] Computer Science [cs]/Systems and Control [cs.SY], Music Classification, NIME, [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], [INFO] Computer Science [cs], Real-Time, [INFO.INFO-SD] Computer Science [cs]/Sound [cs.SD], Electric Guitar, [INFO.INFO-ET] Computer Science [cs]/Emerging Technologies [cs.ET], Instrumental Playing Techniques, [INFO.INFO-IR] Computer Science [cs]/Information Retrieval [cs.IR], Music AI, [INFO.INFO-HC] Computer Science [cs]/Human-Computer Interaction [cs.HC], Python, Max/MSP
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
