publication . Preprint . 2018

Deep Predictive Models in Interactive Music

Martin, Charles P.; Ellefsen, Kai Olav; Torresen, Jim;
Open Access English
  • Published: 31 Jan 2018
Musical performance requires prediction to operate instruments, to perform in groups and to improvise. In this paper, we investigate how a number of digital musical instruments (DMIs), including two of our own, have applied predictive machine learning models that assist users by predicting unknown states of musical processes. We characterise these predictions as focussed within a musical instrument, at the level of individual performers, and between members of an ensemble. These models can connect to existing frameworks for DMI design and have parallels in the cognitive predictions of human musicians. We discuss how recent advances in deep learning highlight the...
free text keywords: Computer Science - Sound, Computer Science - Artificial Intelligence, Computer Science - Human-Computer Interaction, Computer Science - Neural and Evolutionary Computing, Electrical Engineering and Systems Science - Audio and Speech Processing
Download from
53 references, page 1 of 4

[1] N. S. Altman. An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician, 46(3):175-185, 1992. doi:10.1080/00031305.1992.10475879.

[2] C. Ames. Automated composition in retrospect: 1956-1986. Leonardo, 20(2):169-185, 1987. doi:10.2307/1578334.

[3] C. Ames. The Markov process as a compositional model: A survey and tutorial. Leonardo, 22(2):175-187, 1989. doi:10.2307/1575226.

[4] J. A. Biles. Improvizing with genetic algorithms: Genjam. In E. R. Miranda and J. A. Biles, editors, Evolutionary Computer Music, pages 137-169. Springer London, London, 2007. doi:10.1007/978-1-84628- 600-1_7. [OpenAIRE]

[5] S.-J. Blakemore, D. M. Wolpert, and C. D. Frith. Central cancellation of self-produced tickle sensation. Nature Neuroscience, 1(7):635-640, 1998. doi:10.1038/2870. [OpenAIRE]

[6] A. R. Brown and T. Gifford. Prediction and proactivity in real-time interactive music systems. Int. Workshop on Musical Metacreation, pages 35-39, 2013. URL:

[7] J.-P. Cáceres, R. Hamilton, D. Iyer, C. Chafe, and G. Wang. To the edge with China: Explorations in network performance. In ARTECH 2008: Proc. 4th Int. Conf. Digital Arts, pages 61-66, 2008.

[8] B. Caramiaux, N. Montecchio, A. Tanaka, and F. Bevilacqua. Adaptive gesture recognition with variation estimation for interactive systems. ACM Transactions on Interactive Intelligent Systems, 4(4):18:1-18:34, 2014. doi:10.1145/2643204.

[9] B. Caramiaux and A. Tanaka. Machine learning of musical gestures. In Proceedings of the International Conference on New Interfaces for Musical Expression, NIME '13, pages 513-518, 2013. URL: http://nime. org/proceedings/2013/nime2013_84.pdf.

[10] K. Chakraborty, K. Mehrotra, C. K. Mohan, and S. Ranka. Forecasting the behavior of multivariate time series using neural networks. Neural networks, 5(6):961-970, 1992. [OpenAIRE]

[11] B. A. Clegg, G. J. DiGirolamo, and S. W. Keele. Sequence learning. Trends in Cognitive Sciences, 2(8):275-281, 2017/11/28 1998. URL:, doi:10.1016/ S1364-6613(98)01202-9. [OpenAIRE]

[12] S. Davies. Themes in the Philosophy of Music. Oxford University Press, Oxford, UK, 2005.

[13] D. Eck and J. Schmidhuber. Finding temporal structure in music: Blues improvisation with LSTM recurrent networks. In Proc. 12th IEEE Workshop on Neural Networks for Signal Processing, pages 747-756, 2002. doi:10.1109/NNSP.2002.1030094.

[14] J. Engel, C. Resnick, A. Roberts, S. Dieleman, D. Eck, K. Simonyan, and M. Norouzi. Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders. ArXiv e-prints, Apr. 2017. URL: 1704.01279.

[15] R. Fiebrink, P. R. Cook, and D. Trueman. Human model evaluation in interactive supervised learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '11, pages 147-156, New York, NY, USA, 2011. ACM. doi:10.1145/1978942.1978965.

53 references, page 1 of 4
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue