Pengi: An Audio Language Model for Audio Tasks

descriptionPublicationkeyboard_double_arrow_right Article , Conference object , Preprint 01 Jan 2023Embargo end date: 01 Jan 2023Publisher:Neural Information Processing Systems Foundation, Inc. (NeurIPS)Journal:Advances in Neural Information Processing Systems 36

Authors: Soham Deshmukh; Benjamin Elizalde; Rita Singh; Huaming Wang;

doi: 10.52202/075280-0795 , 10.5281/zenodo.8385050 , 10.5281/zenodo.8385049 , 10.48550/arxiv.2305.11834 , 10.5281/zenodo.8387083

arXiv: 2305.11834

Pengi: An Audio Language Model for Audio Tasks

- Summary
- Subjects
- Metrics

Abstract

Pengi is an Audio Language Model that leverages Transfer Learning by framing all audio tasks as text-generation tasks. It takes as input, an audio recording, and text, and generates free-form text as output. The unified architecture of Pengi enables open-ended tasks and close-ended tasks without any additional fine-tuning or task-specific extensions. The code repository is: microsoft/Pengi: An Audio Language model for Audio Tasks (github.com)

Related Organizations

View all View all

Keywords

FOS: Computer and information sciences, Sound (cs.SD), zero-shot, acoustic scenes, Audio and Speech Processing (eess.AS), sound events, FOS: Electrical engineering, electronic engineering, information engineering, Pengi, audio captioning, audio question answering, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Impact byBIP!

	selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).	4
	popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.	Top 10%
	influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).	Average
	impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.	Top 10%