Powered by OpenAIRE graph
Found an issue? Give us feedback
addClaim

Evaluating OpenAI Whisper and its variants for increasing the accessibility of audio collections

Authors: Trizna, Michael; Sanchez, Crystal; Cain, Emily; Custer, Mark;

Evaluating OpenAI Whisper and its variants for increasing the accessibility of audio collections

Abstract

Original abstract: The development at the Smithsonian Institution of a central Media Asset Delivery Service (MADS) aims to address accessibility compliance by remediating digital content. MADS supports audio and video delivery with captions through Smithsonian's Digital Asset Management System (DAMS). Machine generated captioning has previously not been accurate enough to justify use of staff time to fix the results. However, with the release of OpenAI's Whisper model, the Smithsonian launched a pilot project that aimed to test the feasibility of tools to help remediate centrally delivered audio files and make them available through the new MADS player. The Smithsonian Transcription Center is a digital volunteer program that allows the public to help transcribe the Smithsonian’s historical documents and collections records to improve accessibility and discoverability. Volunteers have transcribed over 200 hours of audio on Transcription Center, which we used alongside audio transcriptions generated by professional vendor support, as a baseline for comparing and evaluating the output of different versions of the Whisper model. We conducted a meticulous evaluation, comparing Whisper-generated captions against human-transcribed content from varied contexts, shedding light on their relative strengths and limitations.

Related Organizations
Keywords

Audio processing, Artificial intelligence not elsewhere classified, Critical heritage, museum and archive studies, Speech recognition, Machine learning not elsewhere classified

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average
Upload OA version
Are you the author of this publication? Upload your Open Access version to Zenodo!
It’s fast and easy, just two clicks!