Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ACM Transactions on ...arrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ACM Transactions on Asian and Low-Resource Language Information Processing
Article . 2026 . Peer-reviewed
License: CC BY NC ND
Data sources: Crossref
addClaim

Transformer Model for Direct Translation of Spoken Text to Arabic Sign Language (ArSL)

Authors: Samah Abbas; Dimah Alahmadi; Hassanin Al-Barhamtoshy;

Transformer Model for Direct Translation of Spoken Text to Arabic Sign Language (ArSL)

Abstract

Arabic Sign Language (ArSL) is a formal language used by Deaf and Hard of Hearing communities. It helps them understand Arabic spoken by hearing people, particularly in formal speech, such as news or sermons. The production of sign language systems can support communication by translating spoken languages into ArSL videos. However, current systems often fail to meet user needs due to poor translation of grammatical structures and limited ability to capture the textual meaning of speech. This research focused on a neural machine translation (NMT) approach to convert spoken Arabic texts into Arabic Sign Language (ArSL) using a transformer-based model to overcome these difficulties. The ArSL video of the sign is produced as a series of 2D skeletal poses. The proposed methodology is based on an encoder-decoder architecture: texts are represented by a transformer model, and a posture decoder generates gestures. The model is evaluated using quantitative criteria, such as mean square error (MSE) and the dynamic time warping (DTW) measure, as well as qualitative evaluations by sign language experts and users from the Deaf and Hard of Hearing community. The study uses Friday sermon data as a reference to build the translation model and applies optimization techniques such as temporal modulation, noise addition, and resizing to enhance performance. The results show that the enhanced model improves the quality of gesture representation, reducing DTW by 43.4% relative to the baseline in the non-continuous ArSL test. On the other hand, in the Continuous ArSL test using Short Sermon, the DTW is reduced by around 2.25%. In addition, the enhanced model received high user acceptance, with 75.2% satisfaction. Moreover, unified qualitative evaluation criteria for sign language production are proposed, including clarity, accuracy, consistency, and naturalness, which could serve as a basis for future evaluations in this field. These criteria are used to evaluate the generated ArSL and measure user satisfaction.

Related Organizations
  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average