Transformer with sequence relative position for continuous sign language translation

Jiwei Hu; Lian Ni

doi:10.1117/12.2637117

15 June 2022 Transformer with sequence relative position for continuous sign language translation

Jiwei Hu, Lian Ni

Proceedings Volume 12285, International Conference on Advanced Algorithms and Neural Networks (AANN 2022); 122850T (2022) https://doi.org/10.1117/12.2637117
Event: International Conference on Advanced Algorithms and Neural Networks (AANN 2022), 2022, Zhuhai, China

Abstract

Sign language is the main way for the hearing-impaired people, a huge special group, to communicate with others in society. The use of new information technology in sign language recognition and translation is helpful for smooth communication between hearing impaired and healthy people. With the development of the Transformer network and attention mechanism in machine translation, the study has entered a new process. Aiming at the phenomenon of longer-term dependency, based on Transformer, we propose a continuous sign language translation model that incorporates the sequence relative position into the attention mechanism, replacing the original absolute position encoding. Combining with movement characteristics, we use image difference technology to dynamically calculate difference threshold and use image blur detection to adaptively extract key frames. Experimental results on RWTH-PHOENIX-Weather 2014T Dataset verify the effectiveness of the proposed model.

Citation Download Citation

Jiwei Hu and Lian Ni "Transformer with sequence relative position for continuous sign language translation", Proc. SPIE 12285, International Conference on Advanced Algorithms and Neural Networks (AANN 2022), 122850T (15 June 2022); https://doi.org/10.1117/12.2637117

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Computer programming

Transformers

Video

Visual process modeling

RGB color model

Data modeling

Feature extraction

Show All Keywords

Keywords/Phrases

Search In:

Publication Years