A multi-attention based fMRI feature extraction method for brain states recognition

Chong Wang; Hongmei Yan; Tao Liu; Wei Sheng; Yun-Shuang Fan; Rong Li; Huafu Chen

doi:10.1117/12.2680732

27 June 2023 A multi-attention based fMRI feature extraction method for brain states recognition

Chong Wang, Hongmei Yan, Tao Liu, Wei Sheng, Yun-Shuang Fan, Rong Li, Huafu Chen

Proceedings Volume 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022); 127050A (2023) https://doi.org/10.1117/12.2680732
Event: Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 2022, Nanjing, China

Abstract

Identifying the experimental conditions from brain activity has been a focus direction in recent functional magnetic resonance imaging (fMRI) studies, which advances our understanding of the brain mechanism and can be applied in brain computer interface systems. However, task fMRI signals contain a lot of noise that is irrelevant to the task (e.g., spontaneous brain activity and physiological noise), and these task-independent components limit the performance of brain states recognition. In this work, we proposed a multi-attention neural network (MANN) to extract task-dependent components of the fMRI data and recognize the task conditions. We employ three attention modules in MANN (temporal, spatial and relational attention modules) to describe the brain activation from multiple dimensions and extract taskrelated fMRI features. We evaluate the proposed model using the emotion task fMRI data from the Human Connectome Project dataset, in which more than 1000 participates are adopted. The MANN achieves a classification accuracy of 99.51% between different task conditions (shape and face). To further investigate how the attention modules work, we visualize the attention weights and perform ablation studies. Our result indicate that the attention modules can learn biologically meaningful brain representations and contribute to the improvement of the classification accuracy. Our model offers a powerful tool for brain states recognition, and has the potential application in clinical diagnosis.

Citation Download Citation

Chong Wang, Hongmei Yan, Tao Liu, Wei Sheng, Yun-Shuang Fan, Rong Li, and Huafu Chen "A multi-attention based fMRI feature extraction method for brain states recognition", Proc. SPIE 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 127050A (27 June 2023); https://doi.org/10.1117/12.2680732

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Functional magnetic resonance imaging

Brain

Data modeling

Feature extraction

Emotion

Fourier transforms

Ablation

Show All Keywords

Keywords/Phrases

Search In:

Publication Years