CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 12177 > Article
Paper
30 April 2022 A study of multimodal head/eye orientation prediction techniques in virtual space
Daiki Higuchi, Yoshitsugu Manabe, Noriko Yata
Author Affiliations +
Proceedings Volume 12177, International Workshop on Advanced Imaging Technology (IWAIT) 2022; 121771I (2022) https://doi.org/10.1117/12.2626118
Event: International Workshop on Advanced Imaging Technology 2022 (IWAIT 2022), 2022, Hong Kong, China
Abstract
Various models have been proposed to predict the future head/gaze orientation of a user watching a 360-degree video. However, most of these models do not take sound information into account, and there are few studies on the influence of sound on users in VR space. This study proposes a multimodal model for predicting head/gaze orientation for 360-degree videos based on a new analysis of users' head/gaze behavior in VR space. First, we focus on whether people are attracted to the sound source of the 360-degree video or not. We conducted a head/gaze tracking experiment with 22 subjects in AV (Audio-Visual) and V (Visual) conditions using 32 videos. As a result, it was confirmed that whether they were attracted to the sound source differed depending on the video. Next, we trained a deep learning model based on the results and constructed and evaluated a multimodal model that combined visual and auditory information. As a result, we were able to construct a multimodal head/gaze prediction model that used the sound source explicitly. However, from the viewpoint of accuracy improvement, we could not confirm any advantage of multimodalization. Finally, a discussion of this problem and prospects is given.
© (2022) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Daiki Higuchi, Yoshitsugu Manabe, and Noriko Yata "A study of multimodal head/eye orientation prediction techniques in virtual space", Proc. SPIE 12177, International Workshop on Advanced Imaging Technology (IWAIT) 2022, 121771I (30 April 2022); https://doi.org/10.1117/12.2626118
ACCESS THE FULL ARTICLE
ORGANIZATIONAL
Sign in with credentials provided by your organization.
INSTITUTIONAL
Select your institution to access the SPIE Digital Library.
SELECT YOUR INSTITUTION
PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
PERSONAL SIGN IN
No SPIE Account? Create one
PURCHASE THIS CONTENT
SUBSCRIBE TO DIGITAL LIBRARY
50 downloads per 1-year subscription
Members: $195
Non-members: $335 ADD TO CART
25 downloads per 1 - year subscription
Members: $145
Non-members: $250 ADD TO CART
PURCHASE SINGLE ARTICLE
Includes PDF, HTML & Video, when available
Members: $17.00
Non-members: $21.00 ADD TO CART
PROCEEDINGS
 6 PAGES
DOWNLOAD PAPER SAVE TO MY LIBRARY
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Video
Eye
Head
Virtual reality
Head-mounted displays
Visual process modeling
Machine learning
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top