Spatiotemporal interest point detector exploiting appearance and motion-variation information

Yanshan Li; Qingteng Li; Qinghua Huang; Rongjie Xia; Xuelong Li

doi:10.1117/1.JEI.28.3.033002

13 May 2019 Spatiotemporal interest point detector exploiting appearance and motion-variation information

Yanshan Li, Qingteng Li, Qinghua Huang, Rongjie Xia, Xuelong Li

Author Affiliations +

Funded by: National Natural Science Foundation of China (NSFC), Natural Science Foundation of Guangdong Province, Guangdong Provincial Science and Technology Department, Science and Technology Program of Guangzhou, Shenzhen Science and Technology Project

Journal of Electronic Imaging, Vol. 28, Issue 3, 033002 (May 2019). https://doi.org/10.1117/1.JEI.28.3.033002

Abstract

As a local invariant feature of videos, the spatiotemporal interest point (STIP) has been widely used in computer vision and pattern recognition. However, existing STIP detectors are generally extended from detection algorithms constructed for local invariant features of two-dimensional images, which does not explicitly exploit the motion information inherent in the temporal domain of videos, thus weakening the performance of existing STIP detectors in a video context. To remedy this, we aim to develop an STIP detector that uniformly captures appearance and motion information for video, thus yielding substantial performance improvement. Specifically, under the framework of geometric algebra, we first develop a spatiotemporal unified model of appearance and motion-variation information (UMAMV), and then a UMAMV-based scale space of the spatiotemporal domain is proposed to synthetically analyze appearance information and motion information in a video. Based on this model, we propose an STIP feature of UMAMV-SIFT that embraces both appearance and motion variation information of the videos. Three datasets with different sizes are utilized to evaluate the proposed model and the STIP detector. We present experimental results to show that the UMAMV-SIFT achieves state-of-the-art performance and is particularly effective when dataset is small.

Citation Download Citation

Yanshan Li, Qingteng Li, Qinghua Huang, Rongjie Xia, and Xuelong Li "Spatiotemporal interest point detector exploiting appearance and motion-variation information," Journal of Electronic Imaging 28(3), 033002 (13 May 2019). https://doi.org/10.1117/1.JEI.28.3.033002

Received: 4 January 2019; Accepted: 8 April 2019; Published: 13 May 2019

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE