Evaluation schemes for video and image anomaly detection algorithms

Shibin Parameswaran; Josh Harguess; Christopher Barngrover; Scott Shafer; Michael Reese

doi:10.1117/12.2224667

12 May 2016 Evaluation schemes for video and image anomaly detection algorithms

Shibin Parameswaran, Josh Harguess, Christopher Barngrover, Scott Shafer, Michael Reese

Proceedings Volume 9844, Automatic Target Recognition XXVI; 98440D (2016) https://doi.org/10.1117/12.2224667
Event: SPIE Defense + Security, 2016, Baltimore, MD, United States

Abstract

Video anomaly detection is a critical research area in computer vision. It is a natural first step before applying object recognition algorithms. There are many algorithms that detect anomalies (outliers) in videos and images that have been introduced in recent years. However, these algorithms behave and perform differently based on differences in domains and tasks to which they are subjected. In order to better understand the strengths and weaknesses of outlier algorithms and their applicability in a particular domain/task of interest, it is important to measure and quantify their performance using appropriate evaluation metrics. There are many evaluation metrics that have been used in the literature such as precision curves, precision-recall curves, and receiver operating characteristic (ROC) curves. In order to construct these different metrics, it is also important to choose an appropriate evaluation scheme that decides when a proposed detection is considered a true or a false detection. Choosing the right evaluation metric and the right scheme is very critical since the choice can introduce positive or negative bias in the measuring criterion and may favor (or work against) a particular algorithm or task. In this paper, we review evaluation metrics and popular evaluation schemes that are used to measure the performance of anomaly detection algorithms on videos and imagery with one or more anomalies. We analyze the biases introduced by these by measuring the performance of an existing anomaly detection algorithm.

Citation Download Citation

Shibin Parameswaran, Josh Harguess, Christopher Barngrover, Scott Shafer, and Michael Reese "Evaluation schemes for video and image anomaly detection algorithms", Proc. SPIE 9844, Automatic Target Recognition XXVI, 98440D (12 May 2016); https://doi.org/10.1117/12.2224667

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available