Countermeasure against backdoor attacks using epistemic classifiers

Zhaoyuan Yang; Nurali Virani; Naresh S. Iyer

doi:10.1117/12.2558255

22 April 2020 Countermeasure against backdoor attacks using epistemic classifiers

Zhaoyuan Yang, Nurali Virani, Naresh S. Iyer

Proceedings Volume 11413, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications II; 114130P (2020) https://doi.org/10.1117/12.2558255
Event: SPIE Defense + Commercial Sensing, 2020, Online Only

Abstract

In machine learning, backdoor or trojan attacks during model training can cause the targeted model to deceptively learn to misclassify in the presence of specific triggers. This mechanism of deception enables the attacker to exercise full control on when the model behavior becomes malicious through use of a trigger. In this paper, we introduce Epistemic Classifiers as a new category of defense mechanism and show their effectiveness in detecting backdoor attacks, which can be used to trigger default mechanisms, or solicit human intervention, on occasions where an untrustworthy model prediction can adversely impact the system within which it operates. We show experimental results with multiple public datasets and explain the reasons with visualization for effectiveness of the proposed approach. This empowers the war fighter to trust the AI on the tactical edge to be reliable and to become sensitive to scenarios with deception and noise where reliability cannot be provided.

Conference Presentation

Citation Download Citation

Zhaoyuan Yang, Nurali Virani, and Naresh S. Iyer "Countermeasure against backdoor attacks using epistemic classifiers", Proc. SPIE 11413, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications II, 114130P (22 April 2020); https://doi.org/10.1117/12.2558255

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available