Measuring resiliency of system of systems using chaos engineering experiments

Thomas Bailey; Patrick Marchione; Pete Swartz; Raed Salih; Michael R. Clark; Robert Denz

doi:10.1117/12.2632779

30 May 2022 Measuring resiliency of system of systems using chaos engineering experiments

Thomas Bailey, Patrick Marchione, Pete Swartz, Raed Salih, Michael R. Clark, Robert Denz

Proceedings Volume 12117, Disruptive Technologies in Information Sciences VI; 1211704 (2022) https://doi.org/10.1117/12.2632779
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

Chaos Engineering (CE), which Netflix introduced in 2008, is used by researchers to assess and find weaknesses in system resiliency. Such weaknesses can arise, when subsystems are individually robust, but that robustness disappears when multiple subsystems are paired together in a System of Systems (SoS). CE researchers develops methods and metrics for finding such fragilities. In this paper, we expand previous examinations of CE experimentation for SoS and introduce Security Chaos Engineering (SCE) for SoS. These SCE experiments include terminating message service, flooding multi queues/message, and injecting corrupted Service. SCE assumes compromise by adding a malicious actor to the tests that can induce adversarial failures into a SoS. For our SoS testbed, we instantiated a virtual Unmanned Aerial Vehicle (VUAV). We use the open-source Chaos Toolkit to run consistent CE and SCE experiments on the VUAV. Chaos Toolkit with SCE exposes the VUAV attack surfaces to evaluate performance and system security. This research allows us to establish an understanding of baseline system performance and gaps in procedures, techniques, and tools from the state of the art as applied to DoD-relevant systems like SoS. We use the load placed on the Central Processing Unit (CPU) and Random-Access Memory (RAM) by the VUAV as metrics for baseline performance. The results showed that these two metrics did not provide enough fidelity in where CE/SCE creates failures. Feeding these results into the CE methodology allows for additional metrics to better pinpoint failures with CE/SCE testing.

Conference Presentation

Citation Download Citation

Thomas Bailey, Patrick Marchione, Pete Swartz, Raed Salih, Michael R. Clark, and Robert Denz "Measuring resiliency of system of systems using chaos engineering experiments", Proc. SPIE 12117, Disruptive Technologies in Information Sciences VI, 1211704 (30 May 2022); https://doi.org/10.1117/12.2632779

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available