Search and Rescue (SAR) operations following natural or anthropogenic disasters are often hindered by smoke, rain, fog and haze. Developing methods that combat Degraded Visual Environments (DVE) and ensure the rapid detection of survivors and rescue personnel after a disaster is crucial to reducing mortality. This paper proposes a novel AI scheme to detect static and moving objects using snapshot NIR hyperspectral data. The proposed model leverages spatial features through object detection to identify objects and, at the same time, applies a semantic segmentation algorithm based on spectral features to validate the presence of firefighters within the detected bounding boxes. The training is optimised for real-time high-performance inference by exporting it to TensorRT. This approach has been successfully demonstrated in various realistic scenarios with an F1-score of 0.923 and 77.9 frames per second.

This conference presentation was presented at SPIE Defense + Commercial Sensing 2023.

Three previously proposed machine learning techniques (a technique based on applying gradient descent training to a fractional value expert system, the fractional value expert system technique’s real time hardware implementation, and a Boolean expert system hardware implementation) are evaluated for application to defense-relevant real time processing challenges. These techniques are defensible, meaning that their decisions are constrained by logical pathways that can be reviewed by a human prior to a decision being made and acted upon by the system (as opposed to simply explained afterwards). These techniques and several conventional techniques are evaluated under multiple defense-relevant real time processing scenarios.