This PDF file contains the front matter associated with SPIE Proceedings Volume 12540, including the Title Page, Copyright information, Table of Contents, and Conference Committee listings.

Object detection in aerial images is a challenging task as some objects are only a few pixels wide, some objects are occluded, and some are in shade. With the cost of drones decreasing, there is a surge in the amount of aerial data, so it will be useful if models can extract valuable features from the aerial data. Convolutional neural networks (CNN) are a useful tool for object detection and machine learning applications. However, machine learning requires labeled data to train and test the CNN models. In this work, we used a simulator to automatically generate labeled synthetic aerial imagery to use in the training and testing of machine learning algorithms. The synthetic aerial data used in this work was developed using a physics-based software tool called Mississippi State University Autonomous Vehicle Simulator (MAVS). We generated a dataset of 871 aerial images of 640x480 resolution and implemented Keras-RetinaNet framework with ResNet 50 as backbone for object detection. Keras-RetinaNet is one of the popular object detection models to be used with aerial imagery. As a preliminary task, we detected buildings in the synthetic aerial imagery and our results show a high mAP (mean Average Precision) accuracy of 77.99% using the state-of-the-art RetinaNet model.

LiDAR-based 3D semantic segmentation is one of the most widely used perception methods to support scene understanding of self-driving vehicles. Most publicly available LiDAR datasets for driving scene segmentation, such as SemanticKITTI, nuScenes, and SemanticPOSS, provide only a single type of LiDAR configuration. Therefore, testing a trained model with a different channel configuration than the training dataset is sometimes inevitable in real-world applications. Despite the significance of this LiDAR channel mismatch problem in the machine learning pipeline, little research has focused on investigating the impact of the LiDAR configuration shift on a model’s test performance. This paper aims to provide referenceable baseline experiments for the LiDAR configuration shifts. We explore the effect of using different LiDAR channels when training and testing a 3D LiDAR point cloud semantic segmentation model, utilizing Cylinder3D for the experiments. A Cylinder3D model is trained and tested on simulated 3D LiDAR point cloud datasets created using the Mississippi State University Autonomous Vehicle Simulator (MAVS) and 32, 64 channel 3D LiDAR point clouds of the RELLIS-3D dataset collected in a real-world off-road environment. Our experimental results demonstrate that sensor and spatial domain shifts significantly impact the performance of LiDAR-based semantic segmentation models. In the absence of spatial domain changes between training and testing, models trained and tested on the same sensor type generally exhibited better performance. Moreover, higher-resolution sensors showed improved performance compared to those with lower-resolution ones. However, results varied when spatial domain changes were present. In some cases, the advantage of a sensor’s higher resolution led to better performance both with and without sensor domain shifts. In other instances, the higher resolution resulted in overfitting within a specific domain, causing a lack of generalization capability and decreased performance when tested on data with different sensor configurations.

As autonomous vehicles rapidly transition to fully automated operations, the ability of these systems to make critical safety decisions in real-time remains of paramount concern. Successful implementation will rely on maintaining the highest possible fidelity of the underlying image data while operating in the challenging automotive light detection and ranging (LIDAR) environment. In addition to three dimensional (3D) depth data, some proposed systems seek to overlay data from multiple sources in order to further improve decision quality and safety margin. The requirement to establish and maintain system calibration over life, including overlay of various fields of view suggests the need for an optical reference that could be used to routinely recalibrate the system in the field. The ability to validate and re-calibrate LIDAR systems in the field without the need for operator intervention would also serve to ensure consistent data quality as well as removing the need to take vehicles offline for periodic re-calibration. Diffractive-Optical-Elements (DOEs) are well suited for use as optical calibration references. In addition to their small size and lithographic precision, fused silica DOEs exhibit extreme stability and mechanical strength, potentially allowing for direct integration into the LIDAR system itself. In this paper, we examine the use of DOEs as LIDAR calibration standards, including the impact of material properties on the stability of the generated reference patterns and consequently on the reliability of the imaging system data.

Three-dimensional object detection is vital for understanding the autonomous vehicle driving environment. Different sensors are used for this purpose, such as cameras and LiDARs. Camera sensors are rich in color and texture information. However, cameras are unsuitable for 3D object detection due to the lack of depth information. Additionally, camera sensors are vulnerable to bad weather, such as snow, fog, and night driving. Autonomous driving needs a fast and accurate perception system for robust operation of the following pipeline, such as path planning and control. LiDAR is a commonly used sensor for 3D object detection because of its 3D information. However, its lack of color and texture information reduces the classification and detection performance. Therefore, there is no complete sensor that works for all situations. In this work, we propose a multi-modal fusion 3D object detection model for autonomous driving to use the best out of LiDAR and camera sensors. The model comprises a feature extraction network and a fusion network. Feature extraction networks transform the image and point cloud data into high-level features before fusion. Then, features from images and LiDAR data are fused. The experimental result on the nuScenes dataset shows the model’s competitive performance for 3D object detection.

Global trends towards mobility and miniaturization along with increasing demands to improve the performance and efficiency of sensor systems, especially in autonomous systems, have led to a need to decrease their size, weight, and power (SWaP). For millimeter wave (mmWave) and optical sensing, optically transparent mmWave antennas provide a solution to meet this need by offering a multi-modal, single aperture system with applications in vehicle windshields, solar cells on satellites, and drone camera lenses, among others. However, integrating these sensing systems while optimizing both the mmWave properties and optical sensing capabilities forces competing design and fabrication requirements. Currently, there also exists limited research and literature on a fabrication procedure for these micromesh structures. This paper presents a photolithographic procedure for fabricating micro-scale optically transparent mmWave transmission lines (TLs) and antennas suitable for use in the design optimization processes. It investigates the fabrication of varying line width versus line gap dimensions as well as the fabrication of different mesh unit cell designs. The fabrication procedure was designed to achieve down to 1-2 μm features and is replicable for both single-sided and dual-sided designs. Fabricated micro-mesh structures exhibit strong performance in both operating regimes, with optical transmission ranging from 80- 95% transmittance in the visible spectrum and high conductivity with sheet resistance less than 1 ohm-per-square.

Time-of-flight LiDAR scenario of road in real time and provide high quality 3-D point cloud with reliable ranging information to assist autonomous driving. In general, limited by speed of light and point cloud density, there is a tradeoff between detection range and the angular resolution. Another challenge is the difficulty to detect weak signals reflected from ultra-long distance targets. Increasing transmission laser power is hard to be a proper solution due to eye safety. In this paper, we demonstrate feasibility of detecting targets at ultra-long distance using a time-of-flight LiDAR with decent point density. At the meanwhile, the transmission laser power at wavelength of 1550 nm is well within eye safety standards. Results show that pedestrian can be detected in a range of 500 m, small vehicle is detected at 900 m with framerate of five and 1000 m with framerate of two. Truck is detectable at a distance up to 1300 m.

Unexpected faulty sensors can seriously compromise the mission of strategic assets driven by artificial intelligence (AI), autonomous unmanned vehicles, and robots. In life threatening operation, these devices must make real time decisions based on the vital information collected from the sensors. AI is vulnerable to sensor failures caused both by malicious actions and by system breakdown. We propose a new scheme that is providing, in real-time, a unique fingerprint of a pair of differential sensors operating as a Physically Unclonable Function (PUF). The solution can be implemented with little computational complexity or power usage on most IOT devices driven by AI and a set of sensors. The differential circuitry ensures the authenticity of the source of the information generated from the sensors, thereby preventing malicious actors from deliberately falsifying sensor data. Without the fingerprint from the sensors, these actors will have reduced opportunities to access the server thus limiting this attack vector. Faulty sensors are identifiable since their erratic fingerprints differ from those identified at launch. The data originating from bad sensors is thereby ignored in favor of the data generated by redundant sensors when available in the system. This technology also ensures that the accuracy of the information from the sensor is within normal tolerances for a real-time AI environment. The fingerprints are used to create unique encryption keys for standardized post quantum cryptographic based communication. We are hereby demonstrating experimentally the relevance of the proposed differential circuits for these targeted applications.

Various tools are now available to assist the roboticist in developing autonomy algorithms for tasks such as path planning or collision avoidance. Many tools support the integration of live or simulated RGB cameras, LIDAR, radar, and IMU sensors. This paper will describe adding an RF sensor. The proposed RF sensor detects radio and locates emitters in the environment for the purpose of collision avoidance. We outline an approach to share data to help locate and avoid collisions. The protocol is designed to maximize safety, privacy, security, timeliness, and other desirable properties discussed in the paper. Preliminary results are shown to illustrate the concepts.

Physically Unclonable Functions (PUFs) are a crucial part of enhancing the security of Internet of Things (IoT) devices and Cyber-Physical Systems (CPS). These devices play a significant role in numerous infrastructure systems including healthcare, business, industry, etc. Such devices typically have the least amount of storage, processing and energy; which causes them to be the weakest component in the security of the system. The CPS is an innovative paradigm that aims to bring together the physical and digital worlds that comprise today’s environment. We propose methods to use pairs of vital sensors and differential amplifier circuitry to derive fingerprints in real time for a CPS. A custom Printed Circuit Board (PCB) used as an Arduino shield, a stepper motor and 3D designs for the test fixture are key components in collecting data from the analog samples that are being used for the enrollment of the sensor based-fingerprint. The output of the differential amplifier circuitry at the various sensors’ orientations will be stored by a server and used as an authentication of the CPS and in determining the health of the sensor in the CPS. We examine the data obtained from two different sensor-pairs and evaluate its reliability and uniqueness using Hamming distance.

Cryptographic protocols are widely deployed to enhance the security of sensitive data shared across autonomous networks. Privacy and security issues are exacerbated due to increased vulnerability of the cryptographic key to various types of attack. It is extremely challenging to distribute cryptographic keys in contested areas on a real-time basis, therefore several protocols have been developed that use the fingerprints of electronic devices embedded in cyber physical systems for one-time use key generation. Various autonomous systems, including vehicles, robots, and industrial machines may be protected by these fingerprints, thus allowing the use of onetime encryption keys, thereby reducing the cybersecurity attack surface. To mitigate certain attacks and enhance security we propose methods to inject random noise in these keys, as well as the associated key recovery schemes. The benefit of noise injection is the ability to incrementally confuse the hacker while adding an extra layer of authenticity. The random percentage of noise injected in the key can only be identified and verified by an authorized party. An unexpected increase or decrease in the percentage of noise notifies the server that the autonomous system is potentially under attack. Experimentally, we demonstrate how noise levels up to 30% of the cryptographic keys can be injected, and how the temperature can alter the percentage of noise. We also quantify methods to increase the errors due to the instability of the fingerprints. Additionally, we review different strategies of noise injection and how we may leverage the injected noise for stronger authentication.

Response-based cryptography (RBC) is an unequal authentication protocol that is used to secure devices in resource-constrained and noisy environments. Such noisy environments may include securing radar on UAVs and in other autonomous settings. The protocol allows for client devices to be equipped with physical unclonable functions (PUFs), which are hardware elements that assign a unique fingerprint to a client device. Because PUFs are physical devices, they have a bit error rate, meaning that their output at a given address (a seed) may vary over time. Thus, a critical component of RBC is the use of a secure server equipped with hardware that enables very high seed search throughput (e.g., compute clusters, graphics processing units (GPUs)), that can find the target seed that authenticates the device. Once the client has been authenticated by the server, the seed can be used for encryption and other tasks (PKEs, DSAs, or KEMs) to enable secure communication. The state-of-the-art research in this area has assumed that the seed search space is uniform, where all seeds searched on the server have the same probability of authenticating the device. Due to high computational cost, this limits the application of PUFs to those with low levels of noise. In this paper, we propose a new seed search algorithm that uses PUF enrollment data to assign a probability that a seed will authenticate a user’s device. This information allows us to increase the level of noise on client devices to enable higher levels of security in addition to being more robust to noisy environments. Additionally, we include an analysis of our proposed work and compare it to the state-of-the-art. We find that our proposed method is at least 3.38× faster than prior work and can search up to a Hamming distance of 5 under 10 seconds using 64×CPU Cores.

The deployment of post-quantum cryptographic algorithms currently under standardization by NIST have the potential to mitigate quantum computer based attacks, which present a whole new range of cybersecurity challenges. We recognize post-quantum cryptography (PQC) key distribution, across autonomous systems operating in contested areas, as a particularly difficult problem to solve. Verifying the origin and integrity of mission instructions is of extreme importance, especially in the context of autonomous vehicles, missiles, and drones; the lack of which can enable opponents to leverage attacks effecting catastrophic damage to life, property, and strategy. This work proposes a protocol for controlled key distribution through the use of PQC algorithms, strengthened by fingerprints of embedded devices. With this framework, we can assure the origin and correctness of digital information sent to autonomous systems, and enforce that only authorized members can send instructions to these devices. This protocol replaces the deterministic, pseudo-random seed in these PQC protocols with a fingerprint derived from the physical disorder of the embedded device to enhance security and maintain the integrity of the system. We demonstrate how this system can be used to send instructions, verify the origin of the instructions upon receipt, and certify the integrity of the instruction packet received. We also discuss situations in which an attacker attempts to falsify information and how to detect that malicious action. We measure the quality and performance of this prototype system by measuring the latency and bit error rates. We demonstrate this prototype system using the CRYSTALS-DILITHIUM digital signature algorithm (DSA).

For autonomous driving, pedestrian and road signs detection are key elements. There is much existing literature available addressing this issue successfully. However, the autonomous system requires a large and diverse set of training samples and labeling in real-world environments. Manual annotation of these samples is somewhat challenging and time-consuming. In this paper, our goal is to get better detection accuracy with minimal training data. For this, we have employed the active learning algorithm. Active learning is a useful method that selects only the effective portion of the dataset for training and reduces annotation costs. Though it uses only a small amount of the training data, it provides a high detection accuracy. In this work, we have chosen the deep active learning model for object detection via the probabilistic model of Choi et al. and modified the depth scale of different layers in the backbone. As real-world data may contain noise, motion, or other disruptions, we modified the original model to obtain improved detection results. In this experiment, we create a customized dataset that contains pedestrians, road signs, traffic lights, and zebra (or pedestrian) crossings to deploy the active learning algorithm. The experimental results show that the active learning model can produce good detection outcomes by accurately detecting and classifying pedestrians, road signs, traffic light, and zebra (or pedestrian) crossings.

We present a cost-effective testbed and advanced software architecture suitable for ADAS applications, which utilizes an onboard GPU for high-performance processing. By implementing an optimized parallel processing method, this innovative architecture can collect and synchronize multi-sensor data, such as camera and radar, with high precision. The software architecture dynamically allocates processing power to each data processing sub-module based on the utilized sensor hardware and optimizes the data flow accordingly. With the aid of parallel processing, the system can effectively reduce the time delay during data acquisition. Indoor and road stereo-camera and dual-radar data acquisition results are presented. Post-processing of the data includes the creation of disparity map depth images and combined radar point clouds identifying departing and approaching vehicles and objects.

Modern automobiles are commonly equipped with a variety of perception modalities, i. e. sensors, to implement various levels of Advanced Driver Assistance Systems (ADAS). The main sensing modalities include camera, light detection and ranging (LiDAR), radar, as well as inertial measurement and global positioning devices. In addition, ultrasonic and acoustic sensors complete the possible perception modalities. ADAS features autonomous driving systems that have been initially developed assuming ideal perception, i.e., weather conditions; however, in reality, the quality of a sensor modality may degrade significantly due to adherent dirt, dust, fog, wet weather, or changing illumination conditions on the road. It is important to quantitatively study the degradation of sensor modalities in non-ideal conditions such as fog, rain, snow, bright sunlight, dust, etc., in order to evaluate sensor degradation and its effects on higher-level functionalities such as lane-keeping, object detection, recognition, and tracking, as well as ego-motion estimation and localization. This paper quantitatively examines the variability of several LiDAR and RGB camera-based features in variable weather conditions, including clear sky, overcast, fog, rain, and snow. Specifically, we derived LiDAR point-cloud-based features such as mean and standard deviation of distances and point cloud size. Also, image color space features were examined, in addition to blurriness and structural detail. Furthermore, we perform feature selection to explore the importance of various combinations of features for the prediction of road weather conditions.

Conventional distributed formation control of multiple vehicle systems (MVSs) has two drawbacks: inflexible formation changes and explicit-value exchanges of vehicles’ information such as position and velocity among all vehicles. Since formation changes are needed, each vehicle is required to update its relative position which is quite difficult in large spatial applications. Firstly, the explicit-value exchanges possibly result in two critical issues. When each vehicle policy needs to keep its information confidential from another unexpected listener, the explicit-value exchanges are invalid for the privacy policies. Additionally, the explicit-value storing or exchanging signals or parameters are much more vulnerable and dangerous to security threats. This work proposes an approach to overcome the above challenges by taking advantage of model predictive control-consensus algorithms to achieve desired formations. We will also allow the computation to be effectively distributed among the vehicle agents according to their computational capabilities. Secondly, we use the highly secure encryption scheme that empowers all computations carried out in encrypted forms, including system parameters and signals. Our results are verified by the formation control of multiple vehicles working in large-scale environments where a ground station does not touch all vehicles due to limited communication ranges and security problems. Compared to cutting-edge studies, the formation of vehicles is still able to be changed securely by the ground station without updating new formations to all vehicles. Besides, the data privacy of each vehicle is preserved by encrypting all physical signals.

Radio frequency (RF) geolocation is the process of using receivers to estimate the general location of a transmitter. One method of RF geolocation is to measure the received signal strength (RSS) at different locations around the transmitter. RSS-based geolocation systems are less expensive and less complex than other systems. Unmanned Aerial Vehicles (UAVs) can be used as a mobile platform for RSS-based geolocation systems. This paper will explore RSS measurements as a method for low-cost geolocation on a UAV. The geolocation system combines with a mapping system on the UAV to produce a 3D map with the estimated location of a RF emitter. Experimental results show a location accuracy of about 12.5 m.

This paper presents a localization system for Unmanned Aerial Vehicles (UAVs), specifically designed for small UAVs to be used in a GPS-denied local area by using estimation algorithms incorporating camera and Light Detection and Ranging (LiDAR) sensor fusion. Localization techniques classically rely on Global Positioning System (GPS) information. However, GPS is subject to jamming. This paper proposes methods of localization without reliance on the GPS. This system utilizes Error-State Extended Kalman Filtering (ESEKF) and methods of camera to LiDAR sensor fusion to correct for error propagations in the aerial vehicle’s estimated location. Initial results from the GPS-denied navigation method showed that the location of the sUAV to an average error of 3.2 m was possible using only texel images and velocity measurements from an experimental flight.

The passing of a wheeled or tracked vehicle over soft or deformable soil creates ruts. The depth of these ruts is proportional to the weight of the vehicle and the soil trafficability; the ability of the soil to support traffic from vehicles. Assessing soil trafficability is often a manual and labor-intensive process. We evaluate the ability of lidar and depth cameras to detect changes in rut depth with the goal of minimizing manual or automated evaluation via soil strength testing. Our sensor-based approach mimics the process used by human operators when measuring rut depth. We compare this approach with machine-centered approaches with the goal of improving correlation between soil strength measurements and rut depth. In general, we find that all sensors are able to measure rut depth within the uncertainty bounds of soil and rut depth models for light vehicles.

Vehicle platooning allows a follower vehicle to reduce energy consumption by following closely behind a cooperative lead vehicle. In a previous work, we described a lidar-based vehicle detection and tracking approach that measures the extents of the back surface of the lead vehicle and allows for automated lane level positioning. In this work, we introduce a follower vehicle energy model and compare the performance of a vehicle centering strategy based on rear backplane geometry as opposed to center of mass and bounding box approaches. Energy efficiency improvements are discussed with respect to the computational complexity of each approach.

The objective of this paper is to detail the design and development of a unique VIL dynamometer system for testing lateral and longitudinal control using simulated and/or recorded data. A floating hub design decouples the vehicle hub rotation while continuing to support the vehicle mass on the tires and suspension of the vehicle. Long travel CV-joints are used to enable full angular displacement/turning radius of the steering wheels and couple the drive axles to the absorbing dynamometer motors. The steering wheels are placed on three degree of freedom (3DOF) motion plates allowing for rotation and translation during steering articulation. The reduced rotary mass and inertia of non-rotating tires however influences the tire-forces when performing lateral maneuvers. A common approach to this problem has been to attach a resisting actuator, like a chain drive, to the rotary base where the wheel is resting. Because additional degrees of freedom are introduced by the use of 3DOF motion plates, this approach is not suitable. To compensate for the lack of physical accuracy without introducing additional hardware to interfere with the wheel or rotary base, the modification of the performance curves directly associated with the electric power steering system is explored. Using experimentally obtained wheel forces, the power steering performance curve can be modified to a function of vehicle velocity. The vehicle’s current velocity can be obtained via CAN from its On-Board-Diagnosis (OBD) system. Improving the physical accuracy of hub-mounted dynamometer systems during lateral maneuvers introduces a cost and space-efficient research platform for the development and testing of automated driving systems and automotive components.

We propose a general mobile robot velocity and distance control design framework that relies on using the Keitz irradiance-flux equation relating the measured irradiance and the output power of far-UVC light fixtures along with an exponential decay model for evaluating the survival of viruses and pathogens subjected to farUVC exposure. Our mathematical framework is capable of computing the optimal distance from vertical target surfaces. Furthermore, given a moving mobile robot rover base with UV light towers mounted on it, we formulate a suitable optimal control problem to derive the optimal velocity profile of the mobile robot for achieving efficient disinfection in terms of disinfection time and energy expenditure.

One of the key features of a Wireless Sensor Network (WSN) device is its constrained resource availability and its security weakness. As a result, the research of the energy and security issues is crucial because of how much they influence WSN behavior. Traditional security procedures are not the best given the restricted resources of nodes, thus it is necessary to analyze new strategies in order to take this particular node attribute into consideration. In this study, a MAC layer impersonation attack is examined, and the performance of a network of four sensor nodes, a gateway, and a server is assessed using the LMAC and BMAC protocols in terms of the average energy used by each node in the WSN under consideration.

One crucial topic for scientific community analysis is the use of drones to support traffic in emergency and critical situations. In all those circumstances when users need to interact without access to other infrastructure, these devices can self-organize into a communication network called FANET. In order to keep the drones’ network architecture consistent, this study suggests to use an ant-based routing technique. It also demonstrates how energy consumption may be a significant factor in the success of a network activity. Energy, overhead, and delivery ratio have all been taken into consideration while evaluating performance.

Multi-waypoint navigation for autonomous robot is in high demand in real-world robotics applications including search and rescue, disaster response, and environment exploration. In this paper, a sensor-based methodology is proposed for validation of autonomous robot multi-waypoint navigation utilizing graph-based models with adjacent node selection. In addition to time and distance efficiency, the proposed graph-based models incorporate safety in relation to the robot’s environment as a driving feature. These models provide path planning for autonomous robot experiments. This methodology is implemented in a real-world environment with simulated dynamic obstacles utilizing a Clearpath Jackal unmanned ground vehicle (UGV) featuring four-wheel drive, GPS, odometry, and a Velodyne VLP-16 LiDAR as the experimental robot configuration. The LiDAR unit utilized has a one hundred meter range with a thirty degree vertical field of view, as the primary navigation sensor. It is utilized to map the environment prior to performing autonomous navigation experiments. This robot configuration is also presented in a simulated environment for more thorough evaluation. The environments to be examined are controlled with obstacles and environment features within the expected view of the LiDAR. The proposed graphbased model is evaluated in terms of planning efficiency from simulated to real-world environments including dynamic obstacle avoidance.

Multi-robot systems have distinct advantages over single-robot systems in terms of operating speed, dependability, and efficiency. They are commonly used for a variety of environmental exploration applications in conjunction with remote sensing. In this paper, a self-organizing map (SOM) neural network algorithm is developed for multirobot task allocation and formation control. Multi-robot task allocation involves controlling a swarm of mobile robots to achieve designated task locations with cooperation and coordination of each robot. This framework is applicable to autonomous remote sensing by a swarm of robots, improving the efficiency and reliability of remote sensing in large environmental exploration areas. In addition to global task allocation, it is also important to control the formation of multi-robots. Each robot has a moving speed, and its field of view describes the visual range perpendicular to the direction of movement. The proposed framework utilizes a particle swarm optimization algorithm to generate the shortest planned trajectory for the multi-robot system. Additionally, a path smoothing technique is used to reduce the overall path and stress on the robots. The simulation studies validate the effectiveness of the SOM-based task allocation and formation control. It can deal with complicated cases such as that the formation changes its geometrical shape to adapt to the new environment in exploration.