This PDF file contains the front matter associated with SPIE Proceedings Volume 11733, including the Title Page, Copyright Information, and Table of Contents.

Introduction to SPIE Defense and Commercial Sensing conference 11733: Geospatial Informatics XI

In recent years, deep neural-networks have gained popularity in maritime detection problems. Successes in deep- learning have been due, partially, to the controlled and constrained nature of the training dataset. However, remote sensing data are highly variable, highly unconstrained, and lack both the quality and quantity of labeled and curated training samples usually required for current state-of-the-art approaches. In this paper we address a lack of class coherency across datasets at varying spatial resolutions by introducing a large 42-class synthetic dataset, Maritime Vessels at Varying Resolutions (MVVR-42). We leverage MVVR-42 in our experimentation, taking advantage of the ability to easily render imagery at varying resolutions and augmenting the training set to produce data points to aid in sensor selection for remote systems. Using state-the-art detection models like YOLO and FasterRCNN, we explore the effect of spatial resolution on performance in ship detection tasks.

The potential of synthetic image generation in machine learning applications has a myriad of both benefits and drawbacks. Synthetic datasets are valuable when real datasets are costly to acquire or difficult to share. Although synthetic imagery has improved in quality over the years, automated evaluation of the synthetic data remains challenging. For example, the Fréchet inception distance does not always correlate with perceptual quality and the perceptual path length operates at the model level rather than the data level. In this work, we propose a new evaluation metric that both correlates with perceptual quality and operates at the data level so that it can function on datasets from any domain. We do this by mapping high dimensional images into a lower dimensional, perceptually disentangled embedding space and computing the distance between distributions of embeddings within this space. We utilize a convolutional autoencoder trained with a linear combination of pixelwise and perceptual losses to perform the mapping and use existing metrics to measure the distance between the distributions of embeddings. We demonstrate efficacy of this metric on the CIFAR-10 dataset.

Infrared to Visible (IR2VIS) image registration suffers from the challenge of cross-modal feature extraction and matching. Conventional methods usually design the same keypoint detector for both Infrared (IR) and Visible (VIS) images. The VIS images are even converted to gray-scale images before the keypoint detection. IR and VIS gray-scale images have different properties which might not be applicable for the same feature detector. Therefore, this paper proposes an IR2VIS image registration method, namely, Image Translation for Image Enhanced Registration (ITIER). The IR images are first translated to realistic VIS images by Wavelet-Guided Generative Adversarial Network (WGGAN) for the convenience of cross-modal feature detection. Then the keypoint detection and matching and the homography transformation, which have been integrated into our ITIER, are conducted on the translated and original VIS images. Experimental results demonstrate that the IR2VIS image registration accuracy is greatly enhanced by the image-to-image translation procedure, which transfers IR images to realistic VIS images.

Deep convolutional neural networks (CNNs) have proven to be successful for learning task-specific features that achieve state-of-the-art performance on many computer vision tasks. For object detection applications, the introduction of region-based CNNs (R-CNNs), and its successors, Fast R-CNN and Faster R-CNN, has produced relatively high accuracies and run-time efficient results. With Faster R-CNN, a region proposal network (RPN) is employed to share convolutional layers for both object proposals and detection with no loss in accuracy. However, these approaches are trained in a fully supervised manner, where a large number of samples for individual object classes are required, and classes are pre-determined by manual annotation. Large-scale supervision leads to limitations in utility for many real-world applications, including those involving difficult-to-detect, small, and sparse target objects in variable environments. Alternatively, exemplar learning is a paradigm for discovering visual similarities in an unsupervised fashion from potentially very small numbers of examples. Surrogate classes or outliers are discovered via the inherent empirical characteristics of the objects themselves. In this work, we merge the strengths of CNN structures with pre-processing steps borrowed from exemplar learning. We employ a semi-supervised approach that combines the ability to use generically-learned class-relatedness with CNN-based detectors. We train and test the approach on a set of aerial imagery generated from unmanned aircraft systems (UAS) for challenging real-world, small object detection tasks.

Domain adaptation is a technology enabling Aided Target Recognition (AiTR) and other algorithms for environments and targets where data or labeled data is scarce. Recent advances in unsupervised domain adaptation have demonstrated excellent performance but only when the domain shift is relatively small. This paper proposes Targeted Adversarial Discriminative Domain Adaptation (T-ADDA), a semi-supervised domain adaptation method by extending the Adversarial Discriminative Domain Adaptation (ADDA) framework. By providing at least one labeled target image per class, T-ADDA significantly boosts the performance of ADDA and is applicable to the challenging scenario where the set of targets in the source and target domains are not the same. The efficacy of T-ADDA is demonstrated by several experiments using the Modified National Institute of Standards and Technology (MNIST), Street View House Numbers (SVHN), and Devanagari Handwritten Character (DHC) datasets and then extended to aerial image datasets Aerial Image Data (AID) and University of California, Merced (UCM).

Object detectors on autonomous systems often have to contend with dimly-lit environments and harsh weather conditions. RGB images alone typically do not provide enough information. Because of this, autonomous systems have an array of other specialized sensors to observe their surroundings. These sensors can operate asynchronously, have various effective ranges, and create drastically different amounts of data. An autonomous platform must be able to combine the many disparate streams of information in order to leverage all of the available information while creating the most comprehensive model of its environment. In addition to multiple sensors, deep learning-based object detectors typically require swaths of labeled data to achieve good performance. Unfortunately, collecting multimodal, labeled data is exceedingly labor-intensive which necessitates a streamlined approach to data collection. The use of video game graphics engines in the production of images and video has emerged as a relatively cheap and effective way to create new datasets. This helps to close the data gap for computer vision tasks like object detection and segmentation. Another unique aspect of using gaming engines to generate data is the ability to introduce domain randomization which randomizes certain parameters of the game engine and generation scheme in order to improve generalization to real-life data. In this paper, we outline the creation of a multi-modal dataset using domain randomization. Our dataset will focus on the two most popular sensors in autonomous vehicles, LiDAR and RGB cameras. We will perform baseline testing of an object detector using a data-fusion deep learning architecture on both our synthetic dataset and the KITTI dataset for comparison.

The study of the vulnerabilities of unmanned aerial vehicles (UAVs) to a wide range of counter-UAV (C-UAV) attacks is well-known and established. While most C-UAV lies in the cyber, sensing, and kinetic domains, there is an emerging threat to these platforms from the perspective of adversarial machine learning (ML). Modern ML approaches are vulnerable to attacks that are largely imperceptible to humans and can be extremely successful in causing undesired false positives and false negatives in real-world scenarios. With the proliferation of ML algorithms throughout the software stack of modern UAVs, these new attacks could have real implications in the security of UAVs. A successful attack on a UAV has real-world consequences such as a collision or takeover of the platform itself. We describe a methodology for understanding the vulnerability of UAVs to these attacks by threat modeling each potential state and mode of the UAV, from powering-on, to various mission modes. In this threat modeling, we consider well-known attacks on deep learning approaches, such as state-of-the-art object detection, but also explore the possibility of novel attacks on traditional computer vision approaches, such as stereo algorithms. We examine one potential threat vector and evaluate the likelihood of success of such an attack given the current progress of adversarial ML.

With the vast amounts of unstructured data that is accrued through surveillance, body-cameras, mobile phones, and other sources, there is a need to perform data synthesis into natural language through automated methods. Recent advances in machine learning have enabled compression of data sequences into short, compact, informal summaries as keyframes and video thumbnails. Additionally, the capability to generate text that describes an overall image or full motion video has rapidly increased. However, generating text in more formal structures, such as reports, remains a relatively unsolved area of Natural Language Generation (NLG). This work is an initial attempt to understand the gap in the data summarization and document generation problem, specifically for the generation of situation reports and study the data annotations necessary to implement an end-to-end pipeline that would ingest data, summarize it, and generate a situation report for easy consumption by a user.

Modeling water distribution networks facilitates assessment of system resiliency, improvements for demand forecasting, and overall optimization of limited system resources. This report serves as an introduction to the fundamentals of Water Distribution Networks (WDN) and provides insight into modern approaches of modeling these critical infrastructures. We provide an overview of core components within a WDN and a literature review and summary of current modeling approaches. We investigate and compare three unique vulnerability assessment methodologies based upon a graph theoretic approach. We assess the merits of each approach and the associated analytics implemented to identify the critical nodes and edges within a network. The first method utilizes a topological approach and segments the network into valve-enclosed sections. Analysis is centered on a depth-first search to identify nodes which would impact the most downstream nodes. The second method fuses topological and hydraulic data calculated using software such as EPANET. Various centrality measures corresponding to portion of network flow are used to assess vulnerability. The last method focuses on pipe (edge) vulnerability, incorporating information such as the average daily flow through each pipe as key parameters to algorithmically assess vulnerability.

Nowadays, we are facing a tremendous increase in the number of forest fires around the world. While in 2010, the world had 3.92Gha of forest cover, covering 30% of its land area, in 2019, there was a loss of forest cover of 24.2Mha according to the Global Forest Watch institute. These fires can take different forms depending on the characteristics of the vegetation and the climatic conditions in which they develop. To better manage this and reduce human, economic and environmental consequences, it is crucial to consider artificial intelligence as a mean to predict the new probable burned area. In this paper, we present FU-NetCast, a deep learning model based on U-Net, past wildfires events and weather data. Our approach uses an intelligent model to study forest fire spread over a period of 24 hours. The model achieved an accuracy of 92.73% and an AUC of 80% using 120 wildfire perimeters, satellite images, Digital Elevation Model maps and weather data.

In recovering from cyber attacks on power grids, restoration steps have also included disconnecting parts of the network to prevent failures from propagating. Inspired by these reports we investigate how this may be performed optimally. We show how throughput can indeed be increased by selectively disconnecting links when the network is currently stressed and unable to meet all of the demands. We also consider the impact of this option on critical node analysis. For defensive as well as offensive scenario planning it is important to be able to identify the critical nodes in a given network. We show how ignoring this option of disconnecting links can lead to misidentifying critical nodes, overstating the impact of these nodes. We outline an iterative procedure to address this problem and correctly identify critical nodes when link disconnection is included in the recovery scheme.

It is estimated that about 60 percent of total global methane emissions are thought to be from anthropogenic sources and about 40 percent from natural sources. Anthropogenic sources encompass a wide range of human activities, including food and energy production and waste disposal. Livestock (through fermentation processes in their digestive system that generates methane and manure management), rice cultivation, landfills, and sewage account for 55-57 percent of global anthropogenic emissions. This paper investigates methane emissions from agricultural land-use and livestock (e.g., poultry and cattle) farming practices in Delaware. Laser-based point sensing can provide a higher spatial and temporal resolution that can complement satellite observations to identify individual sources and broader geographical areas. A detailed understanding of their sources and sinks is necessary to model emissions profile accurately. This paper shows field measurements of methane using mid-IR laser-based sensors and validation with satellite data. We conducted our field deployment locally in the Delaware, Kent, and Sussex county regions focusing on high methane emitting areas. We used the TROPOspheric Monitoring Instrument (TROPOMI) methane satellite data to get a unified emissions map of methane production in Delaware by comparing our ground-based measurements with the satellite data. Furthermore, we examined the satellite data for long-term methane emissions trends to quantify 2020 average methane emissions.

Considering the public health impact of a global pandemic, the reliance on data to understand disease outbreak is important now more than ever. Malaria is the most common mosquito-transmitted disease endemic to certain regions, leading to millions of serious illnesses and deaths each year. Because mosquito vectors are sensitive to environmental conditions such as temperature, precipitation, and humidity, it is possible to map areas currently or imminently at high risk for disease outbreaks using satellite remote sensing. In this paper the authors propose the development of an operational geospatial system for malaria early warning. This can be done by bringing together geographic information system (GIS) tools, artificial neural networks (ANN) for efficient pattern recognition, the best available groundbased environmental data such as epidemiological and vector ecology data, and current satellite remote sensing capabilities. The authors use Vegetation Health Indices (VHI) derived from visible and infrared radiances measured by satellite-mounted Advanced Very High Resolution Radiometers (AVHRR) and available weekly at 4- km resolution as one predictor of malaria risk in Bangladesh. As a study area, we focus on Bangladesh where malaria is a serious public health threat. The technology developed will, however, be largely portable to other countries in the world and applicable to other disease threats. A malaria early warning system will be a boon to international public health, enabling resources to be focused where they will do the most good for stopping pandemics, and will be an invaluable decision support tool for national security assessment and potential troop deployment in regions susceptible to disease outbreaks.