1.

SAR Processing

In SAR processing, traditional results mostly utilize hand-crafted features for feature extraction. The following papers utilized deep CNNs to extract higher-quality features for classification and change detection in SAR imagery analysis. Amrani et al. in “Deep feature extraction and combination for synthetic aperture radar target classification” utilized a pretrained VGG-S net that was fine-tuned on MSTAR SAR data to extract features and a K-nearest neighbor algorithm was used to classify the results. Liu et al. in “Change detection in multitemporal synthetic aperture radar images using dual-channel convolutional neural network” utilized dual CNN channels to extract deep features from SAR imagery for change detection. It is worth noting that their algorithm required no preprocessing or presegmentation. Quach in “Convolutional networks for vehicle track segmentation” overcame the problems of continuity and parallelism in detecting vehicle tracks using current methods by applying small-sized dilated convolutional networks, which exponentially increase the network’s receptive field size in a small number of layers. Dilated convolution places spaces between the pixels.

2.

Ocean Processing

The oceans cover roughly 71% of the Earth’s surface, making ocean remote sensing a very important task. Yao et al. in “Ship detection in optical remote sensing images based on deep convolutional neural networks” is a complicated problem due to the small size of ships and interference from clouds, waves, etc. A deep CNN extracts features and a region proposal network discriminates ships and provides accurate detection bounding boxes. Lima et al. in “Application of deep convolutional neural networks for ocean front recognition” investigated AlexNet, CaffeNet, GoogLeNet, and VGGNet and then developed a custom CNN with fewer layers for this task. In addition to detecting ocean fronts, they also classified them into weak and strong fronts (based on gradient intensity). The reduced model had the smallest learning time and all networks achieved high results despite the small number of training samples.

3.

Classification and Labeling

Maskey et al. in “Deep learning for phenomena-based classification of Earth science images” utilized AlexNet and fine-tuned it on remote sensing imagery in order to classify imagery based on Earth science phenomena, such as dust, hurricane, smoke, etc. The CNN-based approach provided superior results and demonstrated that transfer learning works from AlexNet trained on the ImageNet database. Ha et al. in “Deep convolutional neural network for classifying Fusarium wilt of radish from unmanned aerial vehicles” utilized unmanned aerial vehicles (UAVs) to detect Fusarium wilt in radishes. The field are segmented into radish, ground and mulch, then the deep learning system identifies infected radishes with much higher accuracies than conventional methods. Sun et al. in “Semantic labeling of high-resolution aerial images using an ensemble of fully convolutional networks” used a fully convolutional networks (VGG and ResNet) were augmented with cross-scene learning and fused the results with a conditional random field graph method. To handle the large size of aerial imagery, a split-and-merge method was employed which split the image into tiles, and each tile creates a belief map, which are then merged to form an overall belief map.

Maltezos et al. in “Deep convolutional neural networks for building extraction from orthoimages and dense image matching point clouds” uses height information derived from the a dense image matching algorithm (which does not require LIDAR data or LIDAR/image co-registration) as additional inputs to the deep learning system to identify buildings. The system outperforms shallow methods that attempt LiDAR/image fusion.

4.

Multimodal

Chen et al. in “Knowledge-guided golf course detection using a convolutional neural network fine-tuned on temporally augmented data” use a knowledge-driven region proposal, a CNN detector and knowledge-driven postprocessing. Knowledge-derived rules are applied to propose candidate golf regions. Temporal data augmentation is used to enhance the training data. A final postprocessing step removed errors.

Zhang et al. in “Collaborative classification of hyperspectral and visible images with convolutional neural network” utilized a CNN to extract deep spectral features. Next, effective binarized statistical image features are learned as contextual basis vectors for the high-resolution VIS image, followed by a classifier. Then decision fusion unites the spectral and spatial data and statistical data together.

5.

Spectral-Spatial Processing

Zhao et al. in “Hyperspectral anomaly detection based on stacked denoising autoencoders” uses both spectral features and fused features extracted via a stacked denoising autoencoder from clustered data to detect anomalies is hyperspectral data.

Abdi et al. in “Spectral-spatial feature learning for hyperspectral imagery classification using deep stacked sparse autoencoder” uses an unsupervised stacked sparse autoencoder to extract high-level feature representations of joint spectral-spatial information. A soft classifier is then used to fine-tune the deep learning architecture.

6.

Object Tracking and Recognition

Zheng et al. in “Object tracking by transitive learning using perspective transformation with asymptotic stability” employs multiple frames of local invariant features on and around the object are used to build an object and context template. A nonparametric learning algorithm using transitive matching perspective transformation. The asymptotic stability is shown to be drift-free in terms of long-term tracking. Wang et al. in “DeepPlane: a unified deep model for aircraft detection and recognition in remote sensing images” proposes a model with dual correlative deep networks: the first generates object proposals as well as feature maps; the second network is cascaded upon the first to perform classification and box regression in one shot. The “inception module” which consists of parallel layers of convolutions and pooling and a concatenation layer. This module contains several $1\times 1$ convolution layers that provide feature dimensionality reduction.

Marcum et al. in “Rapid broad area search and detection of Chinese surface-to-air missile sites using deep convolutional neural networks” put forth a deep CNN-based chip detection followed by spatial clustering to rapidly narrow down very large areas for identifying surface-to-air missile sites. The search times were sped up by about 81 times.

7.

Architectural Studies

Luo et al. in “Do deep convolutional neural networks really need to be deep when applied for remote scene classification?” analyze five pre-trained networks (AlexNet, CaffeNet, VGG-VD16, GoogLeNet, Resnet) and show that the features learned in shallow layers of deep CNNs are not general enough for remote scenes and that the depth of CNNs seemingly enhances the generalization power of learned features and is essential for remote scene classification. They also provide a history showing how networks have become deeper over time, and provide a nice chronologic history over time of different approaches classification accuracies on the UC Merced.

8.

Domain Adaptation

Ma et al. in “Deep neural network-based domain adaptation for classification of remote sensing images” utilizes class centroid alignment is used for unsupervised domain adaptation (assuming that the class labels are only available in the source domain). Hyperion, NCALM, and WorldView-2 imagery were analyzed. Also, different network configurations were studied and optimization equations are provided in appendices.

9.

Feature Extraction

Karim et al. in “Comparative analysis of feature extraction methods in satellite imagery” examined several different feature extraction methods for their ability to discriminate in shadowed regions of the image.

10.

Survey Paper

Ball, Anderson, and Chan contributed a survey paper, “Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community,” which provides a list of challenges and open problems in deep learning for remote sensing, discusses modifications of DL architectures for remote sensing, provides an overview of deep learning tools, and gives an extensive summary of remote sensing datasets. Challenges related to (i) inadequate data sets, (ii) human-understandable solutions for modeling physical phenomena, (iii) big data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial, and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the deep learner were discussed in detail. We sincerely thank the numerous reviewers and the authors for their hard work. We also recognize that several papers were submitted that were not a good match to this special section, and these papers were resubmitted to the Journal of Applied Remote Sensing as regular papers.