1.

INTRODUCTION

Dynamic spectrum access (DSA), spectrum sharing, and radio frequency (RF) convergence are concepts that have been receiving ever-increasing attention. As technology has advanced, more and more devices are requiring access to the RF spectrum. The proliferation of various spectrum users and the increasing bandwidth requirements of those spectrum users has driven many to consider the RF spectrum as a commodity. As scarcity for this commodity increases in some frequency bands, the need to more effectively and efficiently use the RF spectrum has impacted both policy and technology.

Recent research in waveform design has focused on facilitating methods for radar and communications spectrum sharing and optimizing performance in multifunction radar. Specifically, radar waveform design has focused on optimizing phase coded waveforms to attain a tactical goal. Furthermore, radar systems are increasingly equipped with waveform management algorithms that employ many dissimilar waveforms to be emitted.

This broad diversity of waveforms in the RF environment poses a challenge for spectrum sensing devices that provide situational awareness to RF spectrum users. Previous works have focused on feature engineering and machine learning for waveform recognition.¹²³⁴ While some works¹²³ focus mainly on feature engineering, other work⁴ places more emphasis on developing a machine learning approach to both classify within-library modulation and detect out-of-library modulations. Similarly, many communities including speech recognition, image classification, and computer vision have begun to place more emphasis on the application of more sophisticated machine learning. To that end, many classification applications have focused on applying neural network algorithms.⁵⁶³

This paper leverages existing feature engineering work¹ to investigate the applicability of sophisticated neural network approaches to the recognition of phase modulated waveforms. We employ the use of features extracted from the ambiguity function (AF) of an intercepted waveform that have shown to aid in accurate classification. This effort will provide a performance comparison between the classifiers in literature¹ and a single hidden layer neural network (SHLNN) classifier, identify fundamental differences in training procedures, and lead to the robust application of a convolutional neural network (CNN) classifier.

The remainder of this paper is organized as follows. Section 2 will briefly discuss the waveform feature extraction process. Section 3 describes the application and results of SHLNN classification methods. Section 4 presents the application and results of CNN classification methods. Finally, Section 5 concludes with a summary of findings and future efforts.

2.

FEATURE EXTRACTION

Given that a radar emits a continuous-time prototype pulse

where ϕ_c(t) is a phase modulation belonging to one of C classes in Table 1. As described,¹ the intercept receiver observes the signal

corrupted by white Gaussian noise v_p(t) with variance and potentially degraded by model mismatch in the receiver. Further, the observed pulses are altered by jitter in complex amplitude A_p, time scaled by α to achieve the designed pulse width, and modulated to carrier frequency, w₀; all of which are unknown to the intercept receiver. Additionally, each pulse from the radar is contained within a window slightly larger than the pulse width to allow for errors in edge detection. The offset within the window, t_p, is assumed to be unknown. Radar pulse observations are then input into the phase modulated waveform classifier in digitized form, y_p[n] = y_p(nT_s), n = 0…N–1, where T_s is the sampling interval assumed to satisfy the Nyquist requirement.

Table 1.

Phase modulation types.1

Because the focus of this paper is on the application and analysis of neural network classifiers to the waveform classification problem, for brevity, we will not overly emphasize the feature extraction process. As such, we compute the two-dimensional discrete Fourier transform of the log-scaled ambiguity function given by

For a more detailed and complete description of the feature extraction process, refer to the ambiguity-based classification algorithm.¹

3.

SINGLE HIDDEN LAYER NEURAL NETWORK CLASSIFICATION

In keeping with the classification strategy in the ambiguity-based classification algorithm,¹ we leverage the classifier training methodology and apply the use of a single hidden layer neural network. Specifically, we train our classifier with the 23 phase modulations displayed in Table 1. For each class, we perform a 1000 iteration Monte Carlo simulation at 10 dB SNR with the pulse widths denoted in Table 1. In each iteration, pseudorandom noise and phase realizations are imparted on the waveform. Additionally, the time offset within the pulse capture window and sampling rate is randomized. This training procedure allows for an intact, yet well-generalized waveform data set to be input into our feature extractor in an effort to safeguard against biases and overfitting in our classification system.

This training procedure is especially well-suited to training Fisher’s linear discriminant (FLD) and support vector machine (SVM) classifiers based on how each forms the separating hyperplane. Because both seek to maximize the margin between two classes, training these classifiers with feature sets that best maintain the structure of the waveform will allow for maximum separation between classes in feature space. For this reason, we train at high SNR. Neural networks behave differently, however. At a fundamental level, neural networks are linear; performing a weighted sum of inputs to form its net activation. The network emits an output that is a nonlinear function of its activation given by

where i and j index the input and hidden layer, respectively, ω and b denote the weights and biases, and σ denotes the nonlinear activation function. Because Equation 4 is a nonlinear function, data input into the SHLNN are mapped non-linearly in feature space altering class separability. To properly utilize this non-linearity it is best to train using a more diverse data set where the structure of each class is widely varied. These widely varied classes allow for the SHLNN to account for the structure of the features in diverse backgrounds. To create a diverse training set, we generate data at multiple SNRs as noted in Section 3.2.

3.1

Single Hidden Layer Neural Network Performance

To test our classification algorithm, we apply a SHLNN using Tensorflow⁷ with different pulse widths than in our training phase, as shown in Table 1. We postulate that a SHLNN should trend similarly to the FLD classifier⁸ and the SVM classifier⁹ given that, at the most fundamental level, a SHLNN behaves linearly.

Figure 1, displays a comparison of the overall accuracy of the FLD, SVM, and SHLNN classifiers for a range of SNRs. Note, that accuracy curves in both plots show a comparison of SHLNN performance under two different training methodologies. This will be explained in further detail in Section 3.2. The left-hand plot in Figure 1 shows that at high SNR, the SHLNN performs similarly to both the FLD and SVM classifiers. As SNR declines below 2 dB, performance degrades quickly. Because good performance is defined as accuracy greater than 90%, it is interesting that the SHLNN performs 2 dB worse than the FLD classifier and 4 dB worse than the SVM classifier.

Figure 1.

(left) Overall classifier accuracy using training method in the ambiguity-based classification algorithm.¹ Note, that while performance is comparable at high SNR, a sharp decline in performance occurs at 0 dB. (right) Overall classifier accuracy using a modified training method. Observe, overall performance increases but reaches as asymptotic limit at about 90% accuracy.

As noted in the ambiguity-based classification algorithm,¹ accuracy alone does not provide a sufficient indication of classifier performance. Instead, the combination of accuracy and the F₁-score given by

where precision is a measure of a classifier’s exactness and sensitivity is a measure of a classifier’s completeness. Figure 2 displays the F₁-score of each class for the FLD, SVM, and SHLNN classifiers. Interestingly, while both the FLD and SVM classifiers outperform the SHLNN, the F₁-scores of the FLD and SHLNN trend similarly for the majority of classes.

Figure 2.

(left) Comparison of classifier F₁-score when trained in a identically. Interestingly, the FLD and SHLNN classifiers show similiar trends in their abilities to classify each modulation. (right) Comparison of classifier F₁-score when the SHLNN incorporates a modified training method. Notice, per-class performance is widely varied.

At this point, it is appropriate to mention the right-hand plots in Figures 1 and 2. Because neural networks, in general, differ greatly from other linear classifiers in the way features are processed, we institute a methodology to improve our training phase. Using this modified training methodology, we are able to drastically improve classifier accuracy as shown in the right-hand plot of Figures 1 and 2. Notice, however that our overall accuracy reaches an asymptotic limit at around 90%. While finding the optimal training set is a difficult problem in neural network research, we do not overly dwell on this for SHLNNs.

4.

CONVOLUTIONAL NEURAL NETWORK CLASSIFICATION

Recently, convolutional neural networks have received increasing notoriety. From applications in facial, hand-writing, and image recognition, CNNs have shown to provide outstanding performance.⁵⁶ Much of this increase in performance is due to its ability to extract meaningful features from large and complex sets of data. Because CNN classifiers have worked well with classifying images, we alter our feature set to maximally utilize the CNN algorithm.

Specifically, we choose to train and test the CNN classifier with features computed directly from the ambiguity function of an intercepted pulse as shown in Figure 3. For each waveform class, one half of the discrete AF is computed and the resulting 101-by-101 image is input into the CNN. Note, that for training the CNN, we follow the both our original¹ and modified training methodology as discussed in Section 3.2.

Figure 3.

Discrete ambiguity function of a Barker 7 waveform.

4.1

Convolutional Neural Network Performance

Figure 4 displays the classification accuracy and and F₁-score of the CNN classifier under both training methodologies. Using the original training methodology,¹ the accuracy of the CNN trends similarly to the FLD, SVM, and SHLNN. Interestingly, good performance (≥ 90%) is achieved at 0 dB but, both the FLD and SVM have higher accuracy. Furthermore, the F₁-score of the CNN classifier appears to closely match the FLD classifier.

Figure 4.

(left) Comparison of overall classifier accuracy. Note, that when all classifiers are trained using the methodology in the ambiguity-based classification algorithm,¹ accuracy trends are similar. Using the modified training approach, performance at low SNR is greatly increased. (right) Comparison of classifier F₁-scores. Note, that while the overall accuracy of the CNN classifier using the modified training is superior to all others tested, the one-vs-all SVM maintains the highest F₁-score.

As shown, performance is drastically increased under the modified training methodology. Training via the modified methodology, overall accuracy of the CNN far exceeds that of any other classifier. Near-perfect accuracy is achieved by the CNN for all SNRs above 0 dB, before gracefully declining at -2 dB. Furthermore, the CNN achieves approximately 60% accuracy at -10 dB; a difference of nearly 32% from the next nearest classifier.

But, as previously stated, overall accuracy is not a complete indicator of the performance of a classifier. The right-hand plot displays the F₁-score of all classifiers. As shown, the CNN classifier using the modified training methodology performs well on all waveform classes. Note, that while the CNN trained using the modified methodology has superior overall accuracy, the SVM classifier has a superior F₁-score on nearly all waveform classes. This is due to the one-vs-all structure of the SVM classifier. Because each class is compared to all remaining classes, false positives are minimized by allowing for the all classification decision, thus, the SVM classifier has superior precision. Conversely, the CNN classifier forces a classification into one of the 23 trained classes resulting in an increased amount of false positives.

Because we seek a classification system that achieves the highest overall performance without respect to any one particular class, assessing our classifiers in terms of an average, or aggregate, F₁-score is appropriate. This measure, like our overall accuracy measure, serves as a composite metric to easily compare all classifiers. Table 2 displays the aggregate F₁-scores for each classifier. For each classifier, the F₁-scores of each waveform class are averaged, providing a measure of the entire classification system. As previously mentioned, the one-vs-all SVM attains the highest aggregate F₁-score followed closely by the CNN using the modified training. The SHLNN using the training methodology in the ambiguity-based classification algorithm¹ provides the worst F₁-score. As previously noted, the relatively low performance of the SHLNN is due to the training methodology. When training using the modified methodology, performance is increased, resulting in an aggregate F₁-score slightly higher than that of the FLD and CNN classifiers under the original training methodology.¹ This modification in our training was especially beneficial to the CNN performance, where the F₁-score experienced a 17% increase.

Table 2.

Aggregate F1-score for each classifier.

Classifier	Aggregate F1-score
Sequential FLD	.7597
One-vs-All SVM	.9818
SHLNN	.6804
SHLNN Modified Training	.7648
CNN	.7491
CNN Modified Training	.9223

5.

CONCLUSION

To enable spectrum sharing, modern RF systems must employ situational awareness algorithms to detect, characterize, and identify other spectrum users. Due to the advances in communications and radar waveform design, incorporating an algorithm to recognize a wide range of modulations provides crucial information to empower spectrum knowledge in RF devices. In this paper, we present a comparison of performance for a single hidden layer neural network and a convolutional neural network applied to the phase modulated waveform recognition problem. We assess our performance against sequential FLD and one-vs-all SVM classifiers and document key findings. Future work will explore incorporating a broader range of waveforms and resolving waveform ambiguities resulting from complex environments.