2.1.

Photoacoustic and Ultrasound Imaging System

The transvaginal coregistered PAT/US imaging system used in the clinical study is shown in Fig. 1. The details of the light delivery subsystem and real-time PAT/US subsystem can be found in our earlier publications.^17–19 Briefly, the PAT/US imaging system consisted of a laser source, a fiber-optic light delivery system, a transvaginal US probe, and a real-time PAT/US data acquisition and processing module. The laser light propagates through a single homemade lens array that consisted of four cylindrical lenses (Edmund Optics Inc., Barrington, New Jersey). The lenses were arranged such that two were in front and the other two behind and orthogonally positioned (rotated 90 deg) to the pair in front. With the incident laser light centered on the lens array, the latter split and focused the beam into four separate and equal spots.¹⁹ Each beam was coupled into a 1 mm core optical fiber with a numerical aperture of 0.48. The distal ends of the illumination fibers were affixed to the compact PAT/US imaging probe. The probe consists of a 128-element transvaginal US array of 6 MHz central frequency and 80% bandwidth (W.L. Gore and Associates, Inc., Newark, Delaware), integrated with a custom-made sheath with a 25 mm outer diameter. The radius of the US probe is 8 mm and elements are evenly spread over 2.56 rad. The sheath encases both the transducer and fibers. In the clinical studies, the probe was covered by a transparent condom, which is typically used in transvaginal US examination of the ovaries. The laser source is a tunable Ti:sapphire laser (LS-2211, Symphotics TII, Camarillo, California) pumped by a Q-switched Nd:YAG laser (LS-2134, Symphotics TII) and delivers 20 ns pulses at 15 Hz repetition rate. The laser fluence on the tissue surface was $\sim 20\mathrm{mJ}/{\mathrm{cm}}^2$ at 750 nm wavelength, and is below the American National Standards Institute safety limit ($25\mathrm{mJ}/{\mathrm{cm}}^2$).²² During in vivo imaging, ovaries were imaged using four preselected wavelengths of 720, 750, 790, and 830 nm with a total tuning time of $\sim 20\mathrm{s}$ for these four wavelengths. For ex vivo studies, the wavelength was fixed at 750 nm. The data acquisition of the imaging system provided up to 15 coregistered image frames per second, with the speed limited by the laser pulse repetition rate. In the coregistered mode, the US and PAT modules ran in a time division multiplexing approach; the PAT module was synchronized with the laser pulses and the US module ran in the periods between the pulses (when no pulses were present).¹⁷ The US transmission focus was fixed to 25 mm in all the experiments.

Fig. 1

The transvaginal coregistered photoacoustic/ultrasound imaging system located at the Radiology Department of the UCONN Health Center for patients’ studies.

2.2.

Human Subjects

This group of human ovaries was obtained from 16 patients (age: 39 to 70 years; average: 55 years) who underwent prophylactic oophorectomy (PO) from March 2014 to January 2015 at the University of Connecticut Health Center (UCHC), and imaged ex vivo using the transvaginal system shown in Fig. 1. These patients were either at risk for ovarian cancer or they had ovarian mass or pelvic mass suggesting malignancy. One patient’s ovaries were excluded from analysis because blue dye was present on both ovaries and caused significant imaging artifacts. The ex vivo imaging was performed by sequentially scanning the ovary using a mechanical stage controlled manually. Five to ten independent image frames were obtained from each ovary depending on the size of the specimen. The ovary was mounted on a supporting frame made of a transparent thin optical fiber and immersed at a depth of $\sim 10\mathrm{mm}$ from the probe surface in a tank filled with calibrated intralipid solution with an absorption coefficient of $0.02{\mathrm{cm}}^{-1}$ and reduced scattering coefficient of $4{\mathrm{cm}}^{-1}$. These optical properties were estimated from measurements of porcine vagina muscle walls. Additionally, two patients who had suspicious ovarian masses were imaged in vivo four to eight days before PO using the hand-held transvaginal probe at the four preselected wavelengths. Scanning was performed at multiple independent locations by an attending US technologist. In addition, ex vivo PAT/US data were obtained from the same ovaries after PO. The ovary side facing the vagina was identified during the surgery by the surgeon using a suture. This way, the comparison of the in vivo and ex vivo images was made feasible. Table 1 provides patient information and diagnosis of the ovaries.

Table 1

Patient information and ovary diagnosis for classification.

This study was approved by the institutional review board of UCHC, and informed consent was obtained from all patients. For the ex vivo studies, the excised ovaries were kept in a 0.9% wt/vol NaCl solution and imaged within a few hours after oophorectomy. After the PAT/US imaging, the ovaries were fixed in 10% formalin solution and returned to the Pathology Department for histological processing. For the histologic evaluation, the ovaries were cut into 5 mm blocks parallel to the imaging plane, dehydrated with graded alcohol, embedded in paraffin, and sectioned into $7\mu \mathrm{m}$ thicknesses using a paraffin microtome. The hematoxylin and eosin (H&E) slides were used to compare with PAT/US imaging findings.

2.3.

Feature Extraction

The conventional delay-and-sum beamforming algorithm was used for both the PAT and US image formation. Each angular PAT and US beam was formed by summation of the delayed radio frequency signals from the nearest 65 channels with respect to the beam direction. The PAT and US beam envelopes were obtained by base-band demodulation and amplitude detection. PAT and US images were constructed by a linear mapping of envelopes after scan conversion.²¹ We paid special attention to selecting visually different images in the image sequence of each ovary to provide more independent data to classifiers. Similar to our earlier study, 17 features were initially extracted from the PAT data for classification of malignant and benign ovarian tissues; these features include two out of three PAT beam spectral features, five PAT beam envelope features, and ten PAT imaging features.²¹

Briefly, spectral features (spectral slopes, mid-band fits, and 0 MHz intercepts) from PAT beams reveal the distribution of the frequency components within the frequency range of the transducer array, and are related to absorber dimensions and chromophore concentrations.^21,23,24 Features from the PAT beam envelopes, such as the summation of envelopes (PAT summation) and the maximum envelope, characterize the light fluence and absorption within the selected suspicious area or region of interest (ROI). The ROI was selected by first estimating the center ($x_0$ and $y_0$) using a Gaussian model fitted to 0 and 90 deg Radon transforms of a PAT image along the $x$- and $y$-axes, respectively.^20,21 The Gaussian model is given as

Linear and nonlinear composite spatial filters developed in Refs. 25 and 26 were constructed to maximize the ratio of output peak to output energy. They acted as a template for malignant ovarian tissues and a template for benign tissues when the construction was applied on training PAT images obtained from malignant ovaries and PAT images from benign ovaries, respectively. The spatial filter constructed from the PAT images of malignant ovaries served as an image template for malignant ovarian tissues (malignant filter). The common spatial frequency features of malignant ovaries, which were not visibly distinguishable, were embedded in this template. The peak output of this spatial filter was the maximum value of the correlation of the input image with this malignant ovary template. A high output from this filter meant a high similarity to malignant ovary and a low output was expected for benign ovaries. (see the Appendix for more details).

Additionally, we included new US spectral features in this study. The US spectrum analysis developed by Lizzi and coworkers^27,28 showed that US pulse-echo spectral parameters were related to the lesion attenuation, sizes, concentrations, and relative acoustic impedances of scatterers. These features have shown promise in cancer detection and classification of the eye,²⁸ prostate,²⁹ and breast.³⁰ We expect these features to be useful additions to improve PAT feature-based classification of malignant and benign ovarian tissues. Three features were extracted from the US beams after calibration with the transducer frequency response for spectral analysis; they are US spectral slope, US 0 MHz intercept, and US mid-band fit. Every US beam segment in the ROI was multiplied by a Hamming window before spectral feature extraction. The averaged value of each spectral feature was used to characterize each imaging frame. In addition, we extracted the ovary area from US images by modeling the ovary as an ellipsoid and measuring its lengths of long and short axes. In the ex vivo data, malignant ovaries had a larger average size than that of benign ones. Furthermore, the mean value of PAT envelop was also extracted in addition to existing maximum PAT envelope and PAT summation features, and found to be valuable in the ex vivo data classification. We also included the menopausal status of patients in our extracted features for classifier input (1 represents postmenopausal, 0 represents premenopausal).

In comparison to our earlier studies, the light delivery setup of our PAT/US system was optimized^18,19 and the data acquisition system was upgraded from 64 channels to 128 channels,¹⁷ thus providing improved signal-to-noise ratio. In addition to the 17 PAT features,²¹ six new features from the US spectrum, mean PAT envelop, ovary area, and patient menopausal status were included to improve the ovarian tissue characterization. Therefore, a total of 23 features were initially extracted.

2.4.

Logistic Model and Support Vector Machine Classifiers

Logistic regression is widely used in medical studies.^31,32 Logistic regression belongs to the class of generalized linear model based on the exponential distribution family. It is a statistical model that can describe the relationship between several predictor variables $X_1,X_2,\dots ,X_k$ and a dichotomous response variable $Y$ (0 or 1). The probability of occurrence of one of the two possible outcomes of $Y$ can be described by the following equation:³³

The SVM optimizes the separation of two populations of data by using a linear or nonlinear kernel. It maps the input data into high-dimensional feature spaces and finds the hyperplane that leads to the best discrimination between the two populations.³⁴ The SVM package in MATLAB^® was used with a radial basis function (RBF) kernel, where the sequential minimal optimization method was used to find a hyperplane threshold that separates the malignant from benign cases. SVM with a polynomial kernel was also investigated; however, the testing results were not as good as those of RBF kernel. Therefore, SVM with RBF kernel was used in our data analysis. The training was terminated when 100% sensitivity and specificity were achieved or when high iterations (1000) had been reached. The trained SVM structure was then applied to the testing samples for validation.

3.1.

Features Selected from Ex Vivo Data

For a total of 23 initial features, there were $2^{23}-1$ choices of feature combinations for final classification because each feature can be included or excluded. The null set (no feature is included for classification) cannot be selected. However, it is computationally intensive to evaluate every one of them for obtaining the optimal subset. Instead, we applied a feature selection method to obtain a suboptimal set. Previously, we simply ranked features according to the decreasing order of $p$ values from Student’s $t$ test of each feature. In this study, we first excluded eight features with $p$ value $>0.05$ and selected 15 features for classification as shown in Table 2. The 15 selected features were further ranked by the forward sequential feature (FSF) selection method.³⁵ The criterion used in the FSF method is the misclassification error (the ratio of the number of misclassified images over the total number of images) of the testing group as shown in Fig. 2. Initially, the selected feature group was empty with all 15 features unselected. We then used each single feature for the classifiers and used the misclassification error of the testing group as the criterion. The feature that generated a minimal misclassification error was placed into the selected group. After that, we chose the second feature from the remaining 14 unselected features whose combination with the selected feature (the first one) gave the minimal misclassification error. This procedure was repeated until all features were selected. In this way, the feature order generated from the FSF could present the estimated optimal subsets for classification performance evaluation. As shown in Fig. 2, when the first nine features in the order generated from the FSF method were used, both classifiers achieved their minimal misclassification error. Therefore, these nine features were selected as optimal features for both classifiers. Note that $p$ values were obtained from images of all patients. $p=0.05$ is a typical value to exclude statistically nonsignificant features. The FSF index was sorted based on the misclassification error for the testing group. Therefore, there was no exact mapping where smaller $p$ values correspond to lower FSF indices.

Table 2

Mean features of ex vivo ovaries and their order after FSF selection. Underscored features are the optimal features used for in vivo classification.

Fig. 2

Misclassification error of both classifiers with respect to different numbers of features in the order from the FSF method. Both classifiers achieved minimal misclassification error when first nine features were used.

3.2.

Features Selected Based on Ex Vivo Data for In Vivo Application

We used the ex vivo studies to determine suitable features for classification of the in vivo data. For the in vivo data, PAT summation, and the max and mean PAT beam envelope were found to be not useful as features. This was because the light fluence on the ovarian tissue surface in the ex vivo and in vivo imaging conditions was different due to the presence of vagina muscle wall in the in vivo study. Additionally, we found the ovary size was not an important feature based on the limited in vivo data. The final five features selected for in vivo classification were PAT spectral slope ($\mathrm{dB}/\mathrm{MHz}$, FSF index #1), US mid-band fit (dB) (FSF index #5), PAT beam envelope homogeneity (FSF index #2), max malignant spatial filter output (FSF index #8), and patient menopausal status (FSF index #4). As shown in Fig. 3, the absolute value of the fitted slope of the malignant sample [Fig. 3(d)] is smaller than that of the benign one [Fig. 3(c)], indicating more high-frequency components in its beams, while the absolute value of US mid-band fit in the malignant case [Fig. 3(f)] is larger than the benign case [Fig. 3(e)].

Fig. 3

(a) Ex vivo coregistered PAT/US image of a premenopausal benign ovarian tissue and (b) ex vivo coregistered PAT/US image of a postmenopausal malignant ovarian tissue. In (a) and (b), the annular fan shapes define the region of beam spectrum feature extraction. (c) The spectrum (blue) and linear fitting (red) of PAT beam of the benign ovarian tissue, (d) the spectrum and linear fitting of PAT beam of the malignant ovarian tissue, (e) the spectrum (blue) and linear fitting (red) of US beam of benign ovary, and (f) the spectrum and linear fitting of US beam of malignant ovarian tissue.

There were in total 201 imaging frames (145 benign and 56 malignant) obtained from the 25 ex vivo ovaries of 15 patients. We then equally and randomly partitioned the data pool into two groups, one group for training and another group for testing. This data processing procedure was repeated 50 times to obtain the average performance of classifiers. The ex vivo data of the two patients imaged in vivo were always kept in the testing group to avoid their influence on in vivo diagnosis. The trained structures of the logistic and SVM classifiers were used for both ex vivo and in vivo data classification.

3.3.

In Vivo Example

An example of coregistered PAT/US image of an enlarged ovarian mass of a 62-year-old postmenopausal patient acquired in vivo is presented in Fig. 4(a). The PAT images reveal a high-level vascularity on the surface of the ovary from multiple images acquired at multiple positions. After oophorectomy, the same ovary was imaged ex vivo and is shown in Fig. 4(b). In the US images, a complex mixed solid and cystic ovarian mass is clearly shown as a homogeneous low echogenic pattern in both in vivo and ex vivo images. Figure 4(c) is the photograph of the excised ovary with lines marking the PAT/US two-dimensional (2-D) scanning positions. The surgical pathology reported $\sim 65\%$ pink-tan, rubbery tissue and 35% cystic structure with early stage carcinoma (T1c) on the ovary surface. This was an accidental finding because of the enlarged ovarian mass seen by the x-ray computed tomography (CT) and transvaginal US. The H&E slides from the marked region showed high-grade serous epithelial carcinoma on the ovary surface. Figure 4(e) highlights the microvessels as pointed to by the arrows. This example demonstrates the clinical potential of coregistered PAT/US in identifying surface abnormal vascularity of a postmenopausal ovary.

Fig. 4

(a) In vivo coregistered PAT/US image of a postmenopausal ovarian mass, (b) ex vivo coregistered PAT/US image of the same ovary after oophorectomy, and (c) photograph of the ovary. The dashed lines illustrate the 2-D PAT/US scan positions and the corresponding H&E cuts. (d) H&E staining of the malignant area. The arrows point to the ovary surface. (e) Microvessels from the rectangular region indicated in (d).

3.4.

Statistics of Ex Vivo and In Vivo Results

Figure 5 shows boxplots and $p$ values of five selected features of benign and malignant ex vivo ovary groups. To provide readers with a statistical performance of these parameters, data from both training and testing sets were used in these plots. The differences were highly statistically significant between benign and malignant groups. These five features were used as predictor variables for the logistic model and SVM to classify benign and malignant ovaries. Figure 6 shows the ROC curve for one set of testing data of the logistic classifier with an AUC of 93.1%.

Fig. 5

Boxplots of ex vivo features: (a) PAT spectral slope; (b) PAT envelope homogeneity; (c) postmenopausal status (1 represents postmenopausal and 0 represents premenopausal); (d) US mid-band fit; and (e) malignant spatial filter output (malignant, $k=0.01$).

Fig. 6

An example of ROC curve of one testing data.

For the training set, the logistic classifier could achieve a sensitivity of 85.6%, specificity of 87%, PPV of 87%, and NPV of 85.8%. The SVM classifier could achieve over 98% sensitivity, specificity, PPV, and NPV as shown in Table 3(a). For the testing set, the logistic and SVM classifiers could achieve a sensitivity of 70.4 and 87.7%, specificity of 95.6 and 97.9%, PPV of 86.6 and 94.4%, and NPV of 89.4 and 95.4%, respectively.

Table 3

Sensitivity, specificity, PPV, and NPV of the logistic and SVM classifiers using five features.

To evaluate the diagnostic potential of PAT/US features in classifying high-grade serous epithelial carcinoma, the most common type of ovarian cancer, we excluded the image frames of patient #4 who had low-grade endometriosis carcinoma from the data analysis. The diagnostic performance of the classifiers is shown in Table 3(b). It is interesting to note that the sensitivity of both classifiers improved for the testing data; the logistic classifier in particular improved significantly. This finding is encouraging and suggests that both classifiers may perform better by identifying aggressive high-grade ovarian cancers, which have significantly different surface vasculature and spectral information than the low-grade carcinomas and benign ovarian tissues. This initial result will need to be validated with a larger patient population.

Figure 7 illustrates the boxplots of the five features that were utilized to classify the two patients whose ovaries were imaged both in vivo and ex vivo. Comparison of the ex vivo and in vivo data shows similarity in terms of the range of the selected features. By utilizing the ex vivo data-trained logistic classifier for the in vivo PAT/US image frames, 12 out of 14 malignant image frames (86% sensitivity) and all 17 benign image frames (100% specificity) were accurately classified. Again, by using the ex vivo data-trained SVM classifier for the in vivo PAT/US imaging frames, 10 out of 14 malignant imaging frames (71% sensitivity) and all 17 benign imaging frames (100% specificity) were correctly classified. Interestingly, by using the ex vivo data-trained logistic classifier, which excluded image frames from the low-grade endometrioid carcinoma, the logistic classifier identified 13 out of 14 malignant imaging frames (94% sensitivity), and one benign image frame was also identified as carcinoma. All 17 benign image frames were identified as benign (94% specificity). The SVM diagnostic performance did not change. Again, no conclusion can be drawn due to the limited in vivo data, and more patient data are needed to validate the result.

Fig. 7

Boxplots for two in vivo patient studies: (a) PAT spectral slope; (b) PAT envelope homogeneity; (c) postmenopausal status; (d) US mid-band fit; and (e) malignant spatial filter output (malignant, $k=0.01$).