2.1.

Holographic Endoscopy

We exploited a novel TM characterization architecture to enable wide-field imaging of quantitative phase- and polarization-resolved (i.e., holographic) properties of biological samples through a flexible MCF bundle (FIGH-06-350G, Fujikura; length of 2 m, 6000 cores, core diameter of $\sim 2.9\mu \mathrm{m}$, core spacing of $4.4\mu \mathrm{m}$, and outer diameter of $350\pm 20\mu \mathrm{m}$). Figure 1 shows a schematic of the system, which is described in detail elsewhere.²⁷ Briefly, data recording for holographic imaging through the fiber is performed in two stages: (1) characterizing the TM of the fiber and (2) recording, via the fiber, amplitude and phase images in two polarizations of the sample.

Fig. 1

Schematic of experimental setup for characterization of an MCF and recording of phase and polarization images of tissue samples and phantoms. Spatial light modulators (SLMs) are used with polarization multiplexers (comprising polarizing beam splitters and waveplates, see Ref. 27) to create polarization-diverse sample illumination and polarization-diverse imaging through an MCF. SLM1 illuminates the phantom or tissue sample with a programmable optical field and generates reference fields for fiber characterization, whereas SLM2 enables imaging of amplitude, phase, and polarization of light exiting the fiber bundle.²⁷

In the fiber characterization stage, a sequence of predetermined fields (an array of spots) is translated across the distal facet of the MCF using a holographic spatial mode and polarization state generator (described in Ref. 27), and the resultant amplitude and phase are imaged in two polarizations. The use of an array of spots exploits the sparse structure of the MCF TM to enable highly parallelized TM characterization measurements resulting in a significant speed-up (12-fold here because 12 spots are used) that is independent of MCF size.

In the image recording stage, the sample (e.g., tissue) is placed at the distal facet of the fiber. At a working distance of 0 mm, imaging is performed through a square subsection of the MCF giving a field of view of $200\times 200\mu \mathrm{m}$, representing $2/3$ of the maximum imaging area. The remaining fiberlets carry a stable phase reference. In this mode, the imaging system is akin to an endomicroscope, but since the working distance can be electronically controlled,²⁷ the field of view could in principle be extended with potential to be a red-flag device.

A broad, Gaussian illumination is then projected onto the sample and the light exiting the other side travels through the fiber and is then recorded, thus performing transmission-mode imaging. The illumination is swept through several different elliptical polarization states. Samples larger than the field of view of the MCF are translated to multiple positions using a stage.

Using this raw data, the TM can be recovered, the image of the sample can be reconstructed and, finally, biologically relevant optical parameters can be extracted. For polarimetric imaging, Jones calculus is applicable here because the light is temporally and spatially coherent due to the laser diode and the single-mode filtering of the cores, respectively, thus depolarization is negligible.²⁸ The total acquisition time for an amplitude, phase, and polarization image set is 8.3 s, and the time taken to fully characterize the fiber is 50.8 min. However, by modifying our setup to use of state-of-the-art TM characterization techniques, we estimate these times could be reduced to $<0.1$ and 22 s, respectively.^27,29

2.2.

Image Data Recovery and Analysis

2.2.3.

Calculating entropy and expectation

To identify biological lesions in otherwise healthy tissue using quantitative phase and polarization images, we apply two image filters: (1) spatial entropy, denoted as $H(..)$ and (2) spatial expectation, denoted as $E(..)$. These can both be determined under a unified framework that estimates the probability distribution of spatial points within a filtering window. The size of the filter window is chosen to be 15 because it is sufficiently narrow so as not to “smooth out” the smallest lesions ($\sim 30\text{pixels}$ here) but offers sufficient data points to reliably fit a two-dimensional (2-D) distribution. This then provides the differential entropy of the inferred distribution as $H(..)$ and the mean as $E(..)$, effectively two spatial filters with a common length scale.

The distribution fitting process is detailed in Fig. 2. We assume here that pixels in this region are independently distributed. Correlation between pixels can be accounted for by computing joint histograms, e.g., using a gray-level co-occurrence matrix,³⁹ but empirically this approach has a negligible impact on resultant entropy maps.

Fig. 2

The distribution fitting algorithm used to compute entropy and expectation metrics from images of amplitude, phase, and polarization. The value of $R$, the size of the filter window, is chosen to be 15 because it is sufficiently narrow so as not to smooth out the smallest lesions ($\sim 30\text{pixels}$ in size in this dataset) but offers sufficient data points to reliably fit a 2-D distribution.

To achieve practical computation speeds, fitting is performed by evaluating predetermined analytic expressions for maximum likelihood parameters of chosen distributions.

The distribution parameters extracted from fitting, mean, and variance are used to determine the “differential entropy” metric, which is derived from the Kullback–Leibler divergence and thus measures how uniformly distributed the values are.⁴⁰ Computed over a spatial region, this gives a measure of spatial heterogeneity indicative of disordered tissue microstructure in tumors. This heterogeneity can then be exploited as an effective contrast mechanism, which has been validated in reflection-mode imaging of tumors using spatial frequency-domain imaging.⁴¹ From the fitted distribution at each pixel, $(x,y)$, we compute the differential entropy, denoted as $H[g(x,y)]$ where $g()$ may be an amplitude or phase value or a polarimetric property.

When applied to quantitative phase images, this entropy metric gives a good indication of scattering-induced heterogeneity. However, when imaging phase, lower power pixels result in higher uncertainty in phase and therefore higher entropy, introducing a power dependence. To isolate the specific influence of the scattering properties of the sample on entropy, we define scattering-induced entropy as

Eq. (2)

$$H_{\mathrm{scat}}(x,y)=\frac{H[{\beta }_{\mathrm{dist}}(x,y)]-H_{\mathrm{pow}}[A^2(x,y)]}{H_{\mathrm{pow}}[A^2(x,y)]},$$

where $H[..]$ is the entropy function, ${\beta }_{\mathrm{dist}}(x,y)$ is the parameter of the distribution fitted to the phase of pixels neighboring $(x,y)$, $H_{\mathrm{pow}}$ is the power-loss-induced entropy determined experimentally using a series of neutral density filters, and $A^2(x,y)$ is the average power level for the neighboring pixels of $(x,y)$.

By computing entropy for a sliding window over the entire raw image, we produce an “entropy filtered” image of the desired quantity. Entropy filtering has been successfully applied in the field of amplitude-only texture analysis for classification of diseased tissues.^42–44 If instead of plotting entropy, we plot the mean of the fitted distributions at each spatial point, this gives images of the spatial expectation, $E$, effectively an averaging filtering on the same length scale as entropy. The use of analytical equations for entropy and expectation combined with efficient GPU parallelization means that entropy and expectation filters take $<0.1\mathrm{s}$ to act on raw data.

2.3.

Preparing Tissue-Mimicking Phantoms

The sensitivity of the holographic endomicroscope to biologically relevant concentrations of scattering and birefringence through phase and polarization imaging was evaluated using tissue-mimicking phantoms. Optically scattering phantoms ($n=2$)⁴⁵ were prepared by adding varying volumes of prewarmed intralipid (20% emulsion, I141, Sigma-Aldrich) to liquid $1.5\mathrm{w}/\mathrm{v}\%$ agar solutions (05039, Fluka) to provide reduced scattering coefficients in the range from 0.125 to $2{\mathrm{cm}}^{-1}$. A 0.75% volume of $0.5{\mathrm{mg}\mathrm{m}\mathrm{l}}^{-1}$ nigrosin (198285, Sigma-Aldrich) was also added to provide an absorption coefficient of $0.005{\mathrm{mm}}^{-1}$. Birefringent phantoms ($n=4$) were prepared using acrylamide/bis-acrylamide, 30% solution (A3699, Sigma-Aldrich) catalyzed using tetramethylethylenediamine (TEMED, T9281, Sigma-Aldrich) and ammonium persulfate (APS, A3678, Sigma-Aldrich). Phantoms were fabricated inside a ventilated fumehood cupboard by mixing 30 ml of acrylamide/bis-acrylamide with $600\mu \mathrm{l}$ of APS and $40\mu \mathrm{l}$ of TEMED under thorough vortexing. For both scattering and birefringent phantoms, the liquid solution was poured into petri dishes (CELLSTAR 628-160, Greiner Bio-One) and then allowed to set at room temperature.

A double integrating sphere (DIS) system⁴⁶ was used to determine the reduced scattering and absorption coefficients of the phantoms. The DIS setup uses two highly reflective spheres (Labsphere), with Lambertian surfaces made from polytetrafluoroethylene, to capture and quantify the amount of transmitted and reflected light. A broadband tungsten halogen lamp (AvaLight-HAL-MINI, Avantes) and two fiber-optic spectrometers (AvaSpec-ULS2048, Avantes) served as the light source and detectors, respectively. The inverse-adding doubling algorithm was used to compute the optical properties of the phantoms from the reflectance, transmittance, and reference measurements.⁴⁷ The scattering-induced entropy is used as the contrast metric when evaluating the holographic endomicroscopy performance.

2.4.

Imaging Biological Samples

To test the ability of the holographic endomicroscope to identify tissue abnormalities in a label-free manner, we used ex vivo samples of mouse esophagus from healthy untreated controls ($n=3$; 7 healthy areas analyzed) and carcinogen-treated animals ($n=6$; 13 distinct lesions analyzed). The esophagus is an ideal model for validating this technology because it is endoscopically accessible, exhibits well-defined multistage carcinogenesis that can be tracked, and has multiple stages of disease represented in space. Samples consisted of a thin layer of esophageal epithelium, providing a representative model of surface scattering and polarimetric effects in the first few superficial cell layers where esophageal tumors are usually detected by endoscopy. All experiments were approved by the University of Cambridge local ethical review committees and conducted according to Home Office project licenses P14FED054 and 70/8866. Experimental mice were doubly transgenic for the inducible Cre allele AhcreERT and the conditional reporter allele EYFP targeted to the Rosa 26 locus (AhcreERTR26flEYFP/wt).⁴⁸ Strains were maintained in a C57Bl6 background. Experiments were carried out with male and female animals. No gender-specific differences were observed.

Esophageal tumors were induced using a chemical carcinogen derived from cigarette smoke (diethylnitrosamine, Sigma-Aldrich). The carcinogen was administered at a concentration of $0.04{\mathrm{mg}\mathrm{l}}^{-1}$ in sweetened drinking water to induce sporadic mutations. Adult animals (over 10-weeks old) were treated 3 days a week for 8 weeks as previously described.⁴⁸ After carcinogen treatment, animals were aged from 6 to 9 months. Controls represent untreated animals. For epithelial tissue preparation, the esophagus was cut open longitudinally, dissected into rectangular pieces of $\sim 5\mathrm{mm}\times 8\mathrm{mm}$, and incubated for 2 to 3 h in 5 mM EDTA at 37°C. The epithelium was then carefully peeled away from underlying connective tissue with fine forceps. The tissue was fixed in 4% paraformaldehyde in phosphate-buffered saline (PBS) for 30 min. For immunofluorescence staining, wholemounts were blocked for 1 h in staining buffer (0.5% bovine serum albumin, 0.25% fish skin gelatin, and 0.5% triton X-100 in PBS) with 10% donkey or goat serum, according to the secondary antibody of choice. Antibodies for keratin 14 (Covance; PRB-155P; 1/1000) and keratin 4 (Vector; VP-C399; 1/1000) were incubated overnight in staining buffer, followed by washing for 2 h with 0.2% Tween-20 in PBS. Samples finally underwent overnight staining with $1\mu \mathrm{g}{\mathrm{ml}}^{-1}$ 4′,6-diamidino-2-phenylindole (DAPI). For imaging, the esophageal epithelium was laid flat on a glass side and a coverslip was placed on top and sealed. These samples are sufficiently thin that they have very high-power transmission and the power-loss-induced entropy is negligible so that $H_{\mathrm{scat}}(x,y)\approx H[{\beta }_{\mathrm{dist}}(x,y)]/k$, where $k=H_{\mathrm{pow}}[A^2(x,y)]$ is a scale factor approximately constant across all samples and spatial positions. Therefore, $H[{\beta }_{\mathrm{dist}}(x,y)]$ is used as the contrast metric for tissue samples.

To identify healthy regions and those containing early abnormalities or definite lesions, reference measurements were taken using bright-field, phase-contrast, and polarization microscopy (BX51-P, Olympus), as well as confocal fluorescence microscopy (Leica TCS SP5 II system: optimal pinhole; speed 400 Hz; line average 3; resolution $1024\times 1024$; image reconstruction using Volocity 6, PerkinElmer). An expert observer (M.A.) examined the confocal fluorescence microscopy images to confirm that no cells were present under the keratin 14 positive epithelial basal layer and to identify abnormal areas and lesions in the issue. The expert observer had previously validated the fluorescence staining approach for identification of abnormal areas and lesions in this mouse model using genetic analyses.⁴⁸ A semiautomated registration process was then applied. The microscopy images were registered using the edges of the tissue samples as corresponding points between the samples; a $2\times 2$ transformation matrix was then found and singular value decomposition was used to obtain scale and rotation. A similar process was then applied to find the transformation between the phase-contrast microscopy image and the stitched endoscopic phase image, using striations in the tissue as corresponding points.

3.1.

Evaluation of Biologically Relevant Variations in Phase and Polarization

Tissue-mimicking phantoms were used to emulate the expected scattering-induced phase variation and polarization effects of diseased tissue. For scattering, we imaged phantoms made of polyacrylamide gel mixed with varying concentrations of intralipid. The raw phase images clearly show a more perturbed wavefront resulting from scattering [Fig. 3(a)]. Spatial scattering-induced phase entropy increases significantly with intralipid scatterer concentration [Fig. 3(b)].

Fig. 3

Biologically relevant variations in phase can be detected. (a) Raw phase images enable the scattering of light due to intralipid to be directly observed, whereas phase entropy images produce quantified scattering maps. (b) Scattering-induced entropy shows a significant linear relationship [$y=(0.20\pm 0.02)x+(0.25\pm 0.02)$, $r^2=0.9816$, $p=0.001$; $n=2$] with increased intralipid scattering concentration. Valid measurements are bounded by the effect of power-loss-induced entropy, and for the last two samples, this is the dominant effect and so they are excluded from the trend. Image scale bar: $200\mu \mathrm{m}$.

For birefringence, we produced transparent phantoms using polyacrylamide gel, which has a strain-dependent birefringence. The birefringent phantoms were mounted on a custom-fabricated mechanical stretcher [Fig. 4(a)] to create a strain-induced birefringence over the endomicroscopic field-of-view. Deformation contours across the birefringent phantoms mounted in the stretcher were assessed by drawing a grid pattern onto the polyacrylamide gel using a marker pen. Qualitative observation of the contours after stretching revealed a macroscopic nonlinearity in the deformation. However, over the small field-of-view of the endomicroscope, the stretch was approximately linear. Due to the constant volume of the material, the change in the thickness upon stretching must be accounted for when plotting retardance, by considering the change in area between the two clamps and dividing the measured retardance by the inferred width. A clear change in retardance can be seen in the raw images and histograms [Fig. 4(b)], and a significant positive trend is observed as strain is increased [Fig. 4(c), $r^2=0.71$]. Other polarization properties did not change significantly.

Fig. 4

Biologically relevant variations in polarization can be detected. (a) Schematic of the phantom stretcher assembly used for mounting and stretching birefringent phantoms. The phantom is held in place by the clamps, which are securely fastened with screws. The stretching screws (one on either side) are turned in uniform increments to stretch the phantom by increasing amounts in the lateral axis. (b) Raw images showing recovered retardance as a function of increasing strain with associated histograms. (c) Retardance increases significantly as a function of strain [$y=(2.91\pm 0.29)x+(0.06\pm 0.03)$, $r^2=0.9725$, $p=0.002$; $n=4$]. Image scale bar: $200\mu \mathrm{m}$.

3.2.

Imaging Early Tumors in Mouse Esophagus

We then sought to test the potential of our quantitative phase- and polarization-resolved imaging to detect early lesions of the esophagus. We imaged samples of mouse esophagus containing abnormal tissue consisting of early preneoplastic tumors ($n=6$, 13 distinct lesions) and control tissues ($n=3$, 7 healthy areas analyzed). Samples consisted of a thin layer of esophageal epithelium, providing a representative model of surface scattering and polarimetric effects in the first few superficial cell layers. As for the phantom study, we used a spatial phase entropy filter, denoted $H(\varphi )$, to quantify scattering. The same filter was applied to the amplitude images to produce amplitude entropy, $H(A)$, as well as to all polarimetric images. Additionally, a spatial averaging filter, denoted $E(p)$, is applied to the raw amplitude, phase, and polarimetric images so that the presented spatial resolution is the same as the entropy filtered images allowing comparison between the metrics.

An example of the full range of imaging parameters available from the holographic endomicroscope for one full sample is illustrated in Fig. 5. Reference images were acquired using bright-field, fluorescence, phase-contrast, and crossed-polarization microscopy [Fig. 5(a)]. The lesion is not visible in the bright-field microscope but is apparent in phase-contrast and cross-polarization microscopy, indicating as expected that phase and polarization can provide information additional to amplitude-only imaging. This is in agreement with our hypothesis that phase and polarization properties of a sample can be measured with high fidelity through an MCF when using a TM recovery approach. The contrast observed in the cross-polarized microscope images [e.g., Fig. 5(a)] suggests polarization effects are present in the lesion but not in healthy tissue. The crossed-polarized microscope cannot resolve whether the underlying cause of this change in polarization is due to birefringence or diattenuation. We therefore consider the biological origins of polarization effects in early tumors—increasing anisotropic tissue microstructure and dense birefringent collagen networks arising during tumor growth—and consider jointly the polarimetric properties that might be affected, ${\theta }_D$, ${\theta }_{\varphi }$, and $\varphi$.⁴⁹ Contrast of these individual parameters may vary between samples due to, for example, sample orientation, so by combining them (a simple sum is used here) we produce a more broadly defined “polarimetric contrast.”

Fig. 5

Example data set showing all imaging parameters available from the holographic endomicroscope for a single mouse sample. Regions of interest are identified based on the legend given on the bottom right. (a) Bright-field, fluorescence, phase-contrast, and crossed-polarization microscopy of a sample containing a single lesion obtained using standard microscopy equipment. (b) Raw images from the holographic endomicroscope, including inferred polarimetric properties. (c) Expectation (i.e., spatial average) of the raw quantities. (d) Entropy of the raw quantities. Scale bar: 1 mm.

In some of the images, rectangular artifacts are observed, arising from stitching together multiple fields of view. Though these contribute noise, the statistical significance of our method is still sufficiently high to distinguish lesion tissue, and they could be reduced in the future by compensating for nonuniform illumination (e.g., speckle averaging illumination⁵⁰).

The raw information obtained from the holographic endomicroscope [Fig. 5(b)] are not informative, but processing to display localized expectation and entropy provides some interesting observations [Figs. 5(c) and 5(d)]. Phase entropy shows high signal in the lesion area but is also sensitive to scattering caused by a tissue fold (verified in the fluorescence image) and the sample edge. The polarization metric of diattenuation axis (${\theta }_D$) also shows a strong signal at the position of the lesion.

Image regions of interest containing lesions or healthy regions from all tissue samples investigated are illustrated in Fig. 6 for the subset of holographic endomicroscope parameters that appeared to be qualitatively more sensitive to the presence of lesions during the full sample analyses. These are compared to fluorescence microscopy data, which is only available after applying a nuclear stain to the samples to highlight areas of abnormal tissue architecture and increased cell density routinely used to identify early tumors.⁴⁸ We provide fluorescence microscopy data here as a gold-standard reference, rather than as an endoscopic comparator. Several label-free quantities measured with our holographic endomicroscope, particularly phase entropy, provide consistently high signal in the lesion areas.

Fig. 6

Holographic endomicroscopy detects a range of early tumor lesions. Extracted regions of interest from full image data sets (see example in Fig. 5) showing sections of healthy tissue and lesions from all nine samples and five endomicroscope parameters. A range of focal and diffuse early tumor lesions can be observed in the reference fluorescence images. Scale bar $400\mu \mathrm{m}$.

Quantitative analysis was then performed using these images. CNR (defined in Sec. 2.4.1) was computed separately for each of the six abnormal tissue samples to evaluate the potential of the holographic endomicroscope parameters to identify lesions within a given tissue sample [Fig. 7(a)]. Sample edges were excluded from the analysis, but folds were included. Phase entropy was found to provide significantly improved CNR compared to amplitude entropy ($p=0.0014$), being comparable to the CNR obtained using the fluorescence nuclear stain ($p=0.150$) but in this case acquired in a label-free manner. Individual polarimetric properties do not show significant increase over amplitude entropy. However, if we sum together properties that share a common biological origin (${\theta }_D$, ${\theta }_{\varphi }$, and $\varphi$ as above⁴⁹) their contrast accumulates. The resultant metric is significantly better than amplitude entropy ($p=0.034$) and again comparable to fluorescence imaging ($p=0.160$). It should be noted that the high variance observed within these CNR calculations is caused by inherent biological variation rather than technical limitation, as CNR was calculated separately for each sample.

Fig. 7

Holographic endomicroscopy enables label-free identification of early lesions in the mouse esophagus. (a) CNR for the different parameters calculated independently for each of the six samples containing lesions. The phase entropy and sum of $\varphi$, ${\theta }_{\varphi }$, and ${\theta }_D$ polarization parameters produce contrast statistically comparable to fluorescence imaging, the gold standard reference modality ($p=0.148$ and $p=0.160$, respectively). Further, these two metrics produce significantly better contrast than amplitude entropy, the best available using conventional endomicroscopes ($p=0.0014$ and $p=0.034$, respectively). Significance determined by paired two-tailed $t$-test, *$p<0.05$ and **$p<0.01$. (b) Receiver operating characteristic curve illustrating performance of different parameters when a binary classifier with varying threshold is applied to discriminate between healthy and lesion tissues. Phase entropy significantly outperforms conventional amplitude imaging, but the large intersample variance of the combined polarization metric results in reduced performance with a simple binary threshold classifier.

Finally, we applied a binary classifier to produce an ROC curve, showing the sensitivity and specificity across the sample set [Fig. 7(b)]. While the sample size is limited, encouragingly the phase entropy metric performs significantly better than amplitude entropy (similar to the existing standard-of-care) and lies close to the fluorescence reference, indicating superior performance for this label-free metric compared to conventional amplitude-only imaging or fluorescence-based approaches that require the application of dyes. The combined polarization metric performs worse with binary classification than amplitude entropy, most likely due to the high intersample variance. The statistically significant CNR suggests there may, nonetheless, be valuable information contained in the polarization data, but a more advanced classifier trained on a larger sample size would likely be required to evaluate the full potential of these techniques.