1.

Introduction

Laparoscopy is a minimally invasive surgical technique that has become the standard of care for many procedures, such as prostatectomies and appendectomies.¹ Compared with traditional open surgery, laparoscopy results in faster recovery times, reduced scarring, and lower medical costs.² However, laparoscopy presents challenges in visualization due to limited field of view (FOV) and poor contrast, which can make it difficult to accurately identify critical anatomical structures, assess tissue perfusion, and determine cancerous tissue margins.³ These limitations can lead to subjective decision-making reliant on the surgeon’s experience,⁴ leading to wide variability in clinical outcomes.⁵ This has spurred the development of optical technologies for surgical guidance, including fluorescence contrast^6,7 and endogenous contrast.^8–10 However, fluorescence imaging using an exogenous dye requires pre- or peri-operative administration of an optical contrast agent, which increases intraoperative workflow complexity. In addition, many of these techniques are qualitative, leading to user subjectivity, which limits their practical clinical value and generalization to downstream tasks employing artificial intelligence. Recent research studies have demonstrated that tissue optical properties can be used to identify tissue types with high sensitivity and specificity,^10,11 underscoring the potential impact of laparoscopic techniques enabling quantitative mapping of optical properties.

Spatial frequency domain imaging (SFDI) is a low-cost, non-contact, wide-field, rapid, technique for mapping tissue optical properties.^12,13 Conventional SFDI involves projecting a 2D sinusoidal pattern of light onto a medium and imaging the back reflectance using a standard camera. A turbid medium scatters the projected light, preferentially attenuating high spatial frequencies. In addition, absorption by tissue chromophores primarily attenuates the low spatial frequencies of the illumination pattern. The acquired diffuse reflectance images at high and low spatial frequencies are used to decouple and estimate absorption and reduced scattering coefficients via lookup tables (LUTs) generated from Monte Carlo light transport simulations. By acquiring multiple measurements with unique illumination wavelengths and spatial frequency measurements, the concentrations of dominant tissue chromophores, such as oxy- and deoxy-hemoglobin, can be mapped. Clinically, SFDI has been successfully translated into surgical interventions, such as measuring ischemic onset in skin flaps,^14,15 kidney,¹⁶ bowel,¹⁴ and liver.¹⁴ In addition, SFDI has shown promise in oncologic margin detection for glioma,¹⁷ esophageal cancer,¹⁸ and colon cancer.¹⁹

Incorporating conventional SFDI into minimally invasive surgery has been challenged by slow measurement acquisition rates, complex tissue geometries, and compatibility of the projection optics with rigid laparoscopes. Conventional SFDI requires at least six images per wavelength (three distinct spatial phases at two spatial frequencies) to estimate a single optical property map, limiting the acquisition rate to $\sim 5\mathrm{Hz}$. To accelerate SFDI toward video-rate imaging, a technique called single snapshot optical properties (SSOP)²⁰ was introduced, which reconstructs an optical property map from a single structured light image. SSOP performs demodulation in the Fourier domain, separating the DC (planar) and AC (spatially modulated) components from a single-phase AC image. To improve the accuracy of optical property estimation in 3D samples, Gioux et al.²¹ used phase profilometry to estimate the 3D sample profile, enabling correction for irradiance variations associated with height and surface angle. There have been numerous approaches to condensing the bulky SFDI projector into an endoscopic form factor. Angelo et al.²⁰ introduced real-time endoscopic SSOP by coupling light through a fixed grating into one of the laparoscope ports. Although this endoscopic SSOP approach produces video-rate optical property measurements, it suffers from a number of weaknesses: (1) undesirable artifacts and reduced accuracy due to Fourier filtering of a single image; (2) its illumination strategy occupies one of the laparoscope ports, precluding stereo imaging; and (3) incompatibility with the modern sensor-at-the-tip design of stereo laparoscopes. More recently, Crowley and Gordon²² developed an “ultra-miniature” SFDI system that produces fringe patterns interferometrically through a fiber array, making it sufficiently small for a standard colonoscope. However, the fiber array produces non-ideal illumination patterns with poor modulation depth. These patterns are susceptible to noise and cross-core coupling, leading to the degradation of optical property estimation. There is also no way of estimating depth and surface shape in this approach, which is crucial for the accurate estimation of optical properties in real-world applications.

To overcome some of the limitations of previous endoscopic SFDI efforts, we build upon our previous work on speckle-illumination SFDI (si-SFDI).²³ We introduce a two-camera laparoscopic si-SFDI system coupled with an active stereo profile estimation scheme to generate surface profile-corrected optical property maps. In this study, we estimate optical properties using only two images: one speckle image (AC) and one speckle-reduced image (DC). This approach improves upon previous si-SFDI work, which required $\ge 10$ images to approximate the DC image and accurately sample the low spatial frequency modulation transfer function (MTF) response. We propose an alternative technique that approximates the DC image using a laser speckle reducer (LSR). To replace the bulky projector system, which previously required a free space laser expanded to illuminate a diffuser, we introduce multimode fiber-based speckle illumination, which allows compact speckle pattern generation over a wide FOV and large depth of field (DOF). In addition to optical property measurements, laser speckle illumination provides synthetic texture for stereo matching, enabling high-fidelity 3D profile estimations of low-contrast samples.²⁴ These 3D reconstructions are then used for downstream SFDI height- and angle-dependent intensity corrections, enabling accurate optical property measurements of non-flat surfaces. We employ two fiber illumination conditions: a high-coherence red laser (HC-R) to generate high-contrast speckle patterns for optical property estimation, and a low-coherence RGB laser (LC-RGB) to produce low-contrast speckle patterns for stereo 3D reconstruction. The LC-RGB source is additionally speckle-reduced to allow speckle-free conventional white light imaging. Overall, this study validates stereo laparoscopic si-SFDI’s ability to accurately and precisely measure profile-corrected optical property maps for tissue-mimicking phantoms arranged with complex geometries and quantitative physiological changes of an in vivo sample, paving the way for optical imaging-guided minimally invasive surgery using quantitative endogenous contrast.

2.

Principles of si-SFDI

Previous work demonstrated that optical properties can be mapped using structured light from laser speckle illumination instead of sinusoidal illumination,²³ whereas conventional SFDI uses sinusoidal illumination to sample discrete spatial frequencies of a tissue’s MTF, si-SFDI uses speckle illumination to sample a wide range of spatial frequency bands of the MTF. si-SFDI relies on the consistency of the power spectral density (PSD), which requires that the integral of the PSD should remain constant for any randomization of the speckle pattern. For an unknown turbid sample, changes in the measured PSD relative to a calibration phantom reflect differences in the sample’s optical properties. For samples with the same absorption but reduced scattering coefficients, incident photons will diffuse deeper into the media and emerge farther from their entry point. This wider spread of photons blurs the speckle pattern, attenuating higher spatial frequencies and narrowing the PSD. Conversely, in samples with constant scattering but higher absorption, photons are absorbed before they diffuse far from their entry point. This reduces the speckle blur, attenuating lower spatial frequencies and broadening the PSD.²⁵

Compared with laser speckle contrast imaging, which measures the blur of subjective speckle patterns, si-SFDI relies on measuring the blur of the objective speckle patterns. Subjective speckle patterns are produced from coherent light interacting with the object and interfering at the detection plane.²⁶ Objective speckle, on the other hand, results from the interference of the coherent source within a passive optical element that imparts spatially varying random phase shifts²⁷ and is independent of the sample properties and detector position or optics. The previous si-SFDI study produced objective speckle patterns by sending coherent illumination through a diffuser. In this study, we produce objective speckle patterns through modal interference of coherent light in a multimode optical fiber.^28,29 The mean objective speckle diameter $d_o$ at the object plane is inversely proportional to the fiber core radius, $a$

To generate optical property maps using si-SFDI, we calculate a PSD for each image pixel based on the radially averaged autocorrelation function (ACF) of a sliding window. The size of this window is selected to match the desired sampling spatial frequency. The ACF function $a_v$ is estimated using the Wiener–Khinchin theorem³² to reduce the computational cost

Following the approach outlined by Cuccia et al.,¹² the AC and DC diffuse reflectance ($R_d$) of a sample are calculated using $R_d=\frac{M}{M_{\mathrm{ref}}}·R_{d,\mathrm{ref},\mathrm{pred}}$, where $M_{\mathrm{ref}}$ is the response parameter of the reference phantom and $R_{d,\mathrm{ref},\mathrm{pred}}$ represents the reflectance value predicted by White Monte Carlo models.³³ We use a pre-computed LUT to determine $({\mu }_a,{\mu }_s^{\prime })$ from $(R_{d,\mathrm{DC}},R_{d,\mathrm{AC}})$ using linear interpolation. The LUT was generated by performing Monte Carlo simulations of $R_{d,AC}$ at $0.1{\mathrm{mm}}^{-1}$ increments from 0.1 to $0.5{\mathrm{mm}}^{-1}$ and summing them together.³⁴

3.

Stereo Measurements for Profile-Corrected Optical Property Mapping

To accurately measure $R_d$, it is important to correct for intensity variation due to sample height and curvature. We perform profile-correction for non-planar sample surfaces and changes in height, by modeling light reflecting from a diffuse surface with a Lambertian model and approximating the source vector as being coincident with the camera vector. In this model, the illumination intensity varies as $\cos (\theta )/z^2$,³⁵ where $\theta$ is the angle between the sample surface normal and the optical axis of the laparoscope. For example, as an object’s distance from the laparoscope tip increases, the image will appear darker, and as $\theta$ increases, it also becomes darker.

4.

si-SFDI Laparoscope System

Figure 1 shows the experimental setup. A rigid dual camera stereo laparoscope (Intuitive Surgical Scholly 311464−05 Da Vinci Robotic 12MM Endoscope 0 deg Tip) with relay optics (Intuitive Surgical 370538−04 Da Vinci Wide Angle Camera Adapter) imaged two views of the object to two 8-bit RGB CMOS color cameras (GS3-U3-41C6C-C 1″ FLIR Grasshopper 3 High Performance USB 3.0 Color Camera) with a stereo baseline of 5 mm. The acquired images were cropped to the sensors’ central $784\times 960\text{pixels}$ to remove obscured pixels. This region of interest (ROI) has an angular FOV (AFOV) of $33\times 39\mathrm{deg}$, resulting in a $4.6\mathrm{cm}\times 5.6\mathrm{cm}$ FOV and $58\mu \mathrm{m}$ object pixel size at a distance of 7.8 cm.

Fig. 1

Laser speckle stereo laparoscope imaging schematics. (a) A high-coherence red laser (HC-R) and low-coherence red/green/blue lasers (LC-RGB) source are coupled into multimode fibers (MMF1/2) with coupling lenses (CLs) to illuminate the full laparoscopic FOV. The HC-R light is cross-polarized (CP1/2) with the imaging channel to reduce specular reflection and isolate diffuse reflectance for si-SFDI optical property estimation. An LSR is toggled to alternate between high-coherence speckle illumination (AC Speckle Image) and uniform illumination (DC Uniform Image). An HWP connected to a rotation mount (RM) was placed in the HC-R beam path to control the polarization angle and tune the power output. The LC-RGB illumination provides synthetic texture that improves stereo matching and the resulting profile estimation. In addition, LC-RGB can be speckle-reduced using an LSR to provide conventional white light images. (b) A photograph of the optical system.

To generate high-contrast red laser objective speckles for measuring optical properties, we used a 639-nm diode-pumped solid-state single longitudinal mode laser (HC-R; CNI Laser, Changchun, CN, MSL-FN-639) with a coherence length greater than 40 m. This light was coupled into a 0.50 NA, $600\text{-}\mu \mathrm{m}$ core step-index multimode fiber optic cable (MMF1; Thorlabs, M53L01, Newton, New Jersey, United States) using an aspheric condenser lens (CL; Thorlabs, ACL25416U). We can select a fiber core diameter that produces the ideal speckle size on the object plane. Importantly, $d_o$ increases with distance [Eq. (1)]. Consequently, when a sample is measured at a greater distance than the reference phantom, the larger projected $d_o$ results in a distance-dependent shift toward lower spatial frequencies in the PSD. However, if the illumination and detection AFOVs are equal, the “apparent” speckle grain size on the image sensor remains constant regardless of distance, and therefore, the pattern’s carrier frequencies do not change on the sensor. We can then account for distance-dependent spatial frequency shifts by first calibrating over several known distances and measuring the sample height, as described in Sec. 3.2.

To ensure that the AFOVs of the illumination and detection were equal and to expand the objective speckle size, the light was spread by an endoscopic diverging lens (DL; Olympus, WTLGLA-187W, Tokyo, Japan), which was mounted on the fiber tip using a custom 3D-printed mount. We selected an illumination fiber diameter that produced an average objective speckle grain size of 5 pixels across all distances. This size satisfies the Nyquist criterion for the red channel of our RGB Bayer pattern (GBRG) image sensor.

At a 7.8-cm calibration distance, we used an $85\times 85\text{-}\text{pixel}$ sliding window to generate the PSD, yielding a spatial frequency resolution of $0.2{\mathrm{mm}}^{-1}$. As mentioned in Sec. 3.2, the sliding window size should change as a function of distance to ensure consistent spatial frequency resolution of the PSD. We defined the AC response using the first two non-zero spatial frequencies (centered at 0.2 and $0.4{\mathrm{mm}}^{-1}$). This approach incorporates the frequency commonly used in SFDI for biological tissue ($0.2{\mathrm{mm}}^{-1}$) while improving stability and reducing noise influence by including the higher frequency component. We used a sliding window step size of 5 to balance the computational speed and spatial resolution of the optical property maps. Smaller steps may improve resolution for highly heterogeneous samples but increase processing time.

To tune the output power of the laser, a half-wave plate (HWP) was mounted to a direct drive rotation mount (RM) that controls the beam’s polarization angle. At a distance of 7.8 cm, we measured a power density of $2.05\mathrm{mW}/{\mathrm{cm}}^2$ for the 639-nm illumination using a standard photodiode power sensor (Thorlabs, S121C/PM100D). This is well below the OSHA safety limit for class I lasers, which specifies a maximum permissible exposure (MPE) for skin at 639 nm of $200\mathrm{mW}/{\mathrm{cm}}^2$ for $>10\text{-}\mathrm{s}$ exposure. In addition, to isolate scattered light from the sample and remove specular reflections, we mounted crossed polarizers at the speckle illumination fiber output (CP1) and the laparoscope imaging input (CP2) using two $Ø1/2^{″}$ Linear Polarizers (Thorlabs, LPVISE050-A). The illumination fiber output was mounted at a 15-deg angle with respect to the camera axis and spaced 8 mm from the outer edge of the laparoscope to balance proximity to the optical axis with full field-of-view illumination. The 8-mm spacing accommodated the size of the linear polarizer used for cross-polarization.

To produce a speckle-reduced, planar illumination (DC) image, a custom-made LSR was attached to MMF1. The LSR utilizes a vibration motor (UXCELL DC 6V 3000RPM Arch Vibrating Wheel) operating at $\sim 2000$ RPM. This process causes multiple random speckle patterns within the exposure time of the image capture, approximating a planar illumination (DC) image. As mentioned in Sec. 3.1, to improve stereo matching through synthetic texture projection, low-contrast RGB laser speckle was generated by coupling a low-coherence RGB laser (LC-RGB) module (Micro RGB Laser Module, Opt Lasers) into a separate 0.50 NA, $400\text{-}\mu \mathrm{m}$ core, step-index multimode fiber optic light guide (MMF2). Similar to MMF1, a diverging lens was used to illuminate the full camera FOV and expand the objective speckle size. The LC-RGB module consists of red ($638\mathrm{nm}+5/-3\mathrm{nm}$), green ($525\mathrm{nm}+10/-5\mathrm{nm}$), and blue ($450\mathrm{nm}\pm 10\mathrm{nm}$) laser diodes. At a distance of 7.8 cm, the power density for each RGB laser diode (red: $0.77\mathrm{mW}/{\mathrm{cm}}^2$, green: $0.62\mathrm{mW}/{\mathrm{cm}}^2$, blue: $0.95\mathrm{mW}/{\mathrm{cm}}^2$) was well below the MPE for skin specified by OSHA for class I lasers at their respective wavelengths. Furthermore, the low-contrast RGB speckle was speckle-reduced using the LSR to approximate incoherent white light illumination.

To measure profile-corrected optical properties for each sample, we acquired one speckle (AC) image and one speckle-reduced (DC) image, each with a 30-ms exposure time and two active stereo RGB images, each at a 90-ms exposure time. The exposure times were chosen based on histogram skewness to optimize the signal within the camera’s dynamic range. The overall data acquisition time is $\sim 340\mathrm{ms}$, which includes the 100-ms delay to allow the LSR to ramp to its steady state. For si-SFDI optical property estimation, we used the red channel of AC and DC images, whereas the RGB images were used for active stereo surface profile estimation. The estimated subjective speckle grain size from Eq. (2) is 0.92 pixels.³¹ This is consistent with experimental images, where subjective speckle grains are $\sim 1\text{pixel}$ in diameter. To reduce the contribution of the $\approx 1\text{pixel}$ subjective speckle while maintaining the optical property information encoded in the blurred $\approx 5\text{pixel}$ objective speckle, we applied a Gaussian filter with a kernel size of 3 pixels to the raw image. All measurements were taken in a dark room to reduce the influence of ambient light and simulate a low light, minimally invasive surgical environment.

5.

Results

5.1.

Optical Properties Measurements of Flat Tissue-Like Phantoms

We first validated our laparoscopic si-SFDI system using 13 homogeneous tissue-mimicking phantoms ($6\mathrm{cm}\times 6\mathrm{cm}\times 2\mathrm{cm}$) with unique combinations of optical properties. The phantoms were fabricated by mixing polydimethylsiloxane (PDMS) with titanium dioxide (${\mathrm{TiO}}_2$) and India Ink for scattering and absorption agents, respectively.⁴² We compared optical properties for each phantom estimated by laparoscopic si-SFDI at 639 nm and conventional three-phase, 0 and $0.2{\mathrm{mm}}^{-1}$ spatial frequency SFDI at 659 nm (Modulim Reflect RS). The calibration phantom used in all of the experiments was a large ($21.5\mathrm{cm}\times 21.5\mathrm{cm}\times 2.5\mathrm{cm}$), flat tissue-mimicking phantom with known optical properties (${\mu }_a=0.024{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=0.99{\mathrm{mm}}^{-1}$; 639 nm). To account for the wavelength mismatch between si-SFDI (639 nm) and SFDI (659 nm), we corrected the measured sample optical properties using the predicted optical properties of India ink and ${\mathrm{TiO}}_2$ at their respective wavelengths. To correct absorption, we used the ratio of India ink’s extinction coefficients ($ϵ$) at the corresponding wavelengths of 639 and 659 nm,⁴³ which resulted in ${\mu }_{a,\mathrm{si}-\mathrm{SFDI}}=1.0361{\mu }_{a,\mathrm{SFDI}}$. To correct for reduced scattering, we used a Rayleigh scattering relationship (${\mu }_s^{\prime }\propto {\lambda }^{-4}$), which resulted in ${\mu }_{s,\mathrm{si}-\mathrm{SFDI}}^{\prime }=1.131{\mu }_{s,\mathrm{SFDI}}^{\prime }$. We compared the mean and standard deviation for central ROIs between a $100\times 100\text{-}\text{pixel}$ laparoscopic si-SFDI ROI and a ground truth $150\times 150\text{-}\text{pixel}$ SFDI ROI. Due to the different imaging scales of si-SFDI and SFDI, we adjusted the sizes of the ROIs to ensure that we sampled comparable proportions of the phantom’s area across both modalities. Figure 2 shows scatter plots of the optical properties estimated from tissue-mimicking phantoms using laparoscopic si-SFDI with one AC image and one DC image. We observed a mean absolute error of 6.4% for absorption and 5.8% for reduced scattering. We found that laparoscopic si-SFDI predicted optical properties fit a $y=x$ ground truth line with a coefficient of determination $R^2$ value of 0.986 for reduced scattering and 0.993 for absorption, as shown in Figs. 2(b) and 2(c), respectively.

Fig. 2

Comparison of laparoscopic si-SFDI (blue circles) and ground truth SFDI (orange triangles) measured optical properties of 13 homogenous tissue-mimicking phantoms. (a) si-SFDI measurement versus SFDI ground truth. si-SFDI measurements ($y$-axis) of (b) reduced scattering and (c) absorption versus SFDI ground truth ($x$-axis). The coefficient of determination $R^2$ is calculated based on the $y=x$ line representing ground truth. Markers indicate mean values and error bars indicate standard deviation.

5.2.

Height and Angle Estimation Using Active Stereo

We evaluated our 3D profile reconstruction method using a flat homogeneous phantom (${\mu }_a=0.013{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=1.26{\mathrm{mm}}^{-1}$; 639 nm) that was vertically translated or rotated in both the $XZ$ (front tilt) and $YZ$ (side tilt) planes. In the $XZ$ plane, positive angles indicated the phantom tilting toward the LC-RGB illumination source, whereas negative angles indicated tilting away, resulting in asymmetric illumination. In the $YZ$ plane, positive and negative angles represent tilts perpendicular to the illumination source, resulting in symmetric illumination. The 3D profile was reconstructed using LC-RGB speckle active stereo images, as shown in Fig. 3(a). We vertically translated the phantom over a 4-cm range, from 5.8 to 9.8 cm, in 1-cm increments. As shown in Fig. 3(b), our method reconstructed the distance to within 0.7 mm of its theoretical value. For side tilt angles ranging from $-40$ to $+40\mathrm{deg}$, the phantom was reconstructed to within 2 deg or less of its theoretical angle, as shown in Fig. 3(c). However, for front tilt angles greater than $+30\mathrm{deg}$ or less than $-30\mathrm{deg}$, we observed a higher deviation of 3 to 6 deg. To obtain the measured height and angle, we sampled a central $100\times 100\text{-}\text{pixel}$ ROI.

Fig. 3

Sample geometry estimation setup and results: (a) Close-up schematic of the speckle illumination conditions. The high-coherence red laser (HC-R) produces high-contrast red laser speckle illumination for optical property estimation and is delivered through the multimode fiber (MMF1). MMF1 illumination is cross-polarized (CP1/2) with the imaging channel to isolate diffuse reflectance. The low-coherence RGB lasers (LC-RGB) produce low-contrast RGB laser speckle illumination for enhanced stereo matching and white light imaging and are delivered through MMF2. Light through MMF1 and MMF2 is positioned in the $XZ$ plane at a 15-deg angle relative to the imaging optical axis. Diverging lenses (DLs) are used to expand the illumination, and LSRs are used to provide speckle-free illumination when necessary. In panel (a), the left half of the sample plane shows HC-R speckles used for si-SFDI optical properties, and the right half shows LC-RGB speckle projection for active stereo profilometry. HC-R speckles were contrast-enhanced for visualization. (b) Active stereo-predicted surface distances (blue circles) compared with the expected ground truth surface distances (line). (c) Active stereo predicted surface angles for front-tilt (blue circles) and side-tilt (red triangles) compared with the expected theoretical angles (line).

5.3.

Profile-Correction of Flat Tissue-Mimicking Phantom

Without geometrical correction, optical property estimation accuracy decreases when the sample’s height or angle differs from the calibration phantom (Fig. 4). A calibration phantom was measured at distances from 5.8 to 9.8 cm in 1-cm increments, all at 0 deg of front and side tilt. Height correction was applied using the virtual calibration phantom model described in Sec. 3.2. Figure 4(a) shows the results for uncorrected and height-corrected optical property measurements for a phantom (${\mu }_a=0.013{\mathrm{mm}}^{-1}$; ${\mu }_s^{\prime }=1.26{\mathrm{mm}}^{-1}$; 639 nm) with different optical properties than the calibration phantom (${\mu }_a=0.024{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=0.99{\mathrm{mm}}^{-1}$; 639 nm). For uncorrected measurements, as the sample moves closer to the camera relative to the reference phantom’s position, absorption is increasingly underestimated, and reduced scattering is increasingly overestimated. The opposite occurs when the sample moves further away from the camera relative to the reference phantom’s position. Height-based correction significantly reduced the absorption coefficient’s mean error per cm from 78.5% to 2.8% over the entire 4-cm range. The reduced scattering coefficient mean error per cm decreased from 21.6% to 1.3% over the same range. For absorption coefficients, the mean of the coefficients of variation decreased from 20.9% to 6.5%, while for reduced scattering coefficients, it decreased from 4.4% to 3.5%.

Fig. 4

Profile-correction of optical properties for a vertically translated and angled tissue-like phantom: Mean and standard deviation of uncorrected (triangles) and profile-corrected (circles) absorption coefficients (top row) and reduced scattering coefficients (bottom row) of a homogenous phantom at 639 nm. The solid line depicts the expected ground truth optical property values measured using conventional SFDI. Panel (a) demonstrates height correction for vertical translations, whereas panels (b) and (c) show angular corrections for front tilt and side tilt, respectively.

To test angle-dependent intensity correction, we placed our flat sample on 3D-printed wedges, varying both front and side tilt from $-40$ to $+40\mathrm{deg}$ in 10-deg increments. We isolated the contribution of angular correction by comparing only height-corrected to height and angle-corrected optical properties. For the remainder of this paper, we will refer to the latter as “profile-corrected” optical properties. In Fig. 4(b), for front tilt from $-40$ to $+40\mathrm{deg}$, the correction reduced absorption coefficient mean percent errors from 60.5% to 10.4% for negative angles and 40.7% to 4.7% for positive angles. Angle correction also decreased reduced scattering coefficient mean percent errors from 14.3% to 5.5% for negative angles and from 7.8% to 1.5% for positive angles. The positive and negative angle error for the front tilt condition is asymmetric. This asymmetry arises because the HC-R illumination is oriented at a $+15\text{-}\mathrm{deg}$ front tilt from the camera’s optical axis [Fig. 3(a)], whereas our simplified angular correction model assumes coincident camera and source vectors. In Fig. 4(c), for side tilts ranging from $-40$ to $+40\mathrm{deg}$, angle correction reduced the mean percent error of the absorption coefficient from 47.4% to 9.3%, and of the reduced scattering coefficient from 11.1% to 3.6%. The errors for positive and negative side tilt angles were combined because side tilt rotation is perpendicular to the HC-R illumination source vector, resulting in symmetrical errors. Furthermore, even though the mean accuracy improved, the results show an increase in the mean coefficients of variation when comparing the profile-corrected to the height-corrected results for all experimental groups: front tilt, side tilt, positive angles, and negative angles. For absorption, the increase goes from 4.1% to 10.6%, and similarly, for reduced scattering, from 1.5% to 2.5% for only height-corrected to profile-corrected, respectively. The increase in the mean of the coefficients of variation is directly related to the noise of the estimated surface normals, which propagates into the profile-corrected optical property maps through the angular correction factor in the denominator of $R_{d,\mathrm{sam},\text{angle}\text{-}\text{height}\text{-}\mathrm{corr}}$ in Eq. (6).

5.4.

Profile-Correction for Multiple Flat Phantoms at Different Heights

To evaluate profile-corrected laparoscopic si-SFDI in a more complex scenario, we imaged three flat phantoms at different heights within the same FOV. The background phantom was positioned at the reference calibration height (distance = 7.8 cm), whereas the two other cubic phantoms were placed at smaller distances. Figure 5(a) shows an oblique photograph of the setup with optical property annotations. As all samples were flat, their surface normals were parallel to the optical axis, resulting in angular correction factors close to unity. Figure 5(b) shows the speckle and DC image captured by the laparoscope. The speckle image is contrast-enhanced for visualization. Figure 5(c) shows the fractional error between laparoscopic si-SFDI and ground-truth SFDI optical properties. It shows absorption (top row) and reduced scattering (bottom row), both with (right column) and without (left column) profile-correction. To compute the fractional error, we first selected a square ROI of the si-SFDI optical property map that contains the three phantoms [approximately the FOV depicted in Fig. 5(b)]. We selected the same ROI on the SFDI optical property map and upsampled this region with linear interpolation to match the pixel dimensions of the si-SFDI ROI. We then compared fractional errors between these two matched ROIs. These errors are overlaid on the 3D sample surface as estimated by active stereo. Referring to Fig. 5(c), for the darkest color phantom [Fig. 5(a); ${\mu }_a=0.036{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=0.78{\mathrm{mm}}^{-1}$; 639 nm; $2\mathrm{cm}\times 2\mathrm{cm}\times 1.3\mathrm{cm}$], profile-correction reduced the fractional error from a 16.1% underestimation of absorption to an overestimation of 10.3%, with an overall 5.8% improvement. The reduced scattering error is reduced from an overestimation of 18.7% to 11.2%. For the lightest color phantom [Fig. 5(a); ${\mu }_a=0.013{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=1.25{\mathrm{mm}}^{-1}$; 639 nm; $2\mathrm{cm}\times 2\mathrm{cm}\times 1.5\mathrm{cm}$], we observed a dramatic improvement in absorption from an underestimation of 58.8% to 4.6%, whereas the reduced scattering improved from a 7.4% overestimation to a 3.7% underestimation. As a control, the background phantom [Fig. 5(a); ${\mu }_a=0.023{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=0.99{\mathrm{mm}}^{-1}$; 639 nm] shows no significant change in fractional error between the uncorrected and profile-corrected conditions. This is expected because the background phantom is located at the reference calibration height and, therefore, should not undergo a height correction. Finally, we should also note that the reduced scattering maps in Fig. 5(c) show a significant amount of noise. When comparing the percent coefficient of variation over an ROI for SFDI and si-SFDI for each phantom, the si-SFDI percent coefficient of variation is consistently 30% higher than that of SFDI.

Fig. 5

Multi-height flat phantoms with profile-correction: (a) Photograph of the three tissue-mimicking phantoms annotated with their optical properties determined by conventional SFDI. (b) Multi-phantom laser speckle (AC; top left) and uniform illumination (DC; bottom right) images used for si-SFDI optical property estimation. The AC data are contrast-enhanced for visualization. (c) Overlay of the optical property fractional error map onto the active stereo estimated 3D point cloud of uncorrected (left column) and profile-corrected (right column). Fractional error was computed between laparoscopic si-SFDI and ground truth optical properties calculated from profile-corrected SFDI.

5.5.

Hemisphere Correction

We tested the profile-correction algorithm on a hemisphere phantom (${\mu }_a=0.012{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=1.28{\mathrm{mm}}^{-1}$; 639 nm; $Ø=1.9\mathrm{cm}$, $\text{height}=1\mathrm{cm}$) with both height and surface normal variations (Fig. 6). The top of the hemisphere was placed near the reference calibration height, and this hemisphere was placed on a flat phantom with different optical properties (${\mu }_a=0.023{\mathrm{mm}}^{-1}$, ${\mu }_s^{\prime }=0.99{\mathrm{mm}}^{-1}$; 639 nm). Figure 6(a) shows the fractional error maps between laparoscopic si-SFDI and ground truth SFDI optical properties overlaid on the active stereo-estimated surface point cloud. For polar angles less than 40 deg, profile-correction reduced the mean percent error for the absorption coefficient from 39.7% to 24.0% and for the reduced scattering coefficient from 16.1% to 4.8%. Moreover, profile-correction decreased the coefficient of variation from 53.7% to 30.5% for the absorption coefficient and from 7.9% to 4.1% for the reduced scattering coefficient. Profile-correction fails for absorption at polar angles greater than 40 deg likely due to inaccuracies in steep surface normal estimations and unaccounted object-to-object diffuse reflectance at the intersection of the hemisphere and flat phantom, which are not addressed by our Lambertian reflectance correction model. For polar angles greater than 40 deg, the mean percent error after profile-correction remained over 70%, necessitating a higher complexity angular correction scheme, which will need to be explored in future studies. In contrast, for polar angles greater than 40 deg, this had a less detrimental effect on the reduced scattering coefficient as the profile-correction algorithm reduced the mean percent error from 28.3% to 8.9%. The reduced scattering coefficient is less sensitive to steep surface normal angles compared with the absorption coefficient likely due to the size of the sliding window used for $0.2{\mathrm{mm}}^{-1}$ spatial frequency resolution relative to the profile variations of the phantom. In addition, the radial averaging of the ACF may smooth out steep angular variations in the local surface structure. It should also be noted that, even after profile-correction, the error for the hemispherical phantom was not radially uniform, with errors being higher in the back. This phenomenon is due to the low signal caused by shadowing when using a single source of angled illumination. The background phantom, on which the hemisphere is placed, also underwent a height correction. This correction reduced the mean percent error for the absorption coefficient from 27.8% to 8.8% and from 8.1% to 3.9% for the reduced scattering coefficient. Figure 6(b) shows a cross-sectional line profile through the central region of the hemisphere, comparing the ground truth data (green solid line), uncorrected data (red dash-dot line), and profile-corrected data (blue dashed line). The absorption line profile demonstrates how the vertical correction shifts the optical properties toward the ground truth values, whereas the angular correction transforms the curved profile into a flat, rectangular shape. Similarly, the reduced scattering profile exhibits the same corrective trends, albeit to a lesser extent. We note that the optical properties at the top of the hemisphere in Fig. 6(b) should be accurate in the uncorrected case as it is near the calibration height. However, consistency is not observed because the sliding window incorporates data from varying depths and surface normals near the hemisphere’s top. This highlights a limitation of using sliding window techniques for measuring small, complex topographies.

Fig. 6

Hemispheric phantom with profile-correction. (a) Overlay of the fractional error map onto the active stereo estimated point cloud of uncorrected (left column), profile-corrected (right column), absorption (top row), and reduced scattering (bottom row) coefficients. (b) Cross-sectional line profile plots through the central region of the hemisphere show the absorption (top row) and reduced scattering (bottom row) coefficients for uncorrected (red dash-dot line), profile-corrected (blue dashed line), and ground truth (green solid line) data.

5.6.

In vivo Optical Property Measurements

The profile-corrected optical properties of a complex in vivo object, consisting of two human fingers with the middle finger constricted by a tourniquet, were measured at multiple time points. This study included informed consent from the participants and was approved by the Johns Hopkins Institutional Review Board (IRB00344195). The decrease in the absorption coefficient at 639 nm revealed the expected physiological changes due to constriction. Figure 7(a) shows the left stereo RGB speckle image overlaid onto the active stereo-estimated point cloud of two fingers from the right hand placed on top of a flat, homogenous phantom. In these experiments, the middle finger (top finger) was constricted using a tourniquet, which can be seen in the top corner of Fig. 7(a). At the time point shown in this image, the middle finger had been constricted for 180 s. The ring finger (bottom finger) was perfused for the entire experiment. The blue box shows the $40\times 40\text{-}\text{pixel}$ ROI used to calculate the mean and standard deviation of optical properties, as shown in Fig. 7(b) at 180 s. The red line shows the line profile location across the constricted middle finger and perfused ring finger used for Fig. 7(c). To assess laparoscopic si-SFDI’s sensitivity to temporally changing absorption optical properties due to physiological changes associated with blood flow constriction, Fig. 7(b) presents profile-corrected optical properties of the constricted middle finger measured at 30-s intervals for 16 consecutive measurements, spanning a total of 450 s. The optical properties were obtained using both profile-corrected laparoscopic si-SFDI (red triangles) and ground truth from the commercial SFDI system with profile correction (blue circles). The tourniquet was applied after the $t=0\mathrm{s}$ measurement (first arrow) and removed after the $t=240\mathrm{s}$ measurement (second arrow). The hand was moved between the laparoscopic si-SFDI system and the commercial SFDI system after each measurement, and a 3 s offset was added to each time stamp to account for the time to switch between systems. Note that the ground truth SFDI markers are offset by 3 s to account for the time delay caused by the slight movement of the hand to the SFDI imaging FOV and the subsequent image acquisition. The mean percent error between laparoscopic si-SFDI and ground truth SFDI was 8.2% and 5.8% for absorption and reduced scattering, respectively. As the finger constriction began, we observed a rapid increase in absorption of $0.025{\mathrm{mm}}^{-1}$ within the first 30 s, followed by a gradual rise of $0.017{\mathrm{mm}}^{-1}$ over the next 210 s [Fig. 7(b)]. Upon tourniquet release, absorption returned to the baseline value of $0.020{\mathrm{mm}}^{-1}$ within 30 s. These observations at 639 nm align with known physiological responses associated with increased deoxyhemoglobin and total hemoglobin levels. The gradual absorption increase during constriction and rapid post-occlusion recovery to baseline is consistent with findings reported by Ikawa and Karita.⁴⁴ By contrast, as expected, the reduced scattering [Fig. 7(b); right plot] did not change during constriction and remained at a baseline value of $\sim 1.33{\mathrm{mm}}^{-1}$. To assess the performance of our profile-correction algorithm on in vivo data and examine the spatial changes in optical properties due to constriction, Fig. 7(c) presents cross-sectional line profiles for absorption and reduced scattering. The profiles compare uncorrected data (red dash-dot line), profile-corrected data (blue dashed line), and ground truth profile-corrected SFDI data (green solid line). This figure highlights the changes in absorption at 639 nm between the constricted and perfused fingers at a single time point ($t=180\mathrm{s}$). In the absorption and reduced scattering line profiles [Fig. 7(c)], we thresholded low signals prevalent due to shadowing at the cracks and edges of raised surfaces. The threshold was determined by analyzing the histogram of the flat-field corrected image. Based on the estimated profiles for the hemisphere (Fig. 6) and fingers (Fig. 7), both objects exhibit similar angular diversity up to moderate angles. Moving 5 mm radially from their peaks, the angle changes by $\sim 6\mathrm{deg}$ per mm, with the angles becoming steeper further outward. At large angles, the finger model diverges from the phantom model with large crevices and occlusions from neighboring fingers. Our model fails at these steep height changes. For absorption [Fig. 7(c); left plot], we saw slight improvements due to height-correction, but minimal changes from angular-correction. By contrast, for reduced scattering [Fig. 7(c); right plot], the profile-correction algorithm did not yield significant improvements between the uncorrected and profile-corrected line profiles. To obtain overlaid images for consistent line profile analysis, ground truth SFDI optical property maps were registered to si-SFDI optical property maps through an affine transformation. The line profile represents an average from a rectangle with a 10-pixel thickness.

Fig. 7

Profile-corrected, time-series, in vivo finger occlusion study. (a) Left stereo RGB speckle image overlaid onto the active stereo-reconstructed point cloud. The blue square indicates the ROI analyzed in panel (b), and the red line represents the line profile for panel (c) at $t=180\mathrm{s}$. (b) Time-series comparison of profile-corrected laparoscopic si-SFDI and ground truth SFDI estimated absorption (left) and reduced scattering (right) coefficients. Tourniquet constriction began after the $t=0\mathrm{s}$ measurement (first arrow) and was removed after the $t=240\mathrm{s}$ measurement (second arrow). (c) Comparison of absorption (left) and reduced scattering (right) coefficient profiles at $t=180\mathrm{s}$. Each plot shows uncorrected laparoscopic si-SFDI (red dash-dot line), profile-corrected laparoscopic si-SFDI (blue dashed line), and ground truth profile-corrected SFDI (green solid line).

6.

Discussion

In this work, we introduced a novel stereo laparoscope implementation of 3D profile-corrected laser speckle si-SFDI. The proposed approach enables the simultaneous measurement of tissue topography and profile-corrected optical properties through a commercial stereo laparoscope. This method employed a coherent laser coupled into a multimode fiber to project an objective speckle pattern onto the sample. We characterized the tissue response in the spatial frequency domain by analyzing the PSD of each speckle image, which allows tissue optical properties to be calculated through phantom calibration.²³ For non-flat samples, we used active stereo to estimate the sample’s 3D profile and then used this profile to correct for height- and angle-dependent reflectance variations.

Our experiments showcased the initial critical steps necessary for the clinical implementation of si-SFDI as a non-invasive, quantitative contrast imaging technology for surgical guidance. First, we validated laparoscopic si-SFDI against ground truth SFDI on 13 homogeneous tissue-mimicking phantoms and observed an average error of 6.4% for absorption and 5.8% for reduced scattering. This validation across diverse optical properties demonstrated our method’s accuracy, which is crucial for surgical guidance, as the precision is sufficient to differentiate disease states such as carcinoma.⁴⁵ Second, during surgery, the sample’s distance is typically heterogeneous and changes continuously due to manipulation of the surgical field. Uncorrected results in Fig. 4 demonstrated that optical property estimation for samples at distances different from the calibration phantom requires profile-correction. As the sample distance deviated from the reference calibration distance, the absorption and reduced scattering mean error per cm reached 78.5% and 21.6%, respectively. Applying the height correction over a 4-cm range reduced the mean error per cm to 2.8% for absorption and 1.3% for reduced scattering coefficients. Third, to assist the surgeon, laparoscopic si-SFDI should be able to distinguish different tissue types at various heights based on their optical properties within a single FOV. To validate this capability, we tested our method on complex phantom arrangements with varying optical properties within a single FOV: multiple flat phantoms at different heights (Fig. 5) and a hemispheric phantom (Fig. 6). The profile-correction algorithm maintained the mean absolute error within $0.007{\mathrm{mm}}^{-1}$ for absorption and $0.11{\mathrm{mm}}^{-1}$ for reduced scattering maps. Last, monitoring wide-field optical properties, especially absorption, over time is essential for confidently assessing tissue viability and detecting early-onset ischemia.^14,15,46 Figure 7 demonstrates the ability of laparoscopic si-SFDI to accurately monitor temporal changes in tissue absorption resulting from hypoxia induced by vessel constriction. Moreover, unlike local contact techniques such as tissue oximetry, laparoscopic si-SFDI can identify ischemic regions over a wide FOV without prior knowledge of the ischemic region’s location, making it attractive for image-guided surgery.

The results are promising and have demonstrated feasibility in phantom and simple in vivo studies; however, many factors should be considered to further strengthen this technique. First, to estimate the 3D profile for downstream profile-correction, we restricted our maximum estimated angle to 40 deg (Fig. 3). At tilt angles of $\pm 30$ and $\pm 40\mathrm{deg}$ [Fig. 3(c)], our camera’s limited dynamic range struggled to capture both nearby and distant details of the tilted phantom simultaneously. Therefore, the active stereo-reconstructed point clouds exhibited high levels of noise and error, which propagated through the optical property correction algorithm. An important tradeoff during the profile estimation is the resolution versus accuracy of the measurement. To reduce noise, we downsampled our point cloud through smoothing and grid-fitting.⁴¹ However, this caused the profile estimations to lose important surface normal information at steep steps and curved object interfaces. This was evident in the hemisphere phantom (Fig. 6), the crack between the constricted and perfused fingers [Figs. 7(a) and 7(c)], and other areas with significant height changes (Fig. 5). This tradeoff between resolution and accuracy can be mitigated by imaging at smaller distances although this approach reduces the FOV. Second, even with correct angular surface normal estimation, there were errors associated with angled, single-source illumination (Fig. 4) and inter-object diffuse reflectance (Fig. 6), which exposed the limitations of the Lambertian reflectance assumptions governing our profile-correction algorithm. In Fig. 4, we observed asymmetrical errors in optical property estimations with sample front tilt. These asymmetries could be circumvented by illuminating parallel to the detection axis by utilizing the commercial fiber light guide that is parallel and close to the laparoscope objective lenses. A parallel illumination scheme would not affect profile estimation as surface triangulation is enabled by dual camera disparity. This solution may also reduce barriers with pilot clinical studies because it may be possible to project laser speckle patterns through the laparoscope fiber guide, leaving the distal instrument unmodified. In Fig. 6, we observed large errors in the profile-corrected absorption at the junction of the hemisphere and flat phantom due to the algorithm’s insensitivity to optical property changes caused by inter-object diffuse reflectance. Future work could integrate a modified Lambertian correction,⁴⁷ which demonstrated accurate angular correction up to 75 deg in SFDI by adding a correction factor to the Lambertian correction term, accounting for inter-object diffuse reflectance. Third, our results demonstrated sensitivity to optical property changes when phantoms with diverse optical properties were placed at multiple heights within the same FOV (Fig. 5). This sensitivity was observed when the phantoms’ optical properties were chosen within a 50% absorption and 25% reduced scattering difference of the calibration phantom. Although these results highlight the sensitivity of our profile-corrected si-SFDI method for optical property combinations within this range, we did not test the range at which the error would significantly increase. This limitation is primarily due to the dynamic range of the 8-bit (per channel) color camera used in this proof-of-concept imaging system. For example, Fig. 5 shows increased error in the high-absorbing phantom partly due to the tradeoff in selecting an exposure time. The chosen exposure was optimized for the low-absorbing phantom, causing the high-absorbing phantom’s signal to approach the sensor’s noise floor. To address this challenge and enhance the system’s performance, future studies could employ a 16-bit monochrome camera, which will not only resolve the dynamic range issues but also double the sampling resolution of our laser speckle, enabling the use of smaller objective speckle grains for better local optical property sensitivity.

There are several areas for improvement of laparoscopic si-SFDI to make it amenable to clinical testing. First, to reduce the effects of motion noise, it is critical to improve the imaging speed. This challenge can be addressed using brighter light sources, pulsed illumination, faster exposure times, and red and near-infrared (NIR) wavelengths that are weakly absorbed by vascularized tissue. In addition, using brighter light sources and employing red and NIR wavelengths not only mitigate motion noise but also improve signal-to-noise ratios in highly vascularized and absorbing tissues encountered during surgery. Second, it is important to have video-rate data acquisition. The current prototype requires at least 1 s between consecutive acquisitions. This delay allows the LSR to toggle on and off and ensures that residual vibrations are eliminated, preventing mode scrambling in the AC image acquisition following the previous DC image acquisition. A spinning half-diffuser could be used to increase acquisition speed, allowing for optical property data acquisition at half of the rotational frequency of the optical chopper. Alternatively, it may be possible to estimate optical properties from a single speckle image-previous work⁴⁸ has shown that a DC-illumination can be estimated from a speckle image using a content-aware deep learning approach. Third, it is important to have faster data processing. The current processing speed for profile-corrected si-SFDI is slow due to stereo matching, 3D reconstruction, and sequential window-based computations. Using a 4-core 3.6 GHz processor in MATLAB, it takes $\sim 12\mathrm{s}$ to generate $784\times 960\text{-}\text{pixel}$ profile-corrected optical property maps from raw data acquisitions: 7 s to estimate depth and surface normals from the stereo pair, and 5 s for the si-SFDI algorithm to apply profile correction and estimate optical properties. However, as each window computation is independent, this algorithm is compatible with parallelization using a graphics processing unit. Another possibility is to explore an end-to-end deep learning model trained to directly predict profile-corrected optical properties from a single speckle image. Chen et al.⁴⁹ demonstrated that a single structured light image could be used to predict optical properties in real time. Furthermore, Aguénounon et al.⁵⁰ showed that a twin network could perform both surface profile correction and accurate optical property extraction from a single structured light image. Third, it is necessary to optimize the objective speckle contrast to enhance the sensitivity and accuracy of si-SFDI optical property estimation for diverse samples within the same FOV. To enhance objective speckle contrast, we could potentially use beam shaping techniques to preferentially excite higher-order modes while attenuating lower-order modes.⁵¹

Clinical adoption of our technique as a noninvasive quantitative surgical guidance tool relies on advancing it from laparoscopic optical property estimation to providing clinically relevant surgical information, such as wide-field oximetry measurements and quantitative margin assessments in fluorescence-guided surgery. Incorporating a second wavelength would enable tissue oximetry imaging, which was originally demonstrated in SFDI.¹⁴ Furthermore, when coupled with fluorescence-based contrast mechanisms, optical properties can be used to correct emitted fluorescence, providing quantitative fluorescence estimates⁵² and 3D localization of fluorescently labeled subsurface tumors and structures.^53–55

7.

Conclusion

In this work, we presented a speckle-illumination stereo laparoscope that acquires profile-corrected optical properties. We detailed the system’s design principles, data acquisition, and processing workflow and validated its accuracy on tissue-mimicking phantoms with complex geometries against a commercial SFDI system as ground truth. Furthermore, we demonstrated the system’s capability for in vivo imaging of time-varying physiological changes. By building upon the principles of speckle-illumination wide-field imaging of optical properties, this work lays the foundation for image-guided minimally invasive surgery using quantitative endogenous contrast.