|
1.IntroductionThree-dimensional (3-D) particle-localization using in-line holography is fundamental to many applications, such as biological sample characterization,1,2 flow cytometry,3,4 fluid mechanics,5,6 and optical measurement.7–9 Reconstructing dense samples, however, remains challenging.10 Standard backpropagation method (BPM) can only handle low particle density.10 Compressive holography based on the first Born approximation significantly improves upon BPM by imposing sparsity constraints.11,12 However, it suffers from an underlying single scattering assumption, which becomes invalid at high particle densities where multiple scattering effects become significant. In this work, we propose a framework that accounts for multiple scattering in in-line digital holography and enables accurate 3-D particle localization at high density in a computationally efficient fashion. Multiple scattering induces a nonlinear relation between the permittivity contrast and the scattered field, making it difficult to invert.13 Many algorithms have been proposed to solve the inverse multiple scattering problem and demonstrated improved performance over single-scattering methods, such as iterative Born series,14–18 contrast source inversion,19–22 modified gradient,23,24 series expansion with accelerated gradient descent on Lippmann–Schwinger equation (SEAGLE),25,26 and hybrid methods.27–30 However, computational challenges restrict them to be demonstrated only for small-scale problems. This is because modeling multiple scattering necessitates computing the internal scattered field within the object volume. Furthermore, the effectiveness of existing multiple scattering methods has been demonstrated only under multishot tomography. While multiple measurements do alleviate the ill-posedness of the inverse problem, they also increase acquisition time and system complexity. Therefore, it is of interest to investigate whether one can exploit multiple scattering using only a single-shot measurement. In this work, we demonstrate successful inverse multiple scattering for large-scale problems and reconstruct 100 million voxels from a single 1-megapixel in-line hologram. We show that, even under such highly ill-posed conditions, inversion of multiple scattering is still possible and can be used to improve results compared to single scattering techniques. To calculate multiple scattering, we build our model based on the Born series expansion.13 To make it computationally efficient, we take a multislice approximation by discretizing the 3-D object volume into a series of two-dimensional (2-D) thin axial slices. At each slice, each object voxel takes a uniform refractive index value. Between neighboring slices, the uniform background medium is assumed. By adjusting the voxel size and interslice distance, our model allows us to flexibly trade computational complexity for model accuracy. At the limit when the voxel size equals the interslice distance, our discretization reduces to the existing approaches in Refs. 1415.–16. Our computational structure closely resembles the multislice model (i.e., beam propagation method).31–33 However, our model has the benefit of computing both forward and backward scattering, whereas the latter only accounts for forward multiple scattering. To compute multiple scattering, we introduce a 3-D-to-3-D operator to efficiently evaluate the internal scattered fields within the volume. The computational framework discretizes the 3-D object as a set of 2-D slices, and multiple scattering is modeled as recursive propagation among them. Starting from the initial field, each subsequent recursion estimates the next higher-order scattering term within the object volume. This process can be carried out up to an arbitrary order until the field converges to a steady state. To evaluate the convergence, we adapt a metric derived from the residual error of the internal field.19,25 Next, we devise a 3-D-to-2-D operator that computes the external scattered fields by propagating the multiply scattered internal 3-D field to the 2-D sensor plane. Finally, the intensity measured by the hologram is the interference between the scattered and the unscattered fields. This further complicates the model by introducing the “twin-image” problem.34 If only single scattering is considered, our model reduces to linear compressive holography.11 As a result of multiple scattering computation, the hologram encodes information about the high-angle scattering within the volume, which is otherwise ignored in single scattering-based methods. We show that this extra information leads to better recovery of the scatterers, in particular at larger depths. To solve the inverse scattering problem, we derive an optimization procedure that iteratively minimizes the data-fidelity term measuring the difference between the estimated and measured holograms and imposes a sparsity-promoting regularization on the object. The overall structure of the algorithm follows the proximal-gradient method.35 The key ingredient is the gradient computation of the data-fidelity term. Conveniently, our recursive forward model leads to a similarly structured recursive gradient computation. Further exploiting the convolution structure in the scattering operators, the algorithm is implemented using efficient FFT-based computations. Distinct from prior Born-series-based models,14–18 we do not directly measure the full complex field. The effect of sparsity regularization on the twin-image artifact has been studied using single-scattering11,36 and 2-D multiple-scattering models.27 Here, we show that the sparsity is also effective in suppressing twin-image artifacts under 3-D multiple-scattering models. An important feature of our multislice-based framework is that the 3-D object can be flexibly estimated with any desired number of axial slices, as set by the targeted resolution. In particular, we show that it is possible to use much fewer axial slices to achieve high localization accuracy while still exploiting the extra information contained in the multiple scattering. This allows us to handle much larger scale problems with reduced computational cost as compared to existing techniques that are often limited by fine sampling requirements. Single scattering-based methods tend to underestimate the refractive index contrast. This underestimation can be mitigated by incorporating multiple scattering.17,25,37,38 We show this effect using our multislice-based approach in single-shot in-line holography and demonstrate improved particle localization and axial resolution under multiple scattering. Next, we demonstrate the localization accuracy of our method by imaging 3-D distributed particles in water at various densities in both simulation and experiment. To facilitate quantitative comparison of different methods, we use a classification framework and use the receiver operating characteristic (ROC) curve to determine each method’s best performance. At low particle density, our multiple-scattering model converges to the single scattering solution as expected since the information is dominated by the first-order scattering. At high particle density, our model largely improves the accuracy since multiple-scattering becomes more significant. We observe that the localization accuracy is highly depth-dependent. Following the classification framework, we use the Dice coefficient39 to quantify the localization result slice-by-slice. We show that our multiple-scattering model provides greater improvement at larger depths. 2.Theory and Method2.1.Forward ModelConsider the imaging geometry in Fig. 1(a). An in-line hologram at the measurement plane can be written as where is the scattered field on the measurement plane, is the incident plane wave and is assumed to be real on the hologram plane () without loss of generality, and represents the transverse spatial coordinates on the hologram plane. The self-interference term of the scattered field is ignored; the validity of this assumption is discussed in Sec. 3.2. The scattered field and its “twin-image” contribution is related to the measured hologram after background removal byFig. 1In-line holography with multiple scattering. (a) A plane-wave is incident on a 3-D object containing distributed scatterers. The field undergoes multiple scattering within the volume and then propagates to the image plane. A hologram is recorded, which is then used to estimate the unknown scatterers’ distribution. (b) An inline holography setup is used that consists of a collimated laser for illumination and a 4F system for magnification. (c) The raw data are a single hologram. (d) The reconstruction implements a nonlinear inverse multiple scattering algorithm.40 (e) The output estimates the 3-D distribution of the scatterers. ![]() The background-removed hologram thus represents the real component of the scattered field at the measurement plane and is given as . To model the hologram resulting from multiple scattering up-to the ’th order, we apply the framework of Born series expansion,13,16 which gives two coupled equations: where is the 3-D Green’s function and is the ’th-order multiply scattered field within the volume . This mathematical model is based on the scalar Helmholtz equation,13 and polarization effects are neglected. This internal field is computed recursively within the support using the Born series [Eq. (4)]. The permittivity contrast is related to the refractive index by ,13 where is the index of the homogeneous background medium, and is the wave number in free space. For simplicity, is assumed to be real valued and absorption effects are ignored. We note that the spatial coordinates associated with the object are within the 3-D support, , , whereas the hologram is measured outside the support, . To compute higher order scattering, the initial condition is , and indexes the scattering order. When , is the incident field [from Eq. (4)], and Eq. (3) reduces to the first Born approximation that linearly relates the object to the singly scattered field. When , this relation becomes nonlinear and the second-order multiple scattering is taken into account via modeling of the additional interaction between the object and the field within the volume. For larger , the approximation becomes more accurate by accounting for -order multiple scattering.Equations (3) and (4) can be discretized to get the following recursive forward model: for . Here, and have dimension , and is . , , and are the number of pixels along the , , and within , respectively. Here, represents a diagonal matrix with the vector on the main diagonal.We consider the 3-D volume to be a set of discrete 2-D slices along the longitudinal axis. and are the scattering operators that represent propagation among the object slices and to the hologram plane. propagates the field from the object support to the hologram plane. , where is a block-diagonal matrix. and are the 2-D DFT and the inverse 2-D DFT matrices, respectively, each with dimension . , where is a diagonal matrix representing the discrete transfer function that performs propagation between two slices, from the slice to . This treatment of the operator is similar to that in Ref. 11. is the multiple scattering operator that performs propagation from the object volume to within itself (Fig. 2). , where and . has the dimension of and contains transfer functions that propagate the field from each slice to every slice within the support. There are two methods of computing the elements in the transfer function having dimensions , the direct and the angular spectrum methods.41 We use the direct method, in which the Green’s function is sampled in the spatial domain, followed by slicewise 2-D FFT, where . Fig. 2Illustration of the 3-D internal scattered field operator in Eq. (6). (a) Each object slice is first voxelwise multiplied by the lower order scattered field ; it is then propagated to every other slice within the volume. (b) This computed scattered-field is added to the incident-field to obtain the next higher-order Born-field . This process is recursively applied to compute the multiply scattered field within the volume. ![]() An important numerical treatment to is around the singularity at . We adapt the technique from Refs. 42 and 43 and consider a spherical exclusion zone around of radius , inside which the Green’s function is assumed to take a constant value. Effectively, this assigns an “averaged” Green’s function value around the singular region. Empirically, we found that, at low refractive index, the choice of does not significantly affect the result, as long as the center voxel (at ) is excluded. For a high refractive index, highly affects the convergence of the forward model. We set to match the largest expected radius of the particles. This means that the strong multiple scattering inside each individual particle cannot be reliably modeled at high contrast; hence it is ignored. Only particle–particle interactions are modeled. Correspondingly, during the inversion, sets the largest particle size that can be recovered by our model for high index particles. 2.2.Inverse ProblemTo estimate the object from the holographic measurement, we need to solve Eqs. (5) and (6). Unlike traditional digital holography, this problem is nonlinear when . We devise an inverse scattering algorithm44,45 that minimizes a cost function to compute the estimated object as follows: where is the data fidelity term in which is the real valued measured hologram after background removal, and is the complex-valued scattered field estimate from our model [Eq. (5)]. provides an estimate for [Eq. (2)], represents the L2-norm, is the convex set that constrains the object to be nonnegative, and is the regularization parameter that is empirically tuned. Here, imposes a penalty on the total variation (TV) of the object and is defined as where is the discrete gradient operator with matrices , , and denoting the finite difference operations along -, -, and -directions, respectively.The minimization in Eq. (7) is implemented via the proximal-gradient method,46 in which the ’th iteration is written as where is the proximal operator for TV minimization,47 and is the step size set via backtracking line search.48 The initialization is .Similar to the forward model, the gradient computation is also a -order recursion: for . Here, is the residual; and represent the Hermitian and complex conjugate of the matrix , respectively. The recursion is initialized with . Brute-force evaluation of the gradient is highly computationally intensive for large-scale problems, with each vector having more than a few million elements. We devise a computationally efficient implementation by making use of the FFT-based structures in and operators. This algorithm extends the framework in Ref. 16 from small-scale 2-D to large-scale 3-D problems and further demonstrates reconstruction from intensity-only as opposed to full-field measurements.3.ResultsWe test our model on both simulations and experiments. In our experiment, the inline holography setup uses a linearly polarized HeNe laser (632.8 nm, 500:1 polarization ratio, Thorlabs HNL210L) that is collimated for illumination [Fig. 1(b)]. A 4F system with a objective lens (0.4 NA, CFI Plan Achro) and a 200-mm tube lens is used to collect the scattered field with the Nyquist sampling requirement satisfied. A CMOS sensor (FLIR GS3-U3-123S6M-C) is used to capture the holograms. The object consists of polystyrene microspheres with nominal diameter (Thermofisher Scientific 4009A) suspended in deionized water. The suspension is held in a quartz-cuvette with inner dimensions . We are interested in localizing the individually suspended scatterers. A shutter speed of 5 ms was used and found to be sufficiently fast to capture the holograms without any motion artifacts from suspended particles. The illumination beam diameter is less than the width of the cuvette and larger than the CMOS sensor area to avoid edge artifacts. The front focal plane of the objective lens was set just outside the inner wall of the cuvette for hologram recording. Importantly, Eq. (6) requires computation of high-angle multiply scattered field propagating within the volume; thus the internal field needs to be sampled at the Nyquist rate . In our system, the camera’s pixel-size is , and the effective lateral sampling size after magnification is . This satisfies the sampling requirement in the medium, where the wavelength is . We set the voxel size along axial direction , such that the voxels are cubic. The spacing between slices is assumed to contain uniform background medium and is set to be , approximately matching the system’s axial resolution of . During the computation, zero-padding is used in all FFTs to avoid boundary artifacts. We demonstrate large-scale inverse scattering that reconstructs 100 million voxels in a volume. For large-scale simulation, we model the system parameters to approximately match the physical setup. On such a scale, rigorous solutions such as FDTD are computationally prohibitive, and sample complexity makes analytical solutions such as Mie theory nontrivial. We first study the effect of multiple scattering on simulated holograms using 3-D SEAGLE,25 which is an accurate forward model that incorporates multiple scattering, including scattering within each particle. It is based on a rigorous optimization procedure that solves the Lippmann Schwinger equation. We further simulate the hologram at high particle densities using our model with a sufficiently high scattering order, e.g., , such that the model converges and the simulated field closely estimates the actual. In order to validate the convergence, we present an evaluation metric and show that the model converged within the first few scattering orders for all tested scenarios. Improvement of our method compared to single scattering is presented quantitatively. The Boston University Shared Computing Cluster (SCC) was used for all computations. The average times of computing one iteration for the single- and multiple-scattering models on a grid were 58 and 257 s, respectively. All reconstructions were run for 100 iterations. In what follows, we present our findings. 3.1.Effect of Multiple Scattering in Small-Scale Inversion: A Multislice-Based ApproachIt has been shown that in the presence of strong multiple scattering, the single-scattering models underestimate the permittivity contrast.17,25,37 Here we validate our model on a small-scale simulation and make similar observation by showing that the underestimation is mitigated as multiple scattering is incorporated in the inversion. The utility of our multislice-based computational approach is also demonstrated, in which the number of axial slices can be arbitrarily chosen in the inverse reconstruction. Effectively, we approximate the 3-D object with a fixed number of slices, such that the computation is tractable when expanding to large-scale problems. We simulate a volume of , discretized as a object, containing eight spheres in water, each with refractive index and diameter . In Fig. 3(a), we depict the central region of this object. The multiple scattering is significant in the presence of occluding geometry along the optical axis, and a refractive index contrast of . It is known that occlusion causes strong axial field coupling via multiple scattering between scatterers, which is ignored by the first-order model.49 We therefore expect that incorporating multiple scattering will improve object estimation from the hologram. Fig. 3Small-scale multiple-scattering inversion. (a) An accurate 3-D forward model is used to simulate the hologram. (b) Multislice 3-D reconstruction is performed from a single simulated measurement using our method. The number of slices in the inverse reconstruction can be flexibly chosen. (c) Full 3-D inversion is performed by reconstructing all axial slices in the original object using our method. The multiple-scattering method outperforms the single-scattering method by providing both more accurate permittivity contrast estimation and improved optical sectioning. (d) Our multislice approach enables 3-D reconstruction using a much reduced number of slices while still maintaining the benefit of incorporating multiple scattering. Reconstruction using only three slices is compared to demonstrate the improved localization capability by our method. ![]() An inline hologram is simulated at from the front slice using the SEAGLE. The hologram is then inverted using our multislice-based method incorporating first-, second-order scattering. The scattered intensity is included when simulating the hologram. During the inversion, this term is ignored following the procedure in Sec. 2. Our results indicate that even under this approximation, our model suppresses the underestimation artifacts by incorporating multiple scattering. In order to test the utility of our multislice-based approach, we perform reconstruction for two cases. In the first case, we reconstruct all 37 slices for the object [Fig. 3(c)]. The reconstruction based on the first-order scattering underestimates the refractive indices. We attribute this artifact to the strong axial coupling via multiple scattering between the occluding particles, which is ignored by the first-order model. The underestimation is mitigated when second-order scattering is included in the inverse model. In the second case, we estimate the object using only three slices to perform the inverse scattering reconstruction [Fig. 3(d)]. The reconstruction in this case approximates the 3-D object comprising of three discrete slices. We observe that our method is able to detect the eight spheres as disks at the correct axial locations corresponding to the centers of particles. When using only three slices in the reconstruction, the model has a smaller number of slices to create the same effect at the measurement plane as the 37-slice ground truth object. In order to compensate for this, the reconstructed scattering density can be approximated as the integrated permittivity contrast along the optical axis while still correctly localizing the particles. In the first-order result, we observe smaller contrast and worse axial sectioning. The second-order multiple scattering improves the contrast as well as axial sectioning, resulting in better localization capability. 3.2.Large-Scale Inversion of Multiple-Scattering: SimulationNext, we demonstrate the inversion of multiple scattering from single-shot measurement in large-scale. For this purpose, we design a simulation that involves estimating the concentration of particles in a suspension from its inline hologram. We show that our multiple-scattering model improves the accuracy in estimating the particle density, particularly at larger depths. The simulated volume is , discretized on a grid, in which disk-shaped scatterers of diameter and constant refractive index are suspended randomly in water, in varying densities. The disk shape may not represent actual spheres, as would be in the real application; however, it is taken as an approximation due to stringent sampling requirements for such a large volume. We consider two values of the refractive index contrast and 0.19. For each volume, we first simulate holograms using 20’th-order scattering, followed by the reconstruction using first- and second-order models. The particle density is estimated for each reconstructed volume using the ImageJ 3-D objects counter toolbox.50 The optimal threshold parameter used for calculating the density is determined using the ROC. As a measure of particle density, we consider the geometric cross-section , which corresponds to the fraction of the hologram area directly occluded by the scatterers, defined as where represents the total number of scatterers in the volume. This metric is valid for scattering domains that are not very thick, as in our case. For a collection of identical particles suspended in a homogeneous medium, the geometric and scattering cross-sections are directly related,51 in which the latter is a direct measure of the fraction of the incident light scattered by an object. For higher values of , we thus expect greater contributions of multiple scattering. From the signal processing perspective, also measures the sparsity of the problem as it approximates the ratio between the number of nonzero unknowns and the number of measurements. The values of tested are 0.01, 0.02, 0.05, 0.1, 0.2, and 0.4, corresponding to , 200, 500, 1000, 2000, and 4000, respectively. We simulate five random object volumes for each value of and refractive index contrast and report the mean statistics of the reconstructions.In Sec. 2.1, we assumed that the intensity of the scattered field is negligible in the forward model. In this study, this assumption holds true when , where the contribution of is at least an order of magnitude smaller than the total intensity of the hologram (Fig. 4). For higher particle density, becomes increasingly significant, which leads to greater model error. Fig. 4Effect of particle density on the scattered intensity term contribution in the hologram. (a) Contribution is negligible compared to the hologram for low particle densities and becomes gradually important as the particle density increases. (b) The ratio between the total intensity of the hologram and the terms for all values of tested in the simulation. For , the total intensity of the hologram is at least an order of magnitude larger than the term. ![]() For the series expansion approach used in our model, it is important to evaluate its convergence. In Fig. 5(a), we present the convergence properties of the forward model under our experimental conditions. While in general higher-order terms are required for convergence under stronger scattering, the second-order scattering is sufficient for most of the cases studied. Our convergence metric is defined by the residual error of the field within the 3-D volume,19,25,30 as where . This convergence metric essentially measures the self-consistency of the total internal field.25 For -order scattering, it computes the norm of the residual contribution from ()-order scattering, which must approach zero in the case of convergence.Fig. 5Validation of our multiple-scattering method on large-scale simulation. (a) Convergence properties of the forward model are studied under varying particle densities. Higher-order scattering is generally required for convergence when the object is strongly scattering. In most cases studied, two orders of scattered field sufficiently capture the majority of the contribution. (b) For higher refractive index contrast (), multiple-scattering performs similarly to single-scattering for low concentration (), and better than single-scattering for . Reconstruction fails for very high concentration (), i.e., when the SNR drops below an empirically chosen value of 1 dB. The error in the predicted versus the ground truth particle concentrations also shows a similar trend. (c) For lower contrast (), multiple scattering contributions are negligible and both methods give similar performance. (d) A 3-D rendering depicting localized particles is shown for and . Both methods have similar performance for slices close to the image plane, but our multiple-scattering model performs better at increased depths. ![]() Next, we evaluate the reconstruction accuracy by measuring the signal-to-noise ratio (SNR) where and are the true and estimated objects, respectively. For higher index contrast (), our multiple-scattering model performs consistently better than the single-scattering model for all densities tested [Figs. 5(b) and 5(c)]. In general, the reconstruction performance from both methods drops as the density increases. We attribute this to stronger higher-order scattering and decrease in the object sparsity. The stronger scattering introduces higher-order nonlinearity through Eq. (6), making the problem harder to invert. The decrease in object sparsity leads to an effective smaller measurement-to-unknown ratio further worsening the ill-posedness of the problem.For higher contrast (), at low particle density (), single and multiple scattering methods perform similarly. This is expected since multiple scatterings are weak due to the small scattering cross section. For , our method outperforms the single scattering method, providing a better estimate of the actual particle density. For , the SNR drops below 1 dB, and we empirically consider that the reconstruction has failed [Fig. 5(b)]. For lower index contrast (), both multiple and single scattering methods perform almost identically for all densities tested, which indicates that the contribution from multiple scattering is negligible for low refractive index contrast [Fig. 5(c)]. 3-D renderings and cross-sectional reconstructions at different depths are shown in Fig. 5(d). The depth-dependent performance is highlighted in Figs. 6(a) and 6(b). Close to the hologram plane (), single and multiple scattering reconstructions are similar and match the ground truth. At larger depth (), the single-scattering reconstruction degrades and results in a large number of missing particles [Fig. 6(a)]. Our multiple-scattering model improves the localization at larger depths. By treating particle localization as a binary classification problem, we use the ROC curve to determine the optimal segmentation threshold when quantifying the voxels reconstructed by each method.10 This allows us to evaluate the localization accuracy slice-by-slice, whose statistics are accumulated by five different object volumes for each particle density. The statistic we use for comparison is the Dice coefficient that is used to gauge the similarity of two samples and is defined as where and each represent a voxel from the predicted and ground truth binary segmentation volumes, respectively, and indexes the voxels of each 3-D volume. Consistent with the visual inspection in Fig. 6(a), the Dice coefficient clearly indicates improvement at larger depth using our multiple-scattering model [Fig. 6(b)]. In addition, the area under each ROC provides a direct measure of the algorithm’s overall classification performance. Our results indicate that the multiple-scattering model consistently outperforms the single scattering method [Fig. 6(c)].Fig. 6Reconstruction performance as a function of depth. (a) Segmentation maps of reconstructed slices (zoomed-in regions) at different depths (true positive, white; true negative, black; false positive, green; false negative, pink). For object slices close to the hologram, both multiple and single scattering methods provide high accuracy. At larger depths, the accuracy deteriorates for both methods. Our multiple-scattering method performs notably better at larger depths for higher particle densities. (b) The slicewise Dice coefficient plotted as a function of slice depth also indicates that the multiple-scattering model provides improved segmentation accuracy, especially at greater depth. (c) The particle localization accuracy is quantified using the ROC curve. The curves corresponding to the multiple-scattering solutions consistently have larger areas underneath, indicating better localization accuracy as compared to the single-scattering method in all cases studied. ![]() 3.3.Large-Scale Experimental ValidationFinally, we demonstrate our method on a set of large-scale experiments. We reconstruct over 100 million object voxels () from each 1-megapixel hologram. Our multiple-scattering model significantly improves the 3-D localization accuracy as compared to the BPM and single-scattering methods. Notably, our experimental results closely match the simulation. We prepare polystyrene microsphere suspensions, ranging from dense to sparse concentrations via successive dilution, with corresponding values of 0.4, 0.2, 0.1, 0.05, 0.0125, and 0.0063. Five holograms are recorded at each concentration, and then used for reconstructions and density estimation. Background subtraction is performed on each hologram as a preprocessing step to remove static artifacts. The inversion is performed using our method with second-order multiple-scattering (), the single-scattering methods, and BPM. First, we evaluate the results based on the optimization convergence cost [Fig. 7(a)]. The multiple scattering method converges to a lower value than single scattering for all densities, indicating better fit to the cost function . The cost increases for both methods with , depicting degradation of reconstruction with increase in particle density. Fig. 7Experimental validation of our method in large-scale. (a) The multiple-scattering model converges to a lower cost than the single-scattering model for all concentrations indicating better fit to the cost function. (b) The reconstructed particle density follows a trend similar to the simulation where multiple-scattering performs better than the single-scattering method for ; both methods fail for . (c) As increases, the hologram gradually resembles speckle patterns, as quantified by the CR. ![]() Next, we assess the estimated particle density. Our multiple-scattering model consistently performs better than the single-scattering model for [Fig. 7(b)]. For , reconstruction fails for both methods as also found in our simulation. Hence, we use as an empirical performance bound of our method for this application. Evidently, the recorded holograms gradually resemble speckle patterns as the particle density increases [Fig. 7(c)]. We quantify the hologram’s contrast ratio (CR) at each density, which can be used as an alternative metric. The CR is calculated as the ratio of the standard deviation to the mean.51 At the critical concentration, the CR is around 0.335. Finally, we closely examine the 3-D reconstructions for and (Fig. 8). For the low-density case, single- and multiple-scattering methods perform similarly due to weak multiple scattering. For the high density case where multiple scattering becomes significant, our method outperforms the single-scattering model. Particle localization degrades with increased depth; our multiple-scattering method provides a more uniform estimation and better localization at increased depth, matching our observations in the simulation. While BPM is able to reconstruct individual particles at the low density, it completely fails for high density and resembles speckles extended across the object volume. Fig. 8A 3-D visualization of the localized particles under different concentrations from our experiment and their lateral cross sections at different depths. For low density, both multiple- and single-scattering methods perform similarly. For high density, the underestimation of particles from the single-scattering method is clearly visible, especially at increased depth. Our multiple-scattering model mitigates the underestimation as it accounts for the intercoupling between particles whose strength increases as the depth. The traditional BPM is effective for low density but completely fails for high density and the reconstruction resembles speckles throughout the volume. ![]() 4.ConclusionWe have presented a new computational framework for utilizing multiple scattering in in-line holography for large-scale 3-D particle localization. Our model recursively computes both forward and backward multiple scattering in a computationally efficient manner. Both simulations and experiments demonstrate the significance of modeling multiple scattering in alleviating depth-dependent artifacts and improving the 3-D localization accuracy compared to traditional methods. Our method may open up new opportunities for large-scale imaging applications utilizing multiple scattering. Our model is currently limited by the convergence regime of the classical Born series expansion, preventing its application to particle density higher than 0.1 geometric cross-section. Recent work on convergent Born series expansion52 provides a promising avenue to extend our model to higher scattering scenarios. The multislice structure proposed in our model provides a flexible framework for trading computational cost for model accuracy. Still, higher-order scattering calculation necessitates longer computational times, which is less appealing for applications requiring real-time reconstructions. To facilitate rapid volumetric estimation without sacrificing accuracy, recent machine learning-based inverse scattering approaches53–55 may be explored in the future. AcknowledgmentsThe authors thank Anne Sentenac, Alex Matlock, and Benedict Diederich for helpful discussions regarding multiple scattering, Yujia Xue for discussions on the simulations, and Quentin Lin for help on the experiment. Disclosures: there are no conflicts of interest related to this article. Funding: National Science Foundation (NSF) (Grant Nos. 1813848 and 1813910). ReferencesI. Moon et al.,
“Automated three-dimensional identification and tracking of micro/nanobiological organisms by computational holographic microscopy,”
Proc. IEEE, 97
(6), 990
–1010
(2009). https://doi.org/10.1109/JPROC.2009.2017563 IEEPAD 0018-9219 Google Scholar
T.-W. Su, L. Xue and A. Ozcan,
“High-throughput lensfree 3D tracking of human sperms reveals rare statistics of helical trajectories,”
Proc. Natl. Acad. Sci. U. S. A., 109
(40), 16018
–16022
(2012). https://doi.org/10.1073/pnas.1212506109 Google Scholar
F. C. Cheong et al.,
“Flow visualization and flow cytometry with holographic video microscopy,”
Opt. Express, 17
(15), 13071
–13079
(2009). https://doi.org/10.1364/OE.17.013071 OPEXFF 1094-4087 Google Scholar
J. Garcia-Sucerquia et al.,
“Digital in-line holographic microscopy,”
Appl. Opt., 45
(5), 836
–850
(2006). https://doi.org/10.1364/AO.45.000836 APOPAI 0003-6935 Google Scholar
J. Sheng, E. Malkiel and J. Katz,
“Digital holographic microscope for measuring three-dimensional particle distributions and motions,”
Appl. Opt., 45
(16), 3893
–3901
(2006). https://doi.org/10.1364/AO.45.003893 APOPAI 0003-6935 Google Scholar
L. Tian et al.,
“Quantitative measurement of size and three-dimensional position of fast-moving bubbles in air-water mixture flows using digital holography,”
Appl. Opt., 49
(9), 1549
–1554
(2010). https://doi.org/10.1364/AO.49.001549 APOPAI 0003-6935 Google Scholar
P. Picart et al.,
“Tracking high amplitude auto-oscillations with digital Fresnel holograms,”
Opt. Express, 15
(13), 8263
–8274
(2007). https://doi.org/10.1364/OE.15.008263 OPEXFF 1094-4087 Google Scholar
U. Schnars and W. P. Jüptner,
“Digital recording and numerical reconstruction of holograms,”
Meas. Sci. Technol., 13
(9), R85
(2002). https://doi.org/10.1088/0957-0233/13/9/201 MSTCEP 0957-0233 Google Scholar
F. Soulez et al.,
“Inverse-problem approach for particle digital holography: accurate location based on local optimization,”
J. Opt. Soc. Am. A, 24
(4), 1164
–1171
(2007). https://doi.org/10.1364/JOSAA.24.001164 JOAOD6 0740-3232 Google Scholar
W. Chen et al.,
“Empirical concentration bounds for compressive holographic bubble imaging based on a Mie scattering model,”
Opt. Express, 23
(4), 4715
–4725
(2015). https://doi.org/10.1364/OE.23.004715 OPEXFF 1094-4087 Google Scholar
D. J. Brady et al.,
“Compressive holography,”
Opt. Express, 17
(15), 13040
–13049
(2009). https://doi.org/10.1364/OE.17.013040 OPEXFF 1094-4087 Google Scholar
Y. Rivenson, A. Stern and B. Javidi,
“Compressive Fresnel holography,”
J. Disp. Technol., 6
(10), 506
–509
(2010). https://doi.org/10.1109/JDT.2010.2042276 IJDTAL 1551-319X Google Scholar
M. Born and E. Wolf,
“Scattering from inhomogeneous media,”
Principles of Optics, 695
–734 7 ed.Cambridge University Press, Cambridge
(2003). Google Scholar
W. C. Chew and Y. M. Wang,
“Reconstruction of two-dimensional permittivity distribution using the distorted Born iterative method,”
IEEE Trans. Med. Imaging, 9 218
–225
(1990). https://doi.org/10.1109/42.56334 ITMID4 0278-0062 Google Scholar
F.-C. Chen and W. C. Chew,
“Experimental verification of super resolution in nonlinear inverse scattering,”
Appl. Phys. Lett., 72
(23), 3080
–3082
(1998). https://doi.org/10.1063/1.121547 APPLAB 0003-6951 Google Scholar
U. S. Kamilov et al.,
“A recursive Born approach to nonlinear inverse scattering,”
IEEE Signal Process. Lett., 23 1052
–1056
(2016). https://doi.org/10.1109/LSP.2016.2579647 IESPEJ 1070-9908 Google Scholar
G. A. Tsihrintzis and A. J. Devaney,
“Higher-order (nonlinear) diffraction tomography: reconstruction algorithms and computer simulation,”
IEEE Trans. Image Process., 9
(9), 1560
–1572
(2000). https://doi.org/10.1109/83.862637 IIPRE4 1057-7149 Google Scholar
R. Pierri and G. Leone,
“Inverse scattering of dielectric cylinders by a second-order born approximation,”
IEEE Trans. Geosci. Remote Sens., 37
(1), 374
–382
(1999). https://doi.org/10.1109/36.739072 IGRSD2 0196-2892 Google Scholar
P. M. van den Berg and R. E. Kleinman,
“A contrast source inversion method,”
Inverse Prob., 13 1607
–1620
(1997). https://doi.org/10.1088/0266-5611/13/6/013 INPEEY 0266-5611 Google Scholar
M. T. Bevacquad et al.,
“Non-linear inverse scattering via sparsity regularized contrast source inversion,”
IEEE Trans. Comput. Imaging, 3
(2), 296
–304
(2017). https://doi.org/10.1109/TCI.2017.2675708 Google Scholar
P. M. van den Berg, A. Van Broekhoven and A. Abubakar,
“Extended contrast source inversion,”
Inverse Prob., 15
(5), 1325
–1344
(1999). https://doi.org/10.1088/0266-5611/15/5/315 INPEEY 0266-5611 Google Scholar
P. Van den Berg and A. Abubakar,
“Contrast source inversion method: state of art,”
Prog. Electromagn. Res., 34
(11), 189
–218
(2001). https://doi.org/10.2528/PIER01061103 PELREX 1043-626X Google Scholar
R. E. Kleinman and P. M. van den Berg,
“A modified gradient method for two-dimensional problems in tomography,”
J. Comput. Appl. Math., 42
(1), 17
–35
(1992). https://doi.org/10.1016/0377-0427(92)90160-Y JCAMDI 0377-0427 Google Scholar
R. E. Kleinman and P. M. van den Berg,
“An extended range-modified gradient technique for profile inversion,”
Radio Sci., 28 877
–884
(1993). https://doi.org/10.1029/93RS01076 RASCAD 0048-6604 Google Scholar
H.-Y. Liu et al.,
“SEAGLE: sparsity-driven image reconstruction under multiple scattering,”
IEEE Trans. Comput. Imaging, 4
(1), 73
–86
(2018). https://doi.org/10.1109/TCI.2017.2764461 Google Scholar
Y. Ma et al.,
“Accelerated image reconstruction for nonlinear diffractive imaging,”
6473
–6477
(2018). https://doi.org/10.1109/ICASSP.2018.8462400 Google Scholar
T.-A. Pham et al.,
“Versatile reconstruction framework for diffraction tomography with intensity measurements and multiple scattering,”
Opt. Express, 26
(3), 2749
–2763
(2018). https://doi.org/10.1364/OE.26.002749 OPEXFF 1094-4087 Google Scholar
J. Lim et al.,
“Beyond Born–Rytov limit for super-resolution optical diffraction tomography,”
Opt. Express, 25
(24), 30445
–30458
(2017). https://doi.org/10.1364/OE.25.030445 OPEXFF 1094-4087 Google Scholar
F. Simonetti,
“Multiple scattering: the key to unravel the subwavelength world from the far-field pattern of scattered wave,”
Phys. Rev. E: Stat., Nonlinear, Soft Matter Phys., 73 036619
(2006). https://doi.org/10.1103/PhysRevE.73.036619 Google Scholar
E. Mudry et al.,
“Electromagnetic wave imaging of three-dimensional targets using a hybrid iterative inversion method,”
Inverse Prob., 28 065007
(2012). https://doi.org/10.1088/0266-5611/28/6/065007 INPEEY 0266-5611 Google Scholar
L. Tian and L. Waller,
“3D intensity and phase imaging from light field measurements in an LED array microscope,”
Optica, 2
(2), 104
–111
(2015). https://doi.org/10.1364/OPTICA.2.000104 Google Scholar
U. S. Kamilov et al.,
“Optical tomographic image reconstruction based on beam propagation and sparse regularization,”
IEEE Trans. Comput. Imaging, 2
(1), 59
–70
(2016). https://doi.org/10.1109/TCI.2016.2519261 Google Scholar
U. S. Kamilov et al.,
“Learning approach to optical tomography,”
Optica, 2
(6), 517
–522
(2015). https://doi.org/10.1364/OPTICA.2.000517 Google Scholar
M. Guizar-Sicairos and J. R. Fienup,
“Understanding the twin-image problem in phase retrieval,”
J. Opt. Soc. Am. A, 29
(11), 2367
–2375
(2012). https://doi.org/10.1364/JOSAA.29.002367 JOAOD6 0740-3232 Google Scholar
A. Beck, M. Teboulle,
“Gradient-based algorithms with applications to signal-recovery problems,”
Convex Optimization in Signal Processing and Communications, 42
–88 Cambridge University Press, Cambridge
(2009). Google Scholar
W. Zhang et al.,
“Twin-image-free holography: a compressive sensing approach,”
Phys. Rev. Lett., 121
(9), 093902
(2018). https://doi.org/10.1103/PhysRevLett.121.093902 PRLTAO 0031-9007 Google Scholar
R. Ling et al.,
“High-throughput intensity diffraction tomography with a computational microscope,”
Biomed. Opt. Express, 9
(5), 2130
–2141
(2018). https://doi.org/10.1364/BOE.9.002130 BOEICL 2156-7085 Google Scholar
J. Lim et al.,
“Learning tomography assessed using Mie theory,”
Phys. Rev. Appl., 9
(3), 034027
(2018). https://doi.org/10.1103/PhysRevApplied.9.034027 PRAHB2 2331-7019 Google Scholar
L. Rokach and O. Maimon, Clustering Methods, 321
–352 Springer US, Boston, Massachusetts
(2005). Google Scholar
D. Mas et al.,
“Fast algorithms for free-space diffraction patterns calculation,”
Opt. Commun., 164
(4–6), 233
–245
(1999). https://doi.org/10.1016/S0030-4018(99)00201-1 OPCOB8 0030-4018 Google Scholar
A. D. Yaghjian,
“Electric dyadic Green’s functions in the source region,”
Proc. IEEE, 68
(2), 248
–263
(1980). https://doi.org/10.1109/PROC.1980.11620 IEEPAD 0018-9219 Google Scholar
J. Van Bladel,
“Some remarks on Green’s dyadic for infinite space,”
IRE Trans. Antennas Propag., 9
(6), 563
–566
(1961). https://doi.org/10.1109/TAP.1961.1145064 IRAPAG 0096-1973 Google Scholar
A. Fannjiang,
“Tv-min and greedy pursuit for constrained joint sparsity and application to inverse scattering,”
Math. Mech. Complex Syst., 1
(1), 81
–104
(2013). https://doi.org/10.2140/memocs Google Scholar
A. Fannjiang,
“Compressive sensing theory for optical systems described by a continuous model,”
(2015) https://arxiv.org/pdf/1507.00794.pdf Google Scholar
A. Beck and M. Teboulle,
“Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems,”
IEEE Trans. Image Process., 18
(11), 2419
–2434
(2009). https://doi.org/10.1109/TIP.2009.2028250 IIPRE4 1057-7149 Google Scholar
U. S. Kamilov,
“A parallel proximal algorithm for anisotropic total variation minimization,”
IEEE Trans. Image Process., 26
(2), 539
–548
(2017). https://doi.org/10.1109/TIP.2016.2629449 IIPRE4 1057-7149 Google Scholar
N. Andrei,
“An acceleration of gradient descent algorithm with backtracking for unconstrained optimization,”
Numer. Algorithms, 42
(1), 63
–73
(2006). https://doi.org/10.1007/s11075-006-9023-9 NUALEG 1017-1398 Google Scholar
M. Azimi and A. Kak,
“Distortion in diffraction tomography caused by multiple scattering,”
IEEE Trans. Med. Imaging, 2
(4), 176
–195
(1983). https://doi.org/10.1109/TMI.1983.4307637 ITMID4 0278-0062 Google Scholar
S. Bolte and F. Cordelieres,
“A guided tour into subcellular colocalization analysis in light microscopy,”
J. Microsc., 224
(3), 213
–232
(2006). https://doi.org/10.1111/jmi.2006.224.issue-3 JMICAR 0022-2720 Google Scholar
A. Ishimaru, Wave Propagation and Scattering in Random Media, 2 Academic Press, New York
(1978). Google Scholar
G. Osnabrugge, S. Leedumrongwatthanakun and I. M. Vellekoop,
“A convergent born series for solving the inhomogeneous Helmholtz equation in arbitrarily large media,”
J. Comput. Phys., 322 113
–124
(2016). https://doi.org/10.1016/j.jcp.2016.06.034 JCTPAH 0021-9991 Google Scholar
Y. Sun, Z. Xia and U. S. Kamilov,
“Efficient and accurate inversion of multiple scattering with deep learning,”
Opt. Express, 26
(11), 14678
–14688
(2018). https://doi.org/10.1364/OE.26.014678 OPEXFF 1094-4087 Google Scholar
S. Li et al.,
“Imaging through glass diffusers using densely connected convolutional networks,”
Optica, 5
(7), 803
–813
(2018). https://doi.org/10.1364/OPTICA.5.000803 Google Scholar
Y. Li, Y. Xue and L. Tian,
“Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media,”
Optica, 5
(10), 1181
–1190
(2018). https://doi.org/10.1364/OPTICA.5.001181 Google Scholar
BiographyWaleed Tahir is a PhD student working under supervision of Lei Tian at the Computational Imaging Systems Lab in the Electrical and Computer Engineering Department at Boston University. He received his BE degree in 2013 from the National University of Sciences and Technology, Pakistan, after which he worked as a design engineer at Burqstream Technologies, Pakistan from 2014 to 2016. His research is geared toward computational imaging and machine learning for biomedical imaging. Ulugbek S. Kamilov is a director of the Computational Imaging Group at Washington University in St. Louis. He received his BSc and MSc degrees in communication systems, and his PhD in electrical engineering from EPFL, Switzerland, in 2008, 2011, and 2015, respectively. From 2015 to 2017, he was a research scientist at Mitsubishi Electric Research Laboratories, Cambridge, Massachusetts, USA. His main area of research is computational imaging, with an emphasis on the mathematical and computational aspects of image reconstruction and analysis. Lei Tian directs the Computational Imaging Systems Lab in the Electrical and Computer Engineering Department at Boston University. He received his PhD in 2013 and MS in 2010 from Massachusetts Institute of Technology. He was a postdoctoral associate in the EECS Department at University of California, Berkeley, from 2013 to 2016. His research focuses on computational microscopy, neurophotonics, imaging in complex media, and machine learning for biomedical microscopy. |