1.

INTRODUCTION

Dynamic imaging through scattering media in the single-scattering regime under natural light is a big challenge in many applications – Some of these real world applications include a car identifying the rear-end of the front vehicle hidden in dense fog, fire fighters seeking survivors inside a smoky fire, fighter pilots detecting enemy crafts in clouds, and medical doctors observing biological cells through tissues, to name a few. To overcome this challenge, various methods have been proposed including non-invasive imaging [1], ghost imaging [2, 3] and wavefront shaping [4-9]. Their results have not been satisfactory even though active light sources and scanning mechanisms were used. This is due to the fact that these techniques do not directly detect ballistic photons emanating from objects through scattering media in the single- scattering regime, but only estimate and remove scattering media. In this paper, we propose a novel dynamic imaging technique through scattering media in the single-scattering regime by directly detecting ballistic photons emanating from objects based on statistical optics [10-13] and then show experimental results to support the proposed method. We have named the proposed technique “Peplography.” The word peplography comes from the Greek words πέπλο (péplo; “veiled”) and γραφής (grafís; “writing”). The peplography plays the role in directly detecting ballistic photons associated with the objects from a single peplogram (“veiled image”) based on statistical optics, and reconstructing the three- dimensional (3-D) peplogram using integral imaging [14-18]. Since the peplography is a novel dynamic imaging, it is expected to be an alternative solution to resolve existing problems in dynamic imaging in scattering media.

Recently a method based on the passive light source and image processing technique has been reported [18]. In this method, many different image processing algorithms such as gamma correction, histogram stretching, and histogram specification are used, thus artificial color pixels may be generated. In addition, the reconstructed image may not be reliable and it cannot be implemented in real-time because it uses many different image processing algorithms. On the other hand, the proposed peplography solves the artificial color problem occurred in the previous work [18] and it can have the reliability and physical validity of the reconstructed image because it can directly detect the ballistic photons from the scattering media by using the mathematical model of photon statistics imaging without any uses of image processing techniques. Also, it can be implemented in real-time since it has only two processes such as scattering media estimation and ballistic photon detection processes. Therefore, the proposed peplography can be used to various application fields with much enhanced reliability and physical validity being implemented in real-time.

2.

THEORY

2.1

Concept of peplography

Figure 1 shows a concept of the proposed peplography. The intensity and phase of the light in scattering media are random, assuming that a tiny part of the scattered light is generated by a single ballistic ray through scattering media. The light passing through the scattering media is composed of different parts of the scattered light, which are spatially superimposed and statistically independent as seen in Fig. 1. In the peplography, objects are recorded by a conventional image sensor. The recorded image is referred to as a peplogram. Using a statistical estimation, the scattering media is estimated and removed. The processed peplogram, as shown in Fig. 1, has only a few ballistic photons. To detect these ballistic photons, photon counting detection based on a Poisson distribution can be applied. Finally, a reconstructed peplogram as depicted in Fig. 1 can be obtained. In addition, a 3-D reconstructed peplogram with improved visual quality can be achieved by 3-D passive imaging techniques such as integral imaging.

Figure 1.

Concept of peplography.

2.2

Estimation of scattering media by statistical estimator

In an attempt to remove scattering media from a single peplogram, a statistical estimation is considered. A scattering medium is modeled using the Central Limit Theorem, consisting of constituent parts of the scattered light [11, 18]. A part with dimensions of is assumed to act as Gaussian random variables. When a single peplogram I_p is captured with the conventional imaging device such as a charge-coupled device (CCD) having a resolution of N_x×N_y pixels, the corresponding part is expressed by S_ij(u, v) = I_p(i+u, j+v), where i=0,1,…,N_x-w_x, j=0,1,…,N_y-w_y, u=1,2,…,w_x, and v=1,2,…,w_y. (N_x-w_x+1)×(N_y-w_y+1) parts of the scattered light, which are statistically independent, are obtained. We can estimate the scattering media (in fact, we try to find the unknown parameter which is the sample mean of the Gaussian distribution) using the maximum-likelihood estimation (MLE) [11, 18]. Finally, the estimated parameter (the part of scattering media) is the sample mean of each part of the scattered light:

The peplogram after removing the scattering media is expressed as the following equations:

Figure 2a shows an object image in clear water, serving as a reference image. Toy cars, such as a model of an ambulance and a Hummer, are used as test 3-D objects. The peplogram in turbid water is shown in Fig. 2b, where the volume of water is 38 liters and the volume of milk is 14 milliliters. In addition, the transmitted light intensity is described as I(z) = I₀exp(-kz) by the Beer-Lambert-Bouguer law [13], where I₀ and k represent the light intensity before scattering and the total attenuation coefficient depending on illumination wavelength and media, respectively, and z denotes the thickness of the homogeneous media. If the scattering media are inhomogeneous, k depends on the position. Peplography can be applied to both homogeneous and inhomogeneous media. Using the MLE, the estimated scattering media of turbid water as shown in Fig. 2c is obtained [11], where w_x=w_y=200.

Figure 2.

Procedure of estimating and removing the scattering media from a single peplogram. (a) Original image, (b) single peplogram, (c) estimated scattering media, and (d) removed scattering media from a single peplogram.

2.3

Detection of ballistic photons by statistical optics

The peplogram after removing scattering media, as shown in Fig. 2d, is generated via Eqs. (1) and (2). It should be remarked that the processed peplogram seems to be the recorded image under photon-starved conditions, because only a few ballistic photons may survive through the scattering media. We can directly detect these ballistic photons by means of the photon-counting concept. A photon-counting detector is mathematically modeled by Poisson distribution [12], taking into account the characteristic of the distribution is useful for events that occur rarely in a unit time and space. To detect the ballistic photons from the processed peplogram, as shown in Fig. 2d, the normalized irradiance peplogram by is used, when the total energy must be unit energy for the detection of the ballistic photons. A photon-counting detector model is expressed as follows [12]:

where N_p, c, and γ_c indicate the expected number of the ballistic photons from the processed peplogram, the index of color channels (R, G, and B), and the coefficient of the ballistic photons for each color channel, respectively.

The total energy conveyed by the ballistic photons from the 3-D scene behind scattering media is proportional to the wavelength of illumination [12]. For each basic color channel (i.e., R, G, and B), different coefficients should be applied to the normalized irradiance peplogram Ĩ_p (i, j). The coefficients of the ballistic photons γ_c with R, G, and B channels are set to be 1.4497, 1.1270, and 1, respectively, because we set B channel as the reference and the average wavelengths of each color channel are 685nm (red: 620~750nm), 532.5nm (green: 495~570nm), and 472.5nm (blue: 450~495nm), respectively.

3.

EXPERIMENTAL RESULTS

Using Eq. (3), where the expected number of the ballistic photons N_p=200,000 is set for each color channel, we can reconstruct the peplogram as shown in Fig. 3a. The quality has been substantially improved except for some noise, considering we can recognize the object from the peplogram in turbid water as shown in Fig 2b. However, it is only a two-dimensional (2-D) image with no depth information and its visual quality is yet unsatisfactory. To achieve 3-D peplogram with enhanced visual quality, we introduce a passive 3-D imaging technique, the integral imaging [14, 17], which is used to reconstruct 3-D images by recording and displaying multiple 2-D images with different perspectives through a lenslet array, where the 2-D images are referred to as elemental images. In addition, synthetic aperture integral imaging technique16 as shown in Fig. 3b is used to record elemental images with a high resolution. The focal length of the camera lens is 50mm, the camera has 3,008(H)× 2,000(V) pixels, and the distance between the cameras is 2mm. With the integral imaging method, 10(H)×10(V) elemental peplograms are captured, from which 3-D object images are finally reconstructed at various planes as shown in Figs. 3c and 3d, through computational volumetric reconstruction of integral imaging [15]. This technique is used to project elemental images through a virtual pinhole array on the desired reconstruction plane (depth) and superimpose all of the elemental images. For various reconstruction planes, the overlapped areas are different from each other [15]. As shown in Figs. 3c and 3d, the noise of the reconstructed peplogram is observed to spread out due to the average effect (low-pass filtering) of the computational reconstruction [15]. To obtain the depth information of the scene through the scattering medium, we implement the computational reconstruction at various depths.

Figure 3.

Experimental results for demonstrating the peplography. (a) Reconstructed peplogram with 200,000 ballistic photons, (b) experimental setup, (c) 3-D reconstructed peplogram at z=145mm, and (d) 3-D reconstructed peplogram at z=155mm.

Figure 4 shows the experimental results for the computational reconstruction of a 3-D scene through scattering media in various applications. Figure 4a shows experimental results for cars in turbid water including a single peplogram, a 3-D reconstructed peplogram, and its reference image. We can confirm that even small bubbles are observed despite the object is immersed in turbid water. Figure 4b shows reconstruction results for a wrecked ship in turbid water. In reconstructed image, we can observe small bubbles. We embodied peplography for a real world situation (e.g., foggy weather) as shown in Fig. 4c. The building in dense fog is visualized by a 2-D reconstructed peplogram. Finally, to validate our 3-D peplography, we calculate the mean square error (MSE) for 2-D and 3-D reconstructed peplograms. MSE of 2-D reconstructed peplogram, as shown in Fig. 3a, is approximately 0.0876, whereas MSE of 3-D reconstructed peplogram, as shown in Fig. 3c and Fig. 3d, are about 0.0220 and 0.0218, respectively. As a result, the visual quality of 3-D reconstructed peplogram is four times better than that of a 2-D reconstructed peplogram.

Figure 4.

Experimental results for various applications. (a) Cars in turbid water, 3-D reconstructed peplogram, and reference image. (b) Wrecked ship in turbid water, 3-D reconstructed peplogram, and reference image. (c) Scene in foggy weather, 2- D reconstructed peplogram, and reference image.

4.

DISCUSSION

In this letter, ‘Peplography’ has been proposed as an efficient dynamic imaging technique through strongly scattering media, and its operational performance has been experimentally demonstrated. It can record and reconstruct peplograms by directly detecting ballistic photons based on statistical optics. Since the key idea of the proposed technique is concerned about a direct detection of ballistic photons from the peplogram based on passive light source and no scanning mechanisms, it can implement a dynamic imaging through strongly scattering media. Human life is believed to be better with the help of the proposed technique.