2.1.

Model

An estimate of the single shot range precision is provided using the model provided by Reinhardt et al.,⁴ which is derived from the timing jitter and timing resolution of the detector. The timing resolution, ${\sigma }_{\mathrm{res}}^2$, is provided by

The timing jitter, ${\sigma }_{\text{jitter}}^2$, is provided by

2.2.

Experimental Methods

The experimental methods are discussed in greater detail here. Three experiments were conducted. One experiment involved collecting data with the goal of correction of the camera. The other two experiments were designed to collect data for statistical and visual validation of the correction process. Both experiments conducted for the statistical validation and the correction of the camera imaged a flat checkerboard target at the same range, imaging the target through four right-angle rotations. However, for the correction process, significantly more data were collected: an attenuator was used to control the beam intensity through seven different illumination levels, adding another layer of data. The seven attenuation levels are displayed along with mean signal and noise, in units of photons, in Table 1.

Table 1

Attenuation, mean signal, and noise for seven illumination levels generated from an attenuator used for the collection of data in this experiment.

The intention was to enable range walk error correction; however, analysis shows that these additional data were unnecessary for accomplishing this task. In fact, the results imply that range walk error and non-uniformity are corrected with a single orientation of the target and a single set of frames, if carefully calibrated and chosen.

Table 2

Equipment list for this research.

The experimental setup, as described in Fig. 2, uses a Voxtel (a subsidiary of Allegro Microsystems) DPSS laser with a 1535-nm central wavelength, 5-ns pulse width, $842\text{-}\mu \mathrm{J}$ pulse energy, and variable pulse repetition frequency (PRF) of up to 10 Hz; this setup used a PRF of 5 Hz, in consideration of long-term pulse stability. This laser was chosen partially because it is designed to ship with the LiDAR imager. Part of this effort was aimed at reducing or removing the need for specialized equipment in calibration and characterization setups of this class of imager. The laser, after exiting a $17\times$ collimating beam expander, has a $400\text{-}\mu \mathrm{rad}$ divergence angle and 6.8-mm beam diameter. The beam travels 20 cm downrange to an RPC Photonics engineered diffuser with a 4.33-deg divergence angle, which top-hats the beam to expand the beam to fill part of the target. A critical limitation of the system is derived from the timing system of the 3D flash LiDAR imager, in which the internal system timing of the imager produces a significant enough delay to inhibit ranging within 5 m of the time-of-flight imager. Due to non-uniformity, this 5-m dead-range was found to effect returns through a range of 9.5 m. An engineered diffuser with a 16.3-deg divergence angle was tested and determined to attenuate the signal through beam expansion too much at the appropriate ranges. Smaller divergence angle diffusers were investigated as possible options, but calculations suggest that, to fully fill the FOV, greater signal is required. The target is a $4\mathrm{ft}\times 4\mathrm{ft}$ ($1.22\mathrm{m}\times 1.22\mathrm{m}$) checkerboard pattern that was printed off a large poster printer and attached to two poster boards with additional structural support backing. Analysis of the data shows that, at 1535 nm, the checkerboard squares have reflectivities of 1, 0.953, 0.929, and 0.656 relative to the brightest possible return. A possible uncontrolled, external source of non-uniformity included the physical printing of the pattern onto a paper material, which is imperfect and may introduce streaking and other print artifacts into the target itself. The scene was imaged using an Edmund Optics 50-mm focal length telephoto lens with the camera operating in a full frame mode of operation, contrary to prior work by this team in which a windowed region of interest approach was used (Table 2).^4,14

Fig. 2

The experimental setup (a), in which a 1535-nm laser illuminates a target downrange after passing through an attenuator (b). This attenuator consists of a half-wave plate, motorized rotation stage, beam dump, 50/50 polarization beam splitter; the beam, upon exiting, passes through an engineered diffuser that top-hats the beam while expanding the beam at a uniform 4.33-deg divergence angle. This fills approximately 20% of the FOV at a range of 11.78 m. The 3D flash LiDAR images the checkerboard target using a telephoto lens with a 50-mm focal length.

The flat checkerboard target (Fig. 3) was illuminated 11.82 m downrange from the camera or 11.73 m downrange from the laser; the mean range was 11.78 m (Fig. 4). The distance between the camera and the laser was 23 cm, and thus the bistatic angle was provided by ${\theta }_{\text{bistatic}}=90\mathrm{deg}-{\tan }^{-1}(\frac{z_{\text{camera}}}{z_{\text{bistatic}}})$, such that ${\theta }_{\text{bistatic}}=1.12\mathrm{deg}$. The camera FOV, ${\theta }_{\mathrm{FOV}}$, was provided by, ${\theta }_{\mathrm{FOV}}=2{\tan }^{-1}(\frac{h}{2f})$, such that ${\theta }_{\mathrm{FOV}}=6.15\mathrm{deg}$. At 11.82 m, the FOV filled a square defined by $h_{\mathrm{FOV}}=2z_{tgt}\tan (\frac{{\theta }_{\mathrm{FOV}}}{2})$ or 1.27 m to a side. The laser passed through an RCP Photonics engineered diffuser, with a divergence angle of 4.33 deg, located 11.53 m from the target, resulting in a beam that was 87.2 cm in diameter; thus the beam should, at most, encompass 36.96% of the target area in the FOV.

Fig. 3

The target used was a checkerboard target with $8\times 8$ squares across the array; each square was originally designed to match a $16\times 16$ region of detectors in the FOV at a range of 8 m; however, upon optimizing the return, the range was pushed back to 11.82 m from the camera, with each square matching $11\times 11$ detectors. Note that any discrepancy may be caused by adjustments in the position of the engineered diffuser, which acts to top-hat the beam and diverges the top-hatted beam at an angle of 4.33 deg, thus requiring realignment from the original intended position at 8 m. This target was physically rotated by 90 deg after the required samples were collected at each position.

Fig. 4

The flat checkerboard target was illuminated 11.82 m downrange from the camera or 11.73 m downrange from the laser; the mean range was 11.78 m. The distance between the camera and the laser was 23 cm, and thus the bistatic angle was ${\theta }_{\text{bistatic}}=1.12\mathrm{deg}$. The laser passes through an RCP Photonics engineered diffuser, with a divergence angle of 4.33 deg and located 11.53 m from the target, resulting in a beam that is 87.2 cm in diameter; thus the beam should, at most, encompass 36.96% of the target area in the FOV.

Thus, data were collected for the target at nine positions to ensure full, overlapping coverage of the entire FOV; each square on the checkerboard pattern covered approximately $11\times 11$ detectors, with the illumination filling an average of 31.5% of the FOV. This procedure was performed for both the data collected to be processed into a correction and the data collected for statistical validation purposes. For correction purposes, the illumination was varied using an attenuator; this attenuator, pictured in Fig. 2(b), was constructed out of a Zaber motorized rotation stage housing a half-wave plate, a 50/50 polarization beam splitter, and a beam dump. As the stage rotated from the fast axis to the slow axis, the linear polarization of the beam also rotated, incrementally changing the amount of light transmitted versus redirected to the beam dump; thus, by rotating the half-wave plate, incremental control of beam power was achieved. This incremental attenuation was used to capture the data at seven levels of illumination (Table 3).

Table 3

The area of the FPA that is illuminated is 28.1% on average; typical reasons for variations in the area include illuminating the edge or a corner versus the center of the array. A built-in function in MATLAB used to outline circles was used to determine the radii and centroids of the illuminated area for each position; subsequent compensation for overfilling the FPA allowed for estimation of the area illuminated relative to the full FPA.

The data for validation purposes were collected using a near-identical setup and via methods identical to that used for collecting data for the corrections. The primary differences involved not varying the intensity of the incident illumination and any statistical uncertainties associated with realigning the target and repositioning the beam. While the correction data required significant variations of the windowing used to crop the frame before stitching together a final, full set of frames to be used for processing the corrections (Fig. 5), the validation data were able to use generally more uniform windowing, as seen in Fig. 6.

Fig. 5

The nine separate data captures are displayed (a)–(i), along with the cropped region of interest for the correction frames. The region window was chosen to minimize noise in the return; critical to this selection process were the frames collected in (e) and (f), which were closer to saturation in the intensity return compared with the other frames, possibly due to the larger area of the FOV being illuminated, 35.68% versus 30.32% for all other frames exclusive of these two, which thus required careful selection of a minimized window for both.

Fig. 6

The nine separate data captures are displayed (a)–(i), along with the cropped region of interest for the validation frames. Unlike with the data collected for corrections, the region window was chosen without regard to noise in the return.

Previously collected 3D imagery was also included for visual validation.¹⁷ This imagery displays a scene with four boxes in a similar location and range as the target; the boxes were at a range between 11.5 and 12.5 m and in the same line of sight and platform as the target (Fig. 7).

Fig. 7

Photograph of scene used for validation purposes: four boxes between 11.5 and 12.5 m range (a). The leftmost box in the foreground has a different reflectivity than the others. Photograph of scene used for calibration purposes (b). The target is at 11.78 m.

2.3.

3D Image Correction

A series of image filters and masking operations are required to create the PRNUC from the raw data. The raw frame is corrected for thermally compensated dark-frame non-uniformities. The data collected for this thermal compensation were used in two prior works by this team, and the experimental process for thermal compensation of dark-frame NUC has been partially described.^16,17 However, the thermal compensation algorithm used for this paper presents a unique approach to the problem of using an interpolated lookup table to compensate for thermal drift by enabling this approach without any active knowledge of the in-situ sensor temperature. This correction algorithm is somewhat limited in capability as it requires the scene to optimize toward being structurally flat. This is accomplishable by the data containing enough samples within the frame at the dark level that are assured to trend toward optimal correction at the correct temperature. By comparing the trend of the spatial standard deviation as the raw frames are corrected using the TC-DFNUC, an estimation of the optimum compensation is achieved, as is an estimation of the temperature of the sensor. The spatial standard deviation, $s_{xy}$, is described as

Fig. 8

Trend line of global range precision with DFNUC applied at different interpolated temperatures.

By interpolating the returns in range and intensity, per detector, using a smoothing spline across the six temperature datasets collected, an estimation of the dark-frame non-uniformity was achieved. To correct a set of frames, dark-frame subtraction was performed for the mean frame using all interpolated thermal compensation frames. The frame with the lowest value of the spatial standard deviation, $s_{xy}$, was indexed and provided an estimation of the temperature and the value for the TC-DFNUC.

Two masks were generated; ${\mathcal{M}}_1$ to compensate for dark returns,

Fig. 9

Misalignment between rotations displays as additional noise on the FPA. In this case, the intensity return is displayed from 0 to 300 photons.

Values that met the filtering criteria for creating the masks were assigned a value of 1, whereas all other values were assigned a non-numerical value in MATLAB. These two masks were subsequently used to create a total mask, $\mathcal{M}={\mathcal{M}}_1{\mathcal{M}}_2$, such that any given photo-response frame, $P$, was transformed by $\mathcal{M}$ as $P^{\prime }=\mathcal{M}P$. Further analysis required filtering out of non-numerical values in the process of calculating statistical quantities such as standard deviation and mean.

Once the photo-response frame was multiplied by the mask, the frames were prepared for image filtering; this process required the non-numerical values for the frame to be transformed to the mean value for the respective frame. Also, to reduce noise, the mean value across the frame was substituted for any return below 2300 photons for the intensity return; for the range return, any return above 45 m was substituted with the mean value for the respective frame. This frame, $Y_{xy}$, was then processed with a Gaussian image filter with ${\sigma }_{xy}=0.65$ to produce a smoothed image following variations in intensity or range, without underlying structure, such as that which would be found in gain errors. The ratio of the Gaussian blurred image and the filtered image provided an estimation of the gain errors for the FPA:

Range walk error characterization was accomplished by vectorizing and sorting the data across the thermally compensated dark-frame non-uniformity corrected frame. This provided a global characterization for range walk error encompassing both range walk error and photo-response non-uniformities. A characterization curve was generated by using linear least squares to optimize the vectorized and sorted DFNUC corrected frame in intensity to the corresponding vectorized and sorted frame computed for the absolute error for range relative to the measured true range of 11.775 m.

3.1.

Results

The average of 500 frames for the range return of a single orientation of the validation target is displayed. The range return is displayed as uncorrected [Fig. 10(a)] and corrected [Fig. 10(b) and 10(c)]. Both the uncorrected and corrected averaged frames are displayed from 0 to 40 m, while the corrected averaged frame is also displayed from 5 to 15 m [Fig. 10(c)].

Fig. 10

Range return of a single rotation of the checkerboard target before (a) and after (b) non-uniformity and range walk error corrections. Both the uncorrected and corrected averaged frames are displayed from 0 to 40 m, while the corrected averaged frame is also displayed from 5 to 15 m (c).

The average of 500 frames for the median return of all four rotations of the validation target data is displayed in Figs. 11 and 12. The intensity return is displayed as uncorrected [Fig. 11(a)] and corrected [Fig. 11(b)] and from 2048 to 4096 photons.

Fig. 11

Intensity return, averaged over the four right-angle rotations of the checkerboard target, before (a) and after (b) NUC.

Fig. 12

Range return, averaged over the four right-angle rotations of the checkerboard target, before (a) and after (b) non-uniformity and range walk error corrections. Both the uncorrected and corrected averaged frames are displayed from 0 to 40 m, while the corrected averaged frame is also displayed from 5 to 15 m (c).

The SNR for the intensity return is given in Table 4. The SNR, in dB, is calculated by

Table 4

SNR, intensity return.

The range return for the median of the set of orientations is displayed in Fig. 12. The range return is displayed as uncorrected [Fig. 12(a)] and corrected [Figs. 12(b) and 12(c)]. Both the uncorrected and corrected averaged frames are displayed from 0 to 40 m, while the corrected averaged frame is also displayed from 5 to 15 m [Fig. 10(c)].

The range accuracy for the data shown in Figs. 12 and 13 is displayed in Table 5. The range accuracy, $s_{A,xy}(R_{xy})$ is calculated as

Fig. 13

Histograms of validation target, displaying marked improvement in range accuracy and precision between uncorrected scene and when NUC and range walk error correction are applied.

Table 5

Region of interest method, range accuracy.

The range precision is calculated as the spatial standard deviation of the range return, $s_{xy}(R_{xy})$, provided by Eq. (5), $s_{xy}^2(R_{xy})=\frac{1}{MN}\sum _{i=1}^M\sum _{j=1}^N{(R_{ij}-{\mu }_{xy}(R_{xy}))}^2$ (Table 6).

Table 6

Region of interest method, single shot range precision.

A set of 20 frames was averaged from a 3D scene of boxes, with the results provided here. The average of these 20 frames of the uncorrected frame in intensity is shown in Fig. 14(a), in which the averaged frame corrected of dark-frame and photo-response non-uniformity is provided in Fig. 14(b). Both sets of frames are displayed from 2048 to 4096 photons.

Fig. 14

Intensity return for the PIN 3D flash LiDAR camera, viewing box scene at a median range of 11.75 m; the uncorrected intensity return is displayed in photons in (a), and the intensity return fully corrected of non-uniformities is displayed in (b).

For the same set of data collected from the 3D scene of boxes, the average of the 20 frames of the range return is displayed as both uncorrected of non-uniformity and range walk error [Fig. 15(a)] and corrected of these errors [Fig. 15(b) and 15(c)]. Both the uncorrected and corrected averaged frames are displayed from 0 to 40 m, while the corrected averaged frame is also displayed from 5 to 15 m [Figs. 15(c) and 16].

Fig. 15

Range return for the PIN 3D flash LiDAR camera, viewing box scene at a median range of 11.75 m; the uncorrected range is displayed in (a), as noted with significant range walk error and non-uniformity, and the corrected range is displayed in (b). Both the uncorrected and corrected averaged frames are displayed from 0 to 40 m, while the corrected averaged frame is also displayed from 5 to 15 m (c).

Fig. 16

Histogram of 3D scene with boxes, displaying marked improvement in range accuracy and precision between uncorrected scene and when NUC and range walk error correction are applied.

The range walk was characterized by plotting the range error relative to the true value of 11.78 m as this curve (Fig. 17) trends while intensity increases, from a minimum of 688 photons up to a maximum return of 4096 photons.

Fig. 17

A global characterization of range walk error is shown. The range walk error is nonlinear, insofar as the error has a linear trend until about 2500 photons in the intensity return, when the trend begins to increase in an exponential manner. This is only curtailed by saturation, approaching 4096 photons. Thus, for example, a return with an intensity of 2048 photons hase an associated range walk error of 39 cm, whereas an intensity of 3072 photons has an associated range walk error of 471 cm.

3.2.

Discussion

Range walk error and non-uniformity were corrected for a InGaAs PIN diode 3D flash LiDAR imager; the methods used can be extended to linear-mode avalanche photodiode (LMAPD) 3D flash LiDAR imagers. The threshold for detection was found to be 688 photons. SNR was found to improve by 15.89 to 49.02 dB upon implementing corrections on the intensity return (Table 4). The range precision was found to be 8.04 cm; this matched the single shot range precision computed from a model of 7.36 cm to within 9.24% error relative to the modeled range precision (Table 6). Range accuracy was decreased from a value of 84 cm to less than 2 cm when applying range walk error correction (Table 5). The results for range precision and, in particular, range accuracy suggest that the range walk error correction was successful in significantly improving 3D image quality. This was achieved by sampling the full array and fitting the sorted, myriad data using a linear least squares algorithm.

Limitations of this method included inconsistent patches of dark returns in the data, noisy data, and positional inconsistencies between target orientations which lead to higher than anticipated frame-to-frame jitter. Mitigating all of these issues required extensive image processing and filtering, but additional design considerations have the potential to mitigate them as well. Whether there is a need to capture multiple orientations of the scene or not is of interest for further work, as the method used to correct the range walk and non-uniformity would indicate that neither multiple orientations nor multiple intensity levels are of particular value for this method. This would seem to indicate that, with some modifications of the experimental setup, a single set of frames can be used with the goal of fully correcting a PIN 3D flash LiDAR imager.