Open Access
Add to BibSonomy
Abstract
We propose a special imaging technique as a low-cost solution to profile hidden surfaces through scattering media. The method exploits the asymmetry property of a pair of identical laser beams in propagation through the scattering medium, where scanning the pointing of the paired laser beams allows for a collection of target samples to reconstruct the surface shape of a hidden object. In application, our new method provides alternative solutions to many real-world problems, such as medical imaging, optical communication, environmental sensing, and underwater surveillance that require dealing with a scattering environment that often obscures direct sight of a target area.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Seeing through scattering media is a well-known challenge in countless research frontiers. For example, medical imaging frequently requires in-situ images of targets through blood or interstitial fluid that are not fully transparent. Autonomous driving through raining or foggy environment requires much increased attention and algorithm complexity to discern road conditions through much impaired visibility, even with the help of head-lights or fog-lights. Many counter-measures have been studied to profile targets behind scattering agents through indirect approaches, such as using acoustics [1], single pixel camera systems [2,3], and frequency modulated laser pulses [4]. The acoustics approaches are generally less affected by light scattering, but have a resolution limit due to the relatively large wavelengths of acoustic waves used in the detection. The single pixel camera approaches use multi-sets of spatial patterns with controllable transparency as filters to derive a reasonable reconstruction of an illuminated target through scattering. They essentially differentiate light transmission through different regions of the scattering media as groups of "light rays", so that the fundamental structure of the target can be revealed. The approach of using laser pulses with randomly modulated frequency seeks a matched frequency pattern in the returned signal to identify a strong reflection from the target’s surface, and the depth information of such surface region can be determined through the response time of a matched return. Collectively, the target can be reconstructed with fine details. Direct optical approaches using Lidars or time of flight (TOF) cameras have also been investigated to retrieve information at certain depth layers [5–8]. These methods reveal both reflection (from the target) and back-scattering (from the media) by imaging at different depth layers and post-processing of the data.
To the best of the authors’ knowledge, most of the proposed methods focus on filtering techniques to get away with the scattering media, while methods of utilizing the scattering itself have not been well investigated. We find it possible to use pattern asymmetries from a pair of identical and co-propagating laser beams to indicate the shape of a hidden surface through the scattering media. In fact, there are well-defined theories and equations for laser beam propagation through scattering media [9,10], through which a quantitative analysis of the two beam approach is feasible. We show through proof-of-concept experiments that such method can be used for target recognition or profiling under non-trivial levels of scatterings, which rely on very few characteristics about the scattering agents. A significant advantage of our proposed technique lies in its low requirement for hardware, where only a laser beam, a camera and a few optical components are necessary to form the fundamental system. In application, it serves as a compact and low-cost solution aboard many dispensable systems for various imaging, surveillance and detection tasks.
The rest of the work is arranged as: Part 2 illustrates the mechanism of the two-beam approach. Part 3 demonstrates a prototype system and proof-of-concept experiments for validations. And Part 4 provides conclusions.
2. Mechanisms of utilizing scattering asymmetry in imaging and target profiling
Without loss of generality, we take the common assumption that single scattering events are dominant in the scattering media [11]. Intuitively, this means that the scattering is non-diffusive, where the major portion of light remains unscattered per unit-length of propagation. And we focus our analysis on the near-axis beam patterns. This applies to many real-world situations such as a foggy channel, or an oceanic channel where weak and progressive side/back scatterings from small particles affect the visibility of distant objects. In the forward scattering direction that preserves most of the optical irradiance of a laser beam, its transverse intensity profile near the optical axis can be analytically derived. In our proposed approach of propagating a pair of identical laser beams with a small transverse separation through the scattering media, their "footprints" patterns upon reaching a surface area of the target contain subtle differences from each other, which can be analytically studied. In other words, based on the well-known near-axis beam scattering equations, our method aims at exploiting their transverse differential orders for surface detection. Due to the proximity between the two identical beams in parallel projection, their mutual scattering processes in a relatively long propagation path get offset (cancel each other) by a differential operation. Shall the illuminated area tilts toward one side of the probing laser, its "footprint" on the surface will be smaller in geometric spreading, but stronger in intensity (brightness) when compared with the "footprint" of the other beam. As a result, the differential part can be used to characterize the regional surface tilt as a target sample with a width comparable to the spacing between the two laser beams. Collectively, the surface profile of the target can be reconstructed by the sub-area samples. When the "footprints" are imaged by a side-view camera that sits slightly away from the central beam paths, the intensity profile of the asymmetry pattern can be computed from the image, and translated into a surface sample. Overall, such implementation looks similar to a photoplethysmography device [12,13], with additional considerations of beam scattering in propagation and the use of geometric patterns in signal/image processing. By adopting Arnush’s scattering analysis, the beam’s intensity distribution near the propagation axis can be expressed as:
In Eq. (1), r and z represent the transverse and on-axis coordinates of the propagating beam, respectively. For convenience, we pick the unscaled form of the scattering analysis, where the super-position of the unscattered (the $1^{st}$ term) and scattered (the $2^{nd}$ term) parts of the beam near the optical axis are respectively expressed. Their relative strength is expressed by the constant g denoting stable distribution of the scattering agents throughout the imaging channel. The on-axis attenuation is expressed through $e^{-\!sz}$, where $sz:=\tau$ represents the summation of optical scattering and absorption effects that is often called "optical thickness" in the literature. Intuitively, $\tau$ can be treated as the depth information when the distribution of scattering agents is invariant. Beam spreading is accounted for with $w_0$ representing the expected beam width in propagation without scattering. Similarly, beam scattering is described by the attenuated beam interacting with an expected $\pi \!R^2$ area of scattering, where $R^2:=\;<\;\!r^2\!\;>\;=\frac {4\pi }{3\xi ^2}s\!z^3$ with $\xi$ representing the central size parameter ($\kappa d_0$) of the scattering agents. Overall, Eq. (1) describes a thin laser beam perturbed by scattering events in its vicinity. As the central part of the beam (the $1^{st}$ term) supplies photons to be scattered and those scattered photons only have trivial influence on the central beam, we use an indication function I apertured by $r_0=w_0/2$ to simplify the first term of the beam model in Eq. (1). Later we show that it is the scattered regions of $r\;>\;r_0$ that contribute to the proposed profiling algorithm. To avoid confusion with true scalar field that observes the conservative law of energy, we use $h(.)$ to denote that our analysis is built based on relative scales.To derive the differential part, additional assumptions are taken: (1) The range of optical thickness for the measured surface area is much less than unity. (2) The side view camera is well focused on the "footprints". And (3) the surface is diffusive. These regulations are normally required to ensure the quasi-linear relationship in the differential terms. Correspondingly, the first order differential term of the two beams’ forward scattering patterns is:
In Fig. 1, the envelopes of the two propagating beams are indicated by the green solid curves and the blue dashed curves, respectively. The gray plate represents the sampled sub-aperture area of the target, which is slightly tilted with angle $\theta$ from the normal incidence of the two parallel beams. The local beam patterns $h_1(r_1,z_1)$ and $h_2(r_2,z_2)$ on the target are expressed with local coordinates referenced on the beam centers with sub-indices of 1 and 2, respectively. The differential term $\Delta \!h(r,z)$ is defined as the direct difference between the two beam patterns, which is $\frac {\partial \,h(r,z)}{\partial \,z}$ multiplied by $\Delta \!z(=l\theta )$ under the quasi-linear assumption.
Two major observations can be made: (1) The magnitude of the asymmetry pattern is proportional to the tilt angle of the surface. And (2) the asymmetry pattern is fixed and can be integrated over $r\;>\;r_0$. It is interesting to note that the differential pattern crosses zero when $\tau\;>\;3$ at radius $r=\sqrt {\tau /3-1}R$. Because the radius information is normally traced in range $r_0\;<\;r\;<\;R$, it is reasonable to point out that the use of the asymmetry pattern is most effective near $\tau =3$, and degrades gradually towards the directions of $\tau =0$ and $\tau =6$. Ideally, 2D integration from $r=r_0$ to $r=\infty$ in the asymmetry pattern yields:
3. Prototype system and experiment
To demonstrate the proposed concept, we have established a lab-scale experiment for proof of concept, which is shown in Fig. 2.
In Fig. 2, we show the implementation of the simple system that uses two mirrors, one beam splitter, one low-power (2 mW) He-Ne laser, two dove prisms, one rotational stage and one imaging camera. For both cost and beam shape quality reasons, we have picked a red He-Ne laser instead of green lasers that can penetrate deeper in underwater imaging tasks. The paths of the two identical laser beams are marked by the red numbers with R denoting the right branch, and L denoting the left branch in forward projection. By using the two dove prisms, we are able to adjust the spacing between the two laser beams at millimeter levels and reduce the ratio of their path length difference (which is path 2R). In our experiments, the spacing is fixed at 10 mm. The rotational stage (Thorlabs: K10CR1) implements precise angular controls that are automated through a computer. A water tank of 75 cm depth is used to hold the scattering media simulated by mixing 50 L water with 15 mL antacid liquid. The antacid liquid [14] is often used in lab experiments to simulate scattering phenomenon in underwater environments. The attenuation fact s in Eq. (1) per unit length of propagation is obtained by measuring beam attenuation across the narrow side of the water tank with and without the murky water. Our measured value through such calibaration is $s=0.020 mm^{-1}$. An off-axis camera that avoids direct reflection from the container glasses and laser back-scattering is focused at a designated target depth to image the footprints of the two laser beams through the scattering media.
To examine the accuracy of the asymmetry method, we have mounted a 50.8 mm (2 inches) diameter sandblasted aluminum disk on an adjustable mirror mount as the target, and put the target at the optical depth of $\tau =4$ (which receives $2\%$ of the transmitted optical power at the end of propagation). The spacing between the two laser beams is set at 10 mm for all the experiments. We first manually adjust the target disk to face the projecting beams at normal incidence. An automated scan is invoked to scan from 0 to 0.8 degrees with incremental step-size of 0.04 degrees. In each scan, an image is taken by the camera to extract the relative angle $\theta$ based on the approximation result in Eq. (3). Because the target disk is fixed in orientation, the extracted angles are expected to increase by 0.04 degrees as the projected beams scan across its surface. We then repeat the same process three times by giving the target disk an incremental angle offset around 0.3 degrees through manual rotations on the mirror knobs inside the "murky" water. The results are expected to show the same linearity on top of the angular offsets.
An image example is shown in Fig. 3.
In Fig. 3, we have labeled the front glass reflections of the two laser beams upon entering the water tank, and their actual "footprints" upon reaching the target in the scattering process, respectively. The randomly scattered dark spots in the image correspond to air bubbles gathered at the glass walls of the water tank, which are typical for underwater imaging. These bubbles add additional fluctuations to the evaluated surface angles in our approach. In applying Eq. (3), because the bright lines (caused by side-scattering) along the forward propagation paths of the two beams could also affect the accuracy of the approach, we avoid these areas by only using the shaded quarter sections indicated by Fig. 4.
By applying the quarter areas indicated in Fig. 4 to the approximation of Eq. (3) in image processing, the result is less affected by both the mutual scattering area between the two footprints, as well as the overlapping areas in the imaged paths of the two laser beams. Correspondingly, the extracted angular values from the scanned images versus the expected values in theory are shown in Fig. 5.
In Fig. 5, as $r_0\ll \,R$ at the target depth of $\tau \!=\!4$ with significant beam spreading and attenuation, it is proper to use the approximation form of Eq. (3), where $\theta$ is proportional to the integral result of the differential pattern. In fact, we can swap the order of the integral summation in Eq. (3) and the differential subtraction in Eq. (2), so that the calculation becomes much simpler (which turns into a subtraction between two pixel summations). The linear factor in Eq. (3)’s approximation is derived through retrofitting between the known range of angular scans and the two corner cases of the integrals in pixel values. For additional comparisons, the expected theory values are plotted by the dashed lines for each set of experiments. A small systematic angle offset of the target disk (around -0.04 degrees) due to the manual alignment has been noted and subtracted from the theory lines. Evidently, the asymmetry approach is reasonably effective in extracting the surface angles under severe scattering conditions. The maximum errors and the root-mean-square (RMS) errors for the different sets of surface tilts are summarized in Table 1.
Table 1. Error Statistics for the Tested Small Surface Tilts
In Table 1, we show both the RMS error and the maximum deviated errors in the angular estimation of each experiment with different surface tilts shown in Fig. 5. For each set of experiments (a row in Table 1), the platform subtracts its laser projection angle from the reconstructed relative angle $\theta$ to obtain an estimation of the surface tilt at different sub-aperture regions that correspond to the projections. It is notable that some projected sub-aperture regions report much larger errors (such as the max errors) that will lead to local distortion of reconstructed surface profile. The errors are largely caused by imaged air bubbles on the glass windows that affect local symmetries in the background images. Overall, as suggested by both Fig. 5 and the RMS errors in Table 1, the surface tilt samples can be extracted with reasonable accuracy. To reconstruct the full surface profile when the scanning angles are equally spaced, one can integrate over the surface tilts (which is the extracted $\theta s$ in subtraction with the beam projecting angles) along the scanning direction. Intuitively, for each extracted data point of $\theta$ in Fig. 5 that is higher than its theory expectation, it tilts the reconstructed surface profile further away from the ground truth, and vice versa. The averaged reconstruction accuracy can be equivalently measured by the RMS error in the above experiments as the scanning angles are equally spaced in our experiments. Given the optical depth of $\tau = 4$ and the scanner-to-surface distance of $z = 2.5 m$, every 0.01 degree of RMS angular error in the experiment corresponds with a 0.4 mm RMS surface error in reconstruction. This further confirms that the accuracy of the method is reasonably good (RMS error < 1.0 mm) when all the presumptions are properly met.
It is of great interest to show an additional experiment to demonstrate the methods’ performance versus surface discontinuities. In principle, the proposed differential theory for extracting surface angles only fits for smooth surface changes, and does not fit for revealing sharp changes in the surface profile. However, the algorithm by its nature will treat sharp changes as large angular tilts, which is a compromise that can be acceptable in interpretation. The tested target is a diffusive rectangular block with a square well cutout (1 cm depth) along its horizontal axis. The target is put at $\tau \approx 2.5$ in the water tank and 20 scan samples are taken from $-0.5$ degrees to $0.5$ degrees, which is shown in Fig. 6 and Visualization 2. For comparison, we have also used a bright green LED array as incoherent light sources to image the same target (with the same camera and viewing angle) and show that the target is invisible through the LED illumination.
Fig. 6. Sampled camera images of (a) the pair of the probing lasers on a target that has surface jumps (see Visualization 2), and (b) imaging result with the the bright green LED array as an illumination source.
The external views of the object detection comparison experiment are shown in Fig. 7.
Fig. 7. (a) Experimental setup to compare imaging effects between the two beam illumination and the bright LED array illumination; (b) External view of the scattering media with the occluded target; (c) Side view of the two beam projections onto the target’s surface.
In Fig. 7(a), it is obvious to see that the LED illumination is much brighter than the laser illumination shown in Fig. 7(c). A view of the object hidden in the scattering media is shown in Fig. 7(b). However, the brighter LED illumination does not help with revealing the hidden object, as demonstrated in Fig. 6(b). This is because the strength of the back-scattered light from the media is proportional to the illumination power, which doesn’t improve the visibility of the target when the obstructing scattering media is significantly illuminated by the LED array. Comparatively, Fig. 7(c) shows the two thin beams projecting through the scattering media, which involves with a much smaller scattering volume and therefore leads to much suppressed back-scattering. Consequently, the scanning mechanism can be employed to collect the surface samples from the asymmetry approach for object reconstruction. The processed result of the surface reconstruction is shown in Fig. 8.
Fig. 8. Surface profile reconstruction result in revealing the well-cutout along the horizontal scanning direction.
In Fig. 8, the left y-axis measures the surface angle extraction that is shown by the blue bars. The reconstructed surface profile and the actual surface profile along the horizontal scanning direction are plotted by the solid orange curve and the dashed black curve, respectively. Both curves for surface profiles are measured by the right y-axis. Evidently, the reconstructed surface profile reflects the overall trend of surface profile changes, yet its accuracy is affected by discontinuities in the structure. In application, one can still apply the asymmetry method without knowledge of the hidden target’s surface condition, given that only low spatial frequencies of the surface profile will be recognized. Therefore, the discontinuity problem doesn’t revert the fundamental function of the asymmetry approach.
4. Conclusions
We have systematically demonstrated the theory and proof-of-concept experiments of using the forward scattering asymmetry properties between a pair of identical probing lasers to profile hidden surfaces through scattering media. Benefited by its simplicity, low hardware demand and effectiveness against harsh visibility conditions, the method provides high potential solutions to countless problems that deal with imaging or optical sensing through heavily scattering media.
Scalability is another advantage of the two beam approach, as reducing the size of the system doesn’t affect its functionality. This is ideal for many small unmanned systems to carry out basic tasks in imaging, detection, or diagnosing. In extension, the two beam’s approach can also be viewed as the most fundamental example in combining structured light with scattering analysis for object detection through complex environment. Our study already shows that such combination is effective in imaging and target profiling through scattering media, and further investigation may provide more powerful solutions.
Funding
Naval Air Warfare Center, Aircraft Division (N004211910002).
Acknowledgement
The authors sincerely thank Dr. Linda Mullen for her insightful suggestions and kind help with this research.
Disclosures
The authors declare no conflicts of interest.
References
1. J. Jose, R. G. H. Willemink, S. Resink, D. Piras, J. C. G. van Hespen, C. H. Slump, W. Steenbergen, T. G. van Leeuwen, and S. Manohar, “Passive element enriched photoacoustic computed tomography (per pact) for simultaneous imaging of acoustic propagation properties and light absorption,” Opt. Express 19(3), 2093–2104 (2011). [CrossRef]
2. Y. Jauregui-Sánchez, P. Clemente, J. Lancis, and E. Tajahuerce, “Single-pixel imaging with fourier filtering: application to vision through scattering media,” Opt. Lett. 44(3), 679–682 (2019). [CrossRef]
3. F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22(21), 26338–26350 (2014). [CrossRef]
4. N. Alem, F. Pellen, G. L. Brun, and B. L. Jeune, “Extra-cavity radiofrequency modulator for a lidar radar designed for underwater target detection,” Appl. Opt. 56(26), 7367–7372 (2017). [CrossRef]
5. S. Jeong, D.-Y. Kim, Y.-R. Lee, W. Choi, and W. Choi, “Iterative optimization of time-gated reflectance for the efficient light energy delivery within scattering media,” Opt. Express 27(8), 10936–10945 (2019). [CrossRef]
6. T. Muraji, K. Tanaka, T. Funatomi, and Y. Mukaigawa, “Depth from phasor distortions in fog,” Opt. Express 27(13), 18858–18868 (2019). [CrossRef]
7. D. Borycki, O. Kholiqov, and V. J. Srinivasan, “Correlation gating quantifies the optical properties of dynamic media in transmission,” Opt. Lett. 43(23), 5881–5884 (2018). [CrossRef]
8. M. Alayed, M. A. Naser, I. Aden-Ali, and M. J. Deen, “Time-resolved diffuse optical tomography system using an accelerated inverse problem solver,” Opt. Express 26(2), 963–979 (2018). [CrossRef]
9. M. I. Mishchenko, J. W. Hovenier, and D. W. Mackowski, “Single scattering by a small volume element,” J. Opt. Soc. Am. A 21(1), 71–87 (2004). [CrossRef]
10. D. Arnush, “Underwater light-beam propagation in the small-angle-scattering approximation,” J. Opt. Soc. Am. 62(9), 1109–1111 (1972). [CrossRef]
11. R. Sanchez and N. J. McCormick, “Analytic beam spread function for ocean optics applications,” Appl. Opt. 41(30), 6276–6288 (2002). [CrossRef]
12. A. Trumpp, P. L. Bauer, S. Rasche, H. Malberg, and S. Zaunseder, “The value of polarization in camera-based photoplethysmography,” Biomed. Opt. Express 8(6), 2822–2834 (2017). [CrossRef]
13. H. Lu, Y. Li, H. Li, L. Yuan, Q. Liu, Y. Sun, and S. Tong, “Single-trial estimation of the cerebral metabolic rate of oxygen with imaging photoplethysmography and laser speckle contrast imaging,” Opt. Lett. 40(7), 1193–1196 (2015). [CrossRef]
14. Y. Gu, C. Carrizo, A. A. Gilerson, P. C. Brady, M. E. Cummings, M. S. Twardowski, J. M. Sullivan, A. I. Ibrahim, and G. W. Kattawar, “Polarimetric imaging and retrieval of target polarization characteristics in underwater environment,” Appl. Opt. 55(3), 626–637 (2016). [CrossRef]
