Open Access
Add to BibSonomy
Abstract
The image quality of ghost imaging (“GI”) is degraded by noise such as ambient light. In this experiment, we evaluated the robustness to periodic noise of normal GI using random patterns and that of GI using inverse patterns (“IPGI”) by simulation and experiment. The results confirmed that increasing the number of illuminated patterns per noise period improved the robustness of IPGI to periodic noise. On the other hand, with a high signal-to-noise ratio, the GI system reconstructed better images than IPGI.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Corrections
7 May 2021: A typographical correction was made to Ref. 26.
1. Introduction
Ghost imaging is an imaging technique that computationally reconstructs an image of a target object by correlating the illumination pattern of light with a spatially randomly distributed intensity and the light returning from the target. Initially, GI was recognized as a quantum phenomenon. Imaging with quantum correlation using quantum-entangled light [1,2] was later suggested to be possible by intensity correlation imaging with classical light [3,4], which interested many researchers.
In classical GI [5–7], speckle patterns created by laser projection of rotating frosted glass are illuminated. A beam splitter is used to split the illuminated light, and two-dimensional information on the intensity of the illuminated light is obtained using a CCD or CMOS. Currently, a method called computational ghost imaging (CGI) [8–10], in which the computer-generated patterns are illuminated in space using programmable spatial light modulators such as the digital micromirror devices, is being widely studied. This technique provides much greater flexibility in the design of the illuminated patterns than the conventional GI with speckle patterns. Using this advantage, various studies have sought to optimize the pattern shape for the target [11] and to improve the reconstructed image quality by controlling the projection area in units of pattern pixels [12,13]. The use of spatial light modulators has also enabled the use of regular patterns, such as Hadamard patterns [14]. The design of illuminated patterns is one of the factors that improve the quality of reconstructed images for GI.
The formulas used for correlation calculations have also been studied in GI as a factor for improving the quality of the reconstructed images. DGI [15–17] and NGI [18] are ways to improve the contrast of the reconstructed images by tuning correlation formulas. There has also been an attempt to compensate the temporal fluctuations of the dynamic target within the correlation formula as seen with gradual GI [19]. In recent years, the use of deep learning [20,21] has attracted much attention.
The applications of GI being studied include remote sensing such as LiDAR [22–24], medical imaging [25], and microscopy [26]. In most of these applications, GI cannot avoid the effects of scattering by the transmission medium and noise from ambient light. The effects of scattering have been studied and it has been shown that GI may be more resistant to scattering than conventional techniques [16,27–29]. Studies have also been conducted to reduce the effects of scattering on the reconstructed images by using Wiener deconvolution [30] and deep learning [31,32]. On the other hand, noise caused by ambient light degrades the GI image [24]. This degradation can be suppressed by increasing the number of illuminated patterns, but this increases the time to reconstruct the image and hence doing so without limit is not viable for practical purposes. To commercialize GI, it is necessary to improve the robustness to noise and to reconstruct the image with a small number of illuminated patterns. Although the use of inverse patterns [33] and other techniques have been considered to address this problem, the robustness of GI to noise is not well understood. The effect of noise due to ambient light is a serious issue that is hindering the commercialization of GI.
In this paper, we quantitatively study the robustness of GI using random patterns and GI using inverse patterns to periodic noise through simulations and experiments. We first simulate how the two types of GI behave in the presence of simple sinusoidal noise under ambient light and then perform experiments using a high-speed projector under actual ambient light.
2. Methods
2.1 Methods
In this paper, we use Eq. (1) to reconstruct a GI image with a random pattern, where G(x,y) is the reconstructed image, br is the r-th reflected intensity, < br > is the average of the reflected intensity sequences, and Ir(x,y) is the r-th projection pattern.
Next, inverse pattern ghost imaging (“IPGI”) with an inverse pattern was performed as follows. As shown in Fig. 1, IPGI consists of projecting a random pattern and then repeating the black-and-white reversed pattern. The image is reconstructed by calculating Eq. (2) from the reflected intensity br and the illuminated pattern Ir(x,y).
2.2 Simulation methods
Of the GI and IPGI described above, a numerical simulation was performed in Python on a computer (Intel Core i7-8550U processor, 8.00 GB of RAM). The projection pattern used was a 32×32 pixel-pattern created by using random numbers on a computer and designed so that the ratio of white pixels to black pixels was statistically equal. To evaluate the robustness of GI and IPGI to noise, we added a sinusoidal wave as a periodic noise to the reflected intensity br. The reconstructed image was evaluated by changing the standard deviation (SD) of the sinusoidal wave and the number of illuminated patterns per period. As an evaluation index for the reconstructed image, mean squared error (MSE) in Eq. (3) is used, where Ir(x,y) is the reconstructed image and K(i,j) is the target image.
2.3 Experimental methods
A high-speed DLP projector (STAR-07, ViALUX GmbH) was used to reconstruct GI and IPGI images in a noisy environment. Figure 2 shows the basic configuration of the system. We did not use a special noise source, assuming room lighting on the ceiling as a noise source, which was a fluorescent light with a flicker rate of 120 Hz. Figure 3 shows the photodiode output intensity measured at 20 k samplings/s in the absence of pattern projection, which fluctuates periodically at 120 Hz, indicating that the noise from the fluorescent light is dominant. This noise contains about 20% detector shot noise with a standard deviation ratio. To this periodic noise, we set the pattern projection frame rate of the projector to 1 kHz and 20 kHz. 20 kHz is one of the fastest frame rates for currently available DLP projectors. To change the relative magnitude of the noise, we changed the LED luminance of the projector. This means that the lower the LED brightness, the higher the relative noise, and the worse the signal-to-noise ratio. The projection pattern was the same as the one used in the simulation.
3. Results and discussion
3.1 Simulation results
Figure 4 shows the reconstructed GI and IGI images and MSEs for 1,000 illuminated patterns. The standard deviation (SD) of the noise became larger further to the right, and the number of patterns per noise period became larger further downward. SD = 0 means no noise.
With SD = 100 or less, the GI reconstructed the target regardless of the number of illuminated patterns per noise period. However, at SD = 1,000 and SD = 10,000, GI could not reconstruct the target. This suggests that the reconstructability of GI targets depends on the magnitude of the noise, regardless of the number of patterns per noise period.
With 10 illuminated patterns per noise period, IPGI reconstructed the target at SD = 100 or less. As the number of illuminated patterns per noise period increases to 100 and to 1000, the conditions under which the targets can be reconstructed also tend to increase to SD = 1,000 or less and to SD = 10,000 or less. This trend is not seen in the GI with random patterns. This result indicates that there are some conditions under which the target can be reconstructed in IPGI but not in GI.
When we compare the conditions under which the target can be reconstructed by GI, i.e., the reconstructed images of GI and IPGI with SD = 100 or less, the MSE of the reconstructed images is better in GI than in IPGI. When the signal-to-noise ratio is good, GI may perform better than IPGI.
Figure 5 shows the images reconstructed from 10,000 illuminated patterns, which confirm the same tendency as those from the 1,000 illuminated patterns. Clearly, this trend is common regardless of the number of illuminated patterns.
Figure 6 shows the MSE for the number of illuminated patterns at SD = 1,000. Assuming that the guideline value for image reconstruction is MSE 0.15, the GI (1000 patterns / noise period) reconstructed image reaches this value after 50,000 illuminated patterns. On the other hand, IPGI (1000 patterns / noise period) requires only 5,000 illuminated patterns. At the same pattern frame rate, IPGI requires only 1/10th of the GI illuminated time.
3.2 Experimental results
Figure 7 shows the results of experiments in reconstructing images in GI and IPGI from 1,000 illuminated patterns, and Fig. 8 shows those from 10,000 illuminated patterns. In the GI reconstructed image of Fig. 7, the target was reconstructed up to LED brightness 50%, but the images were degraded as the LED brightness went down to 10% or less and the signal-to-noise ratio worsened. In the IPGI, the images were degraded, at a pattern frame rate of 1 kHz, as the LED brightness went down to 10% or less as in the case of GI. On the other hand, at a pattern frame rate of 20 kHz, there was almost no image degradation even when the LED brightness went down to 10% or less. The images of Fig. 8 reconstructed from 10,000 illuminated patterns show a similar tendency. In Fig. 8, the contrast of the GI images seemed greater than that of the IPGI images when the LED brightness was 50% or more. This difference is clearer than that of 1,000 illuminated patterns.
The conversions of the signal-to-noise ratios for the simulations and experiments are shown in Table 1 and 2. LED Brightness 5% is the condition with the worst signal-to-noise ratio in the experiment, which is between SD = 100 and SD = 1,000 for the simulation condition. Considering the simulation results, robustness can be expected even under the worse experimental conditions.
Table 1. Equivalent SNR in Simulation
Table 2. Equivalent SNR in Experimental
3.3 Discussion
When the number of illuminated patterns is fixed, the number of random patterns that can be illuminated in IPGI is half that in GI. This is because a set of pairs (front and back) of patterns has to be illuminated. In fact, when projecting 1,000 patterns in GI, it was possible to project 1,000 types of random patterns, whereas in IPGI, only 500 types could be illuminated, half that in GI. Therefore, in the simulations and experiments, GI was found to produce better reconstructed images than IPGI under zero or low noise conditions. In particular, when projecting 10,000 patterns, the difference in the number of random patterns illuminated was as large as 5,000, and this difference was more pronounced than when projecting 1,000 patterns.
When noise σ is added to the reflected intensity br of IPGI, the image reconstruction equation can be summarized as Eq. (4). Here, σr is the noise at the r-th pattern projection. This equation demonstrates that the magnitude of the noise component is determined by the pattern and the noise, independently from the target and the reflected intensity. Further, the smaller the difference between σ2r-1 and σ2r, the smaller the noise component becomes. In particular, when σ2r-1 = σ2r, the noise component becomes zero and the image degradation due to noise is eliminated. Figure 9 shows the standard deviation of the difference between σ2r-1 and σ2r during the simulation. For a noise σ of any magnitude, the standard deviation of the difference between σ2r-1 and σ2r becomes smaller as the number of illuminated patterns per period increases. In particular, the figure shows that the standard deviation of the difference between σ2r-1 and σ2r remained at around 30 when the target image was reconstructed, but when the standard deviation increased by one order of magnitude to around 300, the image was not reconstructed. This suggests that the degradation of the image was suppressed as the number of illuminated patterns per noise period increased and the noise component became smaller.
4. Conclusions
We studied the robustness to periodic noise of GI with random patterns and GI with inverse patterns (IPGI) by simulations and experiments. The results confirmed that IPGI enhances robustness to periodic noise by increasing the number of illuminated patterns per period. This was not the case for GI. As a result of the improved robustness, we show that IPGI can reconstruct good images with 1/10th the number of patterns compared to GI under certain worse signal-to-noise ratio conditions. On the other hand, when the signal-to-noise ratio was good, GI provided better reconstructed images than IPGI, particularly with large numbers of illuminated patterns.
In practical applications, depending on the nature of the noise and the performance of the device (e.g., limitations of luminance and projection speed), IPGI may be more advantageous than GI and vice versa. By selecting optimal projection patterns and correlation formulas according to the noise situation, it may be possible to obtain better reconstructed images.
Disclosures
The authors declare no conflicts of interest.
Supplemental document
See Supplement 1 for supporting content.
References
1. T. B. Pittman, Y. H. Shih, D. V. Strekalov, and A. V. Sergienko, “Optical imaging by means of two photon quantum entanglement,” Phys. Rev. A 52(5), R3429–R3432 (1995). [CrossRef]
2. X. Gu and S. Zhao, “Nonorthogonal object identification based on ghost imaging,” Photonics Res. 3(5), 238–242 (2015). [CrossRef]
3. A. F. Abouraddy, B. E. A. Saleh, A. V. Sergienko, and M. C. Teich, “Role of Entanglement in Two-Photon Imaging,” Phys. Rev. Lett. 87(12), 123602 (2001). [CrossRef]
4. R. S. Bennink, S. J. Bentley, and R. W. Boyd, ““Two-Photon” Coincidence Imaging with a Classical Source,” Phys. Rev. Lett. 89(11), 113601 (2002). [CrossRef]
5. A. Gatti, E. Brambilla, M. Bache, and L. A. Lugiato, “Ghost Imaging with Thermal Light: Comparing Entanglement and Classical Correlation,” Phys. Rev. Lett. 93(9), 093602 (2004). [CrossRef]
6. X. Li, C. Deng, M. Chen, W. Gong, and S. Han, “Ghost imaging for an axially moving target with an unknown constant speed,” Photonics Res. 3(4), 153–157 (2015). [CrossRef]
7. W. Gong, “High-resolution pseudo-inverse ghost imaging,” Photonics Res. 3(5), 234–237 (2015). [CrossRef]
8. J. H. Shapiro, “Computational ghost imaging,” Phys. Rev. A 78(6), 061802 (2008). [CrossRef]
9. X. Liu, J. Shi, X. Wu, and G. Zeng, “Fast first-photon ghost imaging,” Sci. Rep. 8(1), 5012 (2018). [CrossRef]
10. K. Shibuya, K. Nakae, Y. Mizutani, and T. Iwata, “Comparison of reconstructed images between ghost imaging and Hadamard transform imaging,” Opt. Rev. 22(6), 897–902 (2015). [CrossRef]
11. D. B. Phillips, M. Sun, J. M. Taylor, M. P. Edgar, S. M. Barnett, G. M. Gibson, and M. J. Padgett, “Adaptive foveated single-pixel imaging with dynamic supersampling,” Sci. Adv. 3(4), e1601782 (2017). [CrossRef]
12. M. Sun, M. P. Edgar, D. B. Phillips, G. M. Gibson, and M. J. Padgett, “Improving the signal-to-noise ratio of single-pixel imaging using digital microscanning,” Opt. Express 24(10), 10476–10485 (2016). [CrossRef]
13. M. Sun, H. Wang, and J. Huang, “Improving the performance of computational ghost imaging by using a quadrant detector and digital micro-scanning,” Sci. Rep. 9(1), 4105 (2019). [CrossRef]
14. L. Wang and S. Zhao, “Fast reconstructed and high-quality ghost imaging with fast Walsh–Hadamard transform,” Photonics Res. 4(6), 240–244 (2016). [CrossRef]
15. F. Ferri, D. Magatti, L. A. Lugiato, and A. Gatti, “Differential Ghost Imaging,” Phys. Rev. Lett. 104(25), 253603 (2010). [CrossRef]
16. M. Bina, D. Magatti, M. Molteni, A. Gatti, L. A. Lugiato, and F. Ferri, “Backscattering Differential Ghost Imaging in Turbid Media,” Phys. Rev. Lett. 110(8), 083901 (2013). [CrossRef]
17. I. Hoshi, T. Shimobaba, T. Kakue, and T. Ito, “Computational ghost imaging using a field programmable,” OSA Continuum 2(4), 1097–1105 (2019). [CrossRef]
18. B. Sun, S. S. Welsh, M. P. Edgar, J. H. Shapiro, and M. J. Padgett, “Normalized ghost imaging,” Opt. Express 20(15), 16892–16901 (2012). [CrossRef]
19. S. Sun, J. Gu, H. Lin, L. Jiang, and W. Liu, “Gradual ghost imaging of moving objects by tracking based on cross correlation,” Opt. Lett. 44(22), 5594–5597 (2019). [CrossRef]
20. M. Lyu, W. Wang, H. Wang, H. Wang, G. Li, N. Chen, and G. Situ, “Deep-learning-based ghost imaging,” Sci. Rep. 7(1), 17865 (2017). [CrossRef]
21. Y. He, G. Wang, G. Dong, S. Zhu, H. Chen, A. Zhang, and Z. Xu, “Ghost Imaging Based on Deep Learning,” Sci. Rep. 8(1), 6469 (2018). [CrossRef]
22. S. Ma, Z. Liu, C. Wang, C. Hu, E. Li, W. Gong, Z. Tong, J. Wu, X. Shen, and S. Han, “Ghost imaging LiDAR via sparsity constraints using push-broom scanning,” Opt. Express 27(9), 13219–13228 (2019). [CrossRef]
23. W. Gong, C. Zhao, H. Yu, M. Chen, W. Xu, and S. Han, “Three-dimensional ghost imaging lidar via sparsity constraint,” Sci. Rep. 6(1), 26133 (2016). [CrossRef]
24. C. Deng, L. Pan, C. Wang, X. Gao, W. Gong, and S. Han, “Performance analysis of ghost imaging lidar in background light environment,” Photonics Res. 5(5), 431–435 (2017). [CrossRef]
25. A. Zhang, Y. He, L. Wu, L. Chen, and B. Wang, “Tabletop x-ray ghost imaging with ultra-low radiation,” Optica 5(4), 374–377 (2018). [CrossRef]
26. Z. Sun, F. Tuitje, and C. Spielmann, “Toward high contrast and high-resolution microscopic ghost imaging,” Opt. Express 27(23), 33652–33661 (2019). [CrossRef]
27. R. E. Meyers, K. S. Deacon, and Y. Shih, “Turbulence-free ghost imaging,” Appl. Phys. Lett. 98(11), 111115 (2011). [CrossRef]
28. Y.-K. Xu, W.-T. Liu, E.-F. Zhang, Q. Li, H.-Y. Dai, and P.-X. Chen, “Is ghost imaging intrinsically more powerful against scattering?” Opt. Express 23(26), 32993–33000 (2015). [CrossRef]
29. Q. Fu, Y. Bai, X. Huang, S. Nan, P. Xie, and X. Fu, “Positive influence of the scattering medium on reflective ghost imaging,” Photonics Res. 7(12), 1468–1472 (2019). [CrossRef]
30. L. Olivieri, J. S. Totero Gongora, L. Peters, V. Cecconi, A. Cutrona, J. Tunesi, R. Tucker, A. Pasquazi, and M. Peccianti, “Hyperspectral terahertz microscopy via nonlinear ghost imaging,” Optica 7(2), 186–191 (2020). [CrossRef]
31. F. Li, M. Zhao, Z. Tian, F. Willomitzer, and O. Cossairt, “Compressive ghost imaging through scattering media with deep learning,” Opt. Express 28(12), 17395–17408 (2020). [CrossRef]
32. Z. Gao, X. Cheng, K. Chen, A. Wang, Y. Hu, S. Zhang, and Q. Hao, “Computational Ghost Imaging in Scattering Media Using Simulation-Based Deep Learning,” IEEE Photonics J. 12(5), 1–15 (2020). [CrossRef]
33. B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, and M. J. Padgett, “Differential Computational Ghost Imaging,” in Imaging and Applied Optics, OSA Technical Digest (online) (Optical Society of America, 2013), paper CTu1C.4.
