Open Access
Add to BibSonomy
Abstract
Single-pixel imaging (SPI) uses a single-pixel detector to create an image of an object. SPI relies on a computer to construct an image, thus increasing both the size and cost of SPI and limiting its application. We developed instant single-pixel imaging (ISPI), an on-chip SPI system that implements real-time imaging at a rate of 25 fps. ISPI uses the instant ghost imaging algorithm we proposed, which leverages signal differences for image creation. It does not require a computer, which greatly reduces its both cost and size. The reconstruct time of ISPI for image creation is almost zero because little processing is required after signal detection. ISPI paves the way for the practical application of SPI.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
The single-pixel imaging (SPI) process is to illuminate an object with structured light, collect transmitted or reflected light from the object using a single-pixel detector, and then perform calculations to create an image of the object [1,2]. Recently, SPI based on ghost imaging (GI) [3–11] and computational ghost imaging (CGI) [12–15] has attracted widespread attention. SPI is of particular interest in areas where array cameras are unavailable or expensive, such as terahertz imaging [16,17], infrared imaging [18] and X-ray imaging [19]. It also has important applications in the fields of three-dimensional imaging [20–23], lidar [24,25], microscopic imaging [26,27], and encryption [28,29]. Recently, encoding methods based on complete orthogonal bases [30,31], such as the Fourier basis [32–35] and the Hadamard basis [36–40], have significantly improved SPI image quality and imaging speed [41]. SPI can also produce high-quality images using the compressive sensing algorithm with sub-Nyquist sampling [42–45].
However, SPI requires a computer to create the image of the object. Image creation is time-consuming and requires considerable computing resources. This requirement constrains the use of single-pixel cameras in practice. Conventional digital cameras use chips for imaging (pictures taken on mobile phones provide a ubiquitous example), which suggests that reducing the dependence of SPI on computers is an important problem to address. There has been research implementing a field-programmable gate array (FPGA) on the SPI, but a computer was also required [46]. The recently developed instant ghost imaging (IGI) algorithm provides an on-chip GI [47–51], thereby making the computer redundant. We first proposed the IGI algorithm in 2015 [49]. Jun-Lin Li introduced the algorithm into computer-based experiments as sequence differential ghost imaging (SDGI) in his doctoral dissertation in 2016 [50]. The IGI algorithm is not application-specific and can be used in SPI to make it computer-independent.
In this paper, we developed instant single-pixel imaging (ISPI), an on-chip SPI system that implements real-time imaging for moving objects at a rate of 25 fps. The IGI algorithm uses signal differences between two consecutive temporal measurements for image reconstruction. The image reconstruction time of the IGI algorithm is almost zero, i.e., no post-processing is necessary. Therefore, ISPI does not require a computer because it has few computational resources demands, which greatly reduces both the size and cost of SPI. Meanwhile, using signal differences have a strong intrinsic ability to resist common mode noise (the noise components of two consecutive signals are approximately equal, therefore, taking the signal difference effectively removes the noise), which gives ISPI an excellent capacity for resisting interference due to optical background noise. This on-chip implementation of ISPI makes SPI more accessible for practical applications.
2. Method
2.1 Instant ghost imaging algorithm
The IGI algorithm takes the form [47,48]:
We define the accumulation term $Acc_n^{IGI} = ({1/2N} )\mathop \sum \nolimits_{i = 1}^n ({{S_{i + 1}} - {S_i}} )[{{I_{i + 1}}(x )- {I_i}(x )} ]$, where n is the ${n^{th}}$ measurement and the index i ranges from 1 to n. The IGI algorithm simultaneously measures and processes. When the last measurement has been made, the accumulation term is the resulting final image. The image calculation processing time is almost zero, so it is considered to be instant.
Furthermore, we gave a detailed proof in [47] that Eq. (1) is approximately equal to the standard background-subtraction-related imaging equation
2.2 Experimental configuration
The equipment used in the experiment is shown in Fig. 1. A 532 nm reflected laser beam is expanded by lens 1. It illuminates the DMD, which operates at a frequency of 20 kHz, with random matrix coding to generate structured light. Lens 2 images DMD on the object. The square object contains a cut-out letter T, whose size is 0.8 mm ${\times}$ 0.8 mm. Transmitted light that passes through the T is collected by the bucket detector.
In the following, we analyze the workflow of ISPI. The FPGA (Xilinx XC7K325T) receives signal ${S_n}$ and calculates ${S_n} - {S_{n - 1}}$ using signal ${S_{n - 1}}$ stored in the inbuilt memory of the FPGA. When the calculation has been completed, ${S_n}$ overwrites ${S_{n - 1}}$. During the same iteration, ${I_n}(x )$ and ${I_{n - 1}}(x )$ are retrieved from FPGA memory to calculate $({{S_n} - {S_{n - 1}}} )[{I_{n + 1}}(x )- {I_n}(x )]$, which is added to $Acc_{n - 1}^{IGI}$ (also stored in FPGA memory) to give $Acc_n^{IGI}$, and then $Acc_n^{IGI}$ overwrite $Acc_{n - 1}^{IGI}$. When the last measurement is completed, the FPGA calculates $Acc_N^{IGI}$, which is the image of the object. Therefore, the image can be obtained in real time after the measurement, and the image reconstruction time is almost zero.
To demonstrate the ability of ISPI to resist interference from optical background noise, we placed an LED capable of producing light at different frequencies and intensities in the configuration to introduce light into the optical path as optical background noise.
3. Results
3.1 Images of the object
Figure 2(a) shows the object, the letter T. The size of the image is 32 pixels × 32 pixels. Figures 2(b)–2(e) show the images produced by ISPI. The total number of measurements N were respectively 800, 1600, 2400, 3200, and 4000. The results show that ISPI can create an accurate image of the object.
Note that the time on each image (the imaging time) is the time required for ISPI to generate an image when the DMD operated at 20 kHz. Because the ISPI image construction time is almost zero, and no post-processing is required, the imaging time is the time required for the DMD to project N patterns.
3.2 Images of a moving object
ISPI can create images of moving objects and produce real-time video at a frame rate of 25 fps. The frame rate is equal to ${V_{DMD}}/N$, where ${V_{DMD}} = 20$ kHz, which is the DMD operating frequency, and N is the number of measurements required to create one image. In Fig. 3, we set $N = 800$, and the frame rate was 25 Hz, which is better than the standard video rate of 24 fps. As shown in the leftmost figure of each row, the object moved from the lower right to the upper left. The speed at which the object moved was controlled by a stepper motor. In row (1) of Fig. 3, the object had a speed of 0.1 mm/s; in row (2), it had a speed of 0.8 mm/s. Since ISPI does not need to perform post-processing calculations, it can produce real-time videos. The related videos of Fig. 3 are shown in Visualization 1 and Visualization 2.
3.3 Resistance to optical background noise
The signal difference algorithm inherently resists common noise modes. To show that ISPI resists optical background noise, we conducted an experiment in which an LED was used to introduce background noise to the laser signal. The LED was controlled to generate optical background noise of different intensities and different frequencies (Fig. 4).
In Fig. 4, row (1) is the signal curve of the bucket detector S for different noise frequencies. Row (2) shows the images produced by a conventional CGI algorithm as Eq. (2). Row (3) shows the results given by ISPI. The total number of imaging measurements was $N = 4000$. Column (a) shows the bucket detector signal S and images obtained with no added background noise, and columns (b)–(d) show the bucket detector signal S and images obtained with different optical background noise operating frequencies.
It can be seen from row (1) that the background noise has a great effect on the measurements made at the barrel detector. Row (2) shows that image quality significantly decreased when the CGI algorithm was subjected to background noise, even to the extent of no image being produced. Row (3) shows that ISPI is extremely resistant to optical background noise.
Furthermore, the ISPI can well resist the optical background noise when
where Q is the intensity of optical background noise. The optical background noise is required to be a slow variation signal compared to $S$. If this condition is satisfied, the ISPI will not be affected by the optical background noise regardless of the type such as Poisson noise, Gaussian noise.4. Discussion and conclusion
In this paper, we described ISPI, an on-chip SPI using the IGI algorithm. Although the previous work has involved the utilization of an FPGA in the field of GI [47], we are the first to implement high-speed real-time SPI that is truly computer-independent. We are able to accomplish this because the IGI algorithm requires few computational resources for image creation. This implementation greatly reduces both the size and cost of SPI, which makes SPI more practical as it aligns the SPI with current digital cameras and phones that require only a chip for imaging and have no need for a computer.
The image creation time of ISPI is almost zero, and the calculation required is completed at the time the image is acquired. Therefore, there is no need for any post-processing. Thus, ISPI is effectively a real-time imaging system, which is why we named it instant.
Another advantage of ISPI is its resistance to optical background noise. SPI and CGI can both resist turbulence and scattering medium [52–54] but do not have a strong capacity for resisting optical background noise [55,56]. We have experimentally verified that ISPI has excellent resistance to optical background noise. In an environment containing optical background noise that changed drastically in intensity, ISPI was almost unaffected by the noise and produced an accurate image of the object because the signal difference algorithm intrinsically resists common noise modes [48].
The ISPI can be improved in three areas: imaging speed, imaging quality, and image size.
4.1 Imaging speed
ISPI currently operates at 20 kHz. This operating frequency is governed by the DMD, not by any part of the ISPI firmware. The FPGA can operate at >100 MHz, making it potentially 5000 times faster than it is at the current operating frequency. Thus, the imaging speed of ISPI can be greatly increased by using faster DMDs or faster light source arrays [41].
4.2 Imaging quality
In this work, a random basis with 0 and 1 was used for imaging. It is well known that the random matrix (non-orthogonal) cannot achieve perfect imaging. However, the orthogonal basis can be used to achieve the perfect imaging under the full sampling condition [30–40]. ISPI using a complete orthogonal basis and the signal difference algorithm are being studied.
4.3 Image size
The ISPI images are currently limited by the internal memory space of the FPGA chip. We plan to augment the FPGA memory with external double data rate-random access memory (DDR-RAM) to create larger images of 1024 pixels × 768 pixels.
In summary, we prototyped ISPI, an on-chip high-speed real-time SPI using the IGI algorithm. ISPI can image objects without post-processing, can produce real-time images of moving objects, and has excellent resistance to optical background noise. It also greatly reduces the cost and size of SPI, reducing reconstruction time to almost zero. We believe this application will greatly promote the practicality of SPI.
Funding
National Natural Science Foundation of China (51727805).
Disclosures
The authors declare no conflicts of interest.
References
1. M. P. Edgar, G. M. Gibson, and M. J. Padgett, “Principles and prospects for single-pixel imaging,” Nat. Photonics 13(1), 13–20 (2019). [CrossRef]
2. M. J. Sun and J. M. Zhang, “Single-pixel imaging and its application in three-dimensional reconstruction: a brief review,” Sensors 19(3), 732 (2019). [CrossRef]
3. T. B. Pittman, Y. H. Shih, D. V. Strekalov, and A. V. Sergienko, “Optical imaging by means of two-photon quantum entanglement,” Phys. Rev. A 52(5), R3429–R3432 (1995). [CrossRef]
4. R. S. Bennink, S. J. Bentley, and R. W. Boyd, ““Two-photon” coincidence imaging with a classical source,” Phys. Rev. Lett. 89(11), 113601 (2002). [CrossRef]
5. F. Ferri, D. Magatti, A. Gatti, M. Bache, E. Brambilla, and L. A. Lugiato, “High-resolution ghost image and ghost diffraction experiments with thermal light,” Phys. Rev. Lett. 94(18), 183602 (2005). [CrossRef]
6. D. Z. Cao, J. Xiong, and K. G. Wang, “Geometrical optics in correlated imaging systems,” Phys. Rev. A 71(1), 013801 (2005). [CrossRef]
7. L. Basano and P. Ottonello, “Experiment in lensless ghost imaging with thermal light,” Appl. Phys. Lett. 89(9), 091109 (2006). [CrossRef]
8. P. Ryczkowski, M. Barbier, and A. Friberg, “Ghost imaging in the time domain,” Nat. Photonics 10(3), 167–170 (2016). [CrossRef]
9. Z. Yang, O. S. Magaña-Loaiza, M. Mirhosseini, Y. Zhou, B. Gao, L. Gao, S. M. H. Rafsanjani, G. L. Long, and R. W. Boyd, “Digital spiral object identification using random light,” Light: Sci. Appl. 6(7), e17013 (2017). [CrossRef]
10. N. Bornman, M. Agnew, F. Zhu, A. Vallés, A. Forbes, and J. Leach, “Ghost imaging using entanglement-swapped photons,” npj Quantum Inf. 5(1), 63 (2019). [CrossRef]
11. Y. Endo, T. Tahara, and R. Okamoto, “Color single-pixel digital holography with a phase-encoded reference wave,” Appl. Opt. 58(34), G149–G154 (2019). [CrossRef]
12. J. H. Shapiro, “Computational ghost imaging,” Phys. Rev. A 78(6), 061802 (2008). [CrossRef]
13. S. S. Welsh, M. P. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. J. Padgett, “Fast full-color computational imaging with single-pixel detectors,” Opt. Express 21(20), 23068–23074 (2013). [CrossRef]
14. S. Sun, W. T. Liu, H. Z. Lin, E. F. Zhang, J. Y. Liu, Q. Li, and P. X. Chen, “Multi-scale adaptive computational ghost imaging,” Sci. Rep. 6(1), 37013 (2016). [CrossRef]
15. H. C. Liu, B. Yang, Q. Guo, J. Shi, C. Guan, G. Zheng, H. Mühlenbernd, G. Li, T. Zentgraf, and S. Zhang, “Single-pixel computational ghost imaging with helicity-dependent metasurface hologram,” Sci. Adv. 3(9), e1701477 (2017). [CrossRef]
16. C. M. Watts, D. Shrekenhamer, J. Montoya, G. Lipworth, J. Hunt, T. Sleasman, S. Krishna, D. R. Smith, and W. J. Padilla, “Terahertz compressive imaging with metamaterial spatial light modulators,” Nat. Photonics 8(8), 605–609 (2014). [CrossRef]
17. J. Zhao, E. Yiwen, K. Williams, X. C. Zhang, and R. Boyd, “Spatial sampling of terahertz fields with sub-wavelength accuracy via probe beam encoding,” Light: Sci. Appl. 8(1), 55 (2019). [CrossRef]
18. N. Radwell, K. J. Mitchell, G. M. Gibson, M. P. Edgar, R. Bowman, and M. J. Padgett, “Single-pixel infrared and visible microscope,” Optica 1(5), 285–289 (2014). [CrossRef]
19. A. X. Zhang, Y. H. He, L. A. Wu, L. M. Chen, and B. B. Wang, “Tabletop x-ray ghost imaging with ultra-low radiation,” Optica 5(4), 374–377 (2018). [CrossRef]
20. B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, and M. J. Padgett, “3-D Computational imaging with single-pixel detectors,” Science 340(6134), 844–847 (2013). [CrossRef]
21. M. J. Sun, M. P. Edgar, G. M. Gibson, B. Sun, N. Radwell, R. Lamb, and M. J. Padgett, “Single-pixelthree- dimensional imaging with time-based depth resolution,” Nat. Commun. 7(1), 12010 (2016). [CrossRef]
22. K. Soltanlou and H. Latifi, “Three-dimensional imaging through scattering media using a single pixel detector,” Appl. Opt. 58(28), 7716–7726 (2019). [CrossRef]
23. L. Zhang, Z. Lin, R. He, Y. Qian, Q. Chen, and W. Zhang, “Improving the noise immunity of 3D computational ghost imaging,” Opt. Express 27(3), 2344–2353 (2019). [CrossRef]
24. W. Gong, C. Zhao, H. Yu, M. Chen, W. Xu, and S. Han, “Three-dimensional ghost imaging lidar via sparsity constraint,” Sci. Rep. 6(1), 26133 (2016). [CrossRef]
25. S. Ma, Z. Liu, C. Wang, C. Hu, E. Li, W. Gong, Z. Tong, J. Wu, X. Shen, and S. Han, “Ghost imaging LiDAR via sparsity constraints using push-broom scanning,” Opt. Express 27(9), 13219–13228 (2019). [CrossRef]
26. Z. Sun, F. Tuitje, and C. Spielmann, “Toward high contrast and high-resolution microscopic ghost imaging,” Opt. Express 27(23), 33652–33661 (2019). [CrossRef]
27. F. Devaux, P. A. Moreau, S. Denis, and E. Lantz, “Computational temporal ghost imaging,” Optica 3(7), 698–701 (2016). [CrossRef]
28. P. Clemente, V. Durán, V. Torres-Company, E. Tajahuerce, and J. Lancis, “Optical encryption based on computational ghost imaging,” Opt. Lett. 35(14), 2391–2393 (2010). [CrossRef]
29. W. Chen, “Ghost identification based on single-pixel imaging in big data environment,” Opt. Express 25(14), 16509–16516 (2017). [CrossRef]
30. K. M. Czajkowski, A. Pastuszczak, and R. Kotyński, “Single-pixel imaging with Morlet wavelet correlated random patterns,” Sci. Rep. 8(1), 466 (2018). [CrossRef]
31. S. M. Khamoushi, Y. Nosrati, and S. H. Tavassoli, “Sinusoidal ghost imaging,” Opt. Lett. 40(15), 3452–3455 (2015). [CrossRef]
32. Z. Zhang, X. Ma, and J. Zhong, “Single-pixel imaging by means of Fourier spectrum acquisition,” Nat. Commun. 6(1), 6225 (2015). [CrossRef]
33. L. Bian, J. Suo, X. Hu, F. Chen, and Q. Dai, “Efficient single pixel imaging in Fourier space,” J. Opt. 18(8), 085704 (2016). [CrossRef]
34. Z. Zhang, X. Wang, G. Zheng, and J. Zhong, “Hadamard single-pixel imaging versus Fourier single-pixel imaging,” Opt. Express 25(16), 19619–19639 (2017). [CrossRef]
35. H. Jiang, S. Zhu, H. Zhao, B. Xu, and X. Li, “Adaptive regional single-pixel imaging based on the Fourier slice theorem,” Opt. Express 25(13), 15118–15130 (2017). [CrossRef]
36. L. Wang and S Zhao, “Fast reconstructed and high-quality ghost imaging with fast Walsh-Hadamard transform,” Photonics Res. 4(6), 240–244 (2016). [CrossRef]
37. M. J. Sun, L. T. Meng, M. P. Edgar, M. J. Padgett, and N. Radwell, “A Russian Dolls ordering of the Hadamard basis for compressive single-pixel imaging,” Sci. Rep. 7(1), 3464 (2017). [CrossRef]
38. C. Zhou, T. Tian, C. Gao, W. Gong, and L. Song, “Multi- resolution progressive computational ghost imaging,” J. Opt. 21(5), 055702 (2019). [CrossRef]
39. M. F. Li, L. Yan, R. Yang, and Y. X. Liu, “Fast single-pixel imaging based on optimized reordering Hadamard basis,” Acta Phys. Sin. 68(6), 064202 (2019). [CrossRef]
40. S. Jiang, X. Li, Z. Zhang, W. Jiang, Y. Wang, G. He, Y. Wang, and B. Sun, “Scan efficiency of structured illumination in iterative single pixel imaging,” Opt. Express 27(16), 22499–22507 (2019). [CrossRef]
41. Z. H. Xu, W. Chen, J. Penuelas, M. Padgett, and M. J. Sun, “1000 fps computational ghost imaging using LED-based structured illumination,” Opt. Express 26(3), 2427–2434 (2018). [CrossRef]
42. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 83–91 (2008). [CrossRef]
43. Y. Bromberg, O. Katz, and Y. Silberberg, “Ghost imaging with a single detector,” Phys. Rev. A 79(5), 053840 (2009). [CrossRef]
44. Y. Wang, Y. Liu, J. Suo, G. Situ, C. Qiao, and Q. Dai, “High speed computational ghost imaging via spatial sweeping,” Sci. Rep. 7(1), 45325 (2017). [CrossRef]
45. K. M. Czajkowski, A. Pastuszczak, and R. Kotyński, “Real-time single-pixel video imaging with Fourier domain regularization,” Opt. Express 26(16), 20009–20022 (2018). [CrossRef]
46. I. Hoshi, T. Shimobaba, T. Kakue, and T. Ito, “Computational ghost imaging using a field-programmable gate array,” OSA Continuum 2(4), 1097–1105 (2019). [CrossRef]
47. Z. Yang, W. X. Zhang, Y. P. Liu, D. Ruan, and J. L. Li, “Instant ghost imaging: algorithm and on-chip Implementation,” Opt. Express 28(3), 3607–3618 (2020). [CrossRef]
48. Z. Yang, W. X. Zhang, M. C. Zhang, D. Ruan, and J. L. Li, “Instant Ghost Imaging: improving robustness for ghost imaging on optical background noise,” OSA Continuum 3(2), 391–400 (2020). [CrossRef]
49. J. L. Li, Z. Yang, and L. L. Long, Patent CN201510151008.2 (2015) (In Chinese).
50. J. L. Li, “Study on second-order correlated imaging with pseudo-thermal light,” Ph.D. thesis, Tsinghua University, (2016) (In Chinese).
51. Z. Yang, J. L. Li, and W. X. Zhang, Patent CN201910795456.4 (2019) (In Chinese).
52. R. E. Meyers, K. S. Deacon, and Y. Shih, “Turbulence-free ghost imaging,” Appl. Phys. Lett. 98(11), 111115 (2011). [CrossRef]
53. Z. Yang, L. Zhao, X. Zhao, W. Qin, and J. Li, “Lensless ghost imaging through the strongly scattering medium,” Chin. Phys. B 25(2), 024202 (2016). [CrossRef]
54. M. Le, G. Wang, H. Zheng, J. Liu, Y. Zhou, and Z. Xu, “Underwater computational ghost imaging,” Opt. Express 25(19), 22859–22868 (2017). [CrossRef]
55. C. Deng, L. Pan, C. Wang, X. Gao, W. Gong, and S. Han, “Performance analysis of ghost imaging lidar in background light environment,” Photonics Res. 5(5), 431–435 (2017). [CrossRef]
56. Z. Li, Q. Zhao, and W. Gong, “Experimental investigation of ghost imaging in background light environments,” J. Opt. 22(2), 025201 (2020). [CrossRef]
