Open Access
Add to BibSonomy
Abstract
Fourier ptychographic microscopy (FPM) is a recently developed computational imaging technique that has high-resolution and wide field-of-view (FOV). FPM bypasses the NA limit of the system by stitching a number of variable-illuminated measured images in Fourier space. On the basis of the wide FOV of the low NA objective, the high-resolution image with a wide FOV can be reconstructed through the phase recovery algorithm. However, the high-resolution reconstruction images are affected by the LED array point light source. The results are: (1) the intensities collected by the sample are severely declined when edge LEDs illuminate the sample; (2) the multiple reconstructions are caused by wavevectors inconsistency for the full FOV images. Here, we propose a new lighting scheme termed full FOV Fourier ptychographic microscopy (F3PM). By combining the LED array and telecentric lens, the method can provide plane waves with different angles while maintaining uniform intensity. Benefiting from the telecentric performance and f‒θ property of the telecentric lens, the system stability is improved and the relationship between the position of LED and its illumination angle is simplified. The excellent plane wave provided by the telecentric lens guarantees the same wavevector in the full FOV, and we use this wavevector to reconstruct the full FOV during one time. The area and diameter of the single reconstruction FOV reached 14.6mm2 and 5.4 mm, respectively, and the diameter is very close to the field number (5.5 mm) of the 4× objective. Compared with the traditional FPM, we have increased the diameter of FOV in a single reconstruction by ∼ 10 times, eliminating the complicated steps of computational redundancy and image stitching.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
High-resolution and wide field-of-view (FOV) microscopic imaging plays an important role in biological and physical sciences. However, it is difficult to obtain both of them at the same time by conventional microscope. People usually have to divide the whole region of interest (ROI) into pieces that match the FOV of microscope, capturing the high-resolution images one by one, and then stitching them together to reconstruct the complete ROI image. It is time consuming and device complex. In recent years, benefiting from the development of digital equipment and computational imaging, researchers have developed a variety of new imaging methods to solve this problem [1–10]. Fourier ptychographic microscopy (FPM) [11,12] is one of the most important methods, which combines wide FOV with high resolution. Besides, it can also reconstruct the phase information of sample. FPM achieves high resolution beyond the diffraction limit of the objective, which equals to the sum of the objective and illumination NAs. Although the spatial resolution of each measurement is low, the images collected with high illumination angles (dark field) contain sub-resolution information [13]. After all illumination NAs having been scanned, high-resolution complex image is reconstructed by nonlinear optimization algorithms [11,14,15].
A serious problem faced by FPM is the signal-to-noise ratio degradation under large angle illumination for high frequency information acquisition. The reason is that the intensities collected by the sample decrease sharply with the illumination angles increase. For a planar LED array, the intensity falloff can be approximately expressed as ∼I = I0 cosθ4 [16], where I0 is the central intensity under normal illumination and θ is the illumination angle. The second problem is that we cannot get an identical wave front vector over the whole FOV with the LED direct illumination. The only way is to divide the whole FOV into pieces to ensure that each sub-FOV could be regarded as a single wave front vector for reconstruction [11,17], and then stitched together to construct the whole image. In addition, in order to meet the needs of image stitching and light intensity correction, the overlap rate between adjacent sub-fields should be guaranteed to be 30% or more. These post-processing steps are cumbersome and bring more calculation consumption.
In terms of intensity fluctuation, some schemes have been proposed to solve it. For a planar LED array, one can modulate the camera exposure time to balance the illumination between the central LED and the external LEDs. However, it is time consuming and more vulnerable to noise. Changing the spatial arrangement of LEDs [16,18] can improve its energy utilization rate to ∼I = I0 cosθ. However, these ways need precise machining, higher installation accuracy and the wavevectors are still different in full FOV. Condenser or lens [19–23]can collect the illumination lights from LEDs and transfer them into plane waves, and the intensity falloff has been changed to ∼ I = I0 cosθ. But there are still some problems in use. First, the size of illumination beam and imaging FOV were limited by the exit pupil diameter. Second, a calibration process is needed before reconstruction, because of the non-linear relationship between the position of LED and its illumination angle. Furthermore, the condensers and lenses have some level of field curvatures [24]. Affected by field curvatures, the illumination light wave fronts will be distorted. Then the distorted wave fronts will lead to a deterioration in quality of reconstruction images.
In this paper, a new illumination method based on telecentric lens (TL) is introduced. We termed this method as Full FOV Fourier ptychographic microscopy (F3PM). It can provide the uniform illumination intensity for different angle and excellent plane wave front for the whole FOV. Furthermore, the relationship between position of LED and its illumination angle is simpler, because of the f‒θ property of the TL. Profiting from the telecentric performance of the lens, the accuracy of the distance between the LED array, the sample and the lens has been greatly reduced, which can significantly improve the reliability of the system [25,26]. Based on the excellent plane wave front, the same wavevector is obtained over the full FOV, and the full FOV image can be reconstructed during one time. Section 2 introduced the principle and its structure of F3PM. Section 3 illustrated the performance results, which indicated that the system could achieve a resolution of 0.78μm and a single reconstruction FOV of 14.6mm2 with 5.4 mm in diameter. The diameter is very close to the field number (5.5 mm) of 4× the objective lens. Compared with traditional illumination methods, we not only correct the illumination intensity, but also improve the diameter of FOV in a single reconstruction by ∼ 10 times.
2. Principle of F3PM
As shown in Fig. 1(a), the optical setup of F3PM consists of five major components: a LED array, a TL, an objective, a tube lens and a CCD camera. A LED array light source (SMD 1919, size of LED is 1.9mm×1.9 mm) is used to provide illumination at different angles, and the central wavelength is $520 \pm 10nm$. The objective (4× magnification, NA=0.1, field number of 5.5 mm, Olympus) and the tube lens (focal length of 18 mm) are used to image the sample. The CCD (Lumenra, infinity 4, 2650×4600 pixels, 9μm) is used to capture the images. The focal length of TL is 12cm, the FOV is 64mm, and the exit pupil diameter is 1cm. The LED array and the sample are located at the focal plane and the exit pupil position of the TL, respectively. The maximum illumination angle of TL is 19°, so the synthetic aperture and the full-pitch resolution of F3PM are 0.42 and 1.24μm, separately. The TL collects light emitted from the normal direction of each LED and collimates it to maintain a uniform plane wave front vector throughout the FOV, shown in Fig. 1(b). Benefiting from the telecentric character of TL, the light emitted from the normal direction of each LED can be collected by TL. Thus, there is a more standard intensity falloff I = I0 cosθ when TL provides plane waves with different angles. In addition, the telecentric character of TL reduces the z-axis adjustment accuracy of LED array greatly and improves the stability of the system [25,26]. When the LED array is slightly defocused, its main light can still pass through the sample plane and its illumination angle does not change.
TL also has the advantage of f‒θ property, which simplifies the relationship between position of LEDs and its illumination angle. There is a θ=x/f relationship between the position of LED and its illumination angle, where x is the position of LED, f is the focal length and θ is the illumination angles, shown in Fig. 1(b). The illumination angle of each LED can be expressed as (θix, θiy) = (xi/f, yi/f), where (xi, yi) is the position of ith LED, shown in Fig. 1(b). The wavevectors for full FOV can be expressed as
The excellent plane wave with the same wavevector provided by TL facilitates the reconstruction process of F3PM. Affected by wavevectors inconsistency, traditional FPM has to divide the full FOV into segments and then reconstructs them sequentially during reconstruction process. For F3PM, the wavevector is the same in the full FOV and the multiple reconstructions are unnecessary. Therefore, F3PM reconstructs the high-resolution image with full FOV directly. Although the parallelism of single reconstruction is lower than the multiple reconstructions when calculated in GPU, the full FOV images reconstruct process of traditional FPM is consists of multiple reconstructions, intensity correction for different patches and image stitching. On the contrary, for F3PM, the computational redundancy caused by image stitching is avoided and the efficiency of reconstruction is improved. The segment size of raw image used by traditional FPM mostly less than 500μm [12]. In contrast, F3PM reconstructs high-resolution images with FOV of 3.82mm×3.82 mm, breaking the limitation of wavevector inconsistency and improving the size of single high-resolution reconstruction nearly ten times. The size of FOV of single reconstruction of F3PM is limited by the field number of 4× objective, which can be larger by employing an objective with larger field number. In addition, if one’s hardware is limited, they can choose a reconstruction size smaller than whole FOV but larger than traditional method used. The reconstruction process becomes more flexible. Moreover, based on some single-shot FPM methods [28,29], there are only a few low-resolution images, a reconstruction for full FOV is faster and more advantageous in real time imaging.
3. Result
To verify the performance of F3PM, USAF 1951 target was imaged. In order to satisfy the sampling criteria and the overlap rate [30,31], the gap of LEDs is 4 mm and the overlap rate in frequency domain is 64%. During the capture process, the central 17×17 LEDs are switched on sequentially to capture 289 images. The reconstruction results of USAF 1951 reconstructed by traditional FPM and F3PM are shown in Fig. 2. We selected 1700×1700 pixels for reconstruction. The diameter of reconstructed area corresponding to the sample plane is 5.4 mm which is close to the field number (5.5 mm) of the objective lens. From Fig. 2(a1)-(a2), as the increases in diameter of reconstruction area, the reconstruction quality of traditional FPM declines rapidly. From Fig. 2(b1)-(b2) and Fig. 2(c1)-(c2), in spite of the size of image reconstructed by F3PM is larger than the traditional FPM, the reconstruction result of edge FOV does not degrade and without wrinkling artifacts [17,32]. In terms of the resolution of reconstruction results, shown in Fig. 2(b3)-(c3), the element 3 of group 9(645 lp/mm, resolution of 0.78μm) can be distinguished. However, influenced by the aberration of objective lens [32], the quality of Fig. 2(c2) which corresponding to the edge region is not as good as the Fig. 2(b2) which corresponding to the center. Fortunately, this difficulty can be overcome by using an objective lens with higher magnification [33].
Fig. 2. 1951 USAF resolution target imaging using traditional FPM and F3PM. (a1) Full FOV reconstruction image of traditional FPM. The high-resolution target is located at the FOV boundary. (b1) Full FOV reconstruction image of F3PM. The high-resolution target is located at the FOV center. (c1) Full FOV reconstruction image of F3PM. The high-resolution target is located at the FOV boundary. The magnified intensity images (a2)-(c2) are corresponding to the regions of red, green and blue boxes in (a1)- (c1). The magnified intensity images (a3)-(c3) are the high-resolution target zoom-in of (a2)-(c2). (a4)-(c4) are the low-resolution images corresponding to the (a3)-(c3).
To demonstrate the full-FOV resolution consistency of F3PM, a biological specimen experiment is carried out using cervical smear cells. The conditions are the same as the above experiments. Intensity and phase map of the cervical smear cells are reconstructed with FOV of 14.6mm2, as shown in Fig. 3(a1) and Fig. 3(b1), respectively. The details of intensity image are shown in (a2)-(a5) and the details of phase image are shown in (b2)-(b4). The ground truths are captured by the conventional microscope (×20 magnification, NA=0.46) and used for comparison. The raw data are the low-resolution images taken by F3PM. The reconstruction quality of Fig. 3(a4)-(a5) are the same as the quality of the ground truths. Therefore, F3PM can perform a single reconstruction of the images with full FOV while maintaining consistent resolution.
Fig. 3. The cervical smear cells reconstruction result. High resolution (a1) intensity and (b1) phase images, with full FOV of 14.6mm2. The magnified (a2)-(a3) intensity images and (b2)-(b3) phase images corresponding to the yellow and blue boxes in (a1) and (b1). (a4)-(a5) intensity images and (b4)-(b5) phase images are the zoom in of (a2)-(a3) and (b2)-(b3). The ground truth images are captured by a conventional microscope (×20 magnification, NA=0.46).
Traditional FPM is approximately coherent imaging, and the size of reconstruction images of F3PM does not meet the requirements of coherent imaging. In our work, the coherence length does not make troubles for us. First, the requirement of spatial coherence for FPM is reduced by capturing the images in spatial domain [34]. FPM only need to maintain the spatial coherence over the scale of the point spread function at the object plane and thus the partially coherent LED illumination can be used for FPM [35]. Second, we use an apodized transfer function [36] model for reconstruction, this model is closer to the real partial coherent imaging model [37]. By taking a fitting value α, the apodized transfer function can close to the real transfer function generated by light source with different size. Because of the small size of the lighting area of the LEDs, we choose the fitting value α equal to 0.2. Finally, the reconstruction results of small patch and full FOV are the same, shown in Fig. 4. Therefore, the coherence does not influence the single full FOV reconstruction under apodized transfer function model.
Fig. 4. (a) Reconstruction result of small patch. (b) Zoom-in of the reconstruction result of full FOV.
4. Conclusions
We present a new illumination method for FPM, named F3PM. F3PM obtains plane wave illumination with uniform intensity and different angels by combining the TL and LED array. TL has three advantages: telecentric character, f‒θ property and excellent plane wave front. The method overcomes the difficulties of intensity fluctuation and wavevectors inconsistency. Based on the same wavevector in full FOV, F3PM reconstructs images with FOV of 14.6mm2 during one time, which is nearly ten times larger than traditional FPM. Furthermore, F3PM reduces calculation redundancy and post processing steps. The performance of F3PM has been demonstrated by imaging USAF 1951 and biological specimen. In our perspective, this method has huge potential in wide FOV, high NA illumination and dynamic imaging of FPM.
Funding
National Natural Science Foundation of China (11604327, 11704377, 11774342, 11804336, 61377032, 61378075, 61405194, 61475152, 61775212, 61805238, 61811530061); Department of Science and Technology of Jilin Province (20170519016JH).
Acknowledgments.
This work is supported by CAS Interdisciplinary Innovation Team.
Disclosures
The authors declare no conflicts of interest.
References
1. A. Greenbaum, Y. Zhang, A. Feizi, P.-L. Chung, W. Luo, S. R. Kandukuri, and A. Ozcan, “Wide-field computational imaging of pathology slides using lens-free on-chip microscopy,” Sci. Transl. Med. 6(267), 267ra175 (2014). [CrossRef]
2. C. Zuo, J. Sun, J. Zhang, Y. Hu, and Q. Chen, “Lensless phase microscopy and diffraction tomography with multi-angle and multi-wavelength illuminations using a LED matrix,” Opt. Express 23(11), 14314–14328 (2015). [CrossRef]
3. S. Witte, V. T. Tenner, D. W. Noom, and K. S. Eikema, “Lensless diffractive imaging with ultra-broadband table-top sources: from infrared to extreme-ultraviolet wavelengths,” Light: Sci. Appl. 3(3), e163 (2014). [CrossRef]
4. L. Wei, A. Greenbaum, Y. Zhang, and A. Ozcan, “Synthetic aperture-based on-chip microscopy,” Light: Sci. Appl. 4(3), e261 (2015). [CrossRef]
5. W. Luo, Y. Zhang, A. Feizi, Z. Göröcs, and A. Ozcan, “Pixel super-resolution using wavelength scanning,” Light: Sci. Appl. 5(4), e16060 (2016). [CrossRef]
6. T. M. Turpin, L. H. Gesell, J. Lapides, and C. H. Price, “Theory of the synthetic aperture microscope,” Proc. SPIE - Int. Soc. Opt. Eng. 2566, 230–240 (1995). [CrossRef]
7. T. R. Hillman, T. Gutzler, S. A. Alexandrov, and D. D. Sampson, “High-resolution, wide-field object recons- truction with synthetic aperture Fourier holographic optical microscopy,” Opt. Express 17(10), 7873–7892 (2009). [CrossRef]
8. S. A. Alexandrov, T. R. Hillman, T. Gutzler, and D. D. Sampson, “Synthetic aperture Fourier holographic optical microscopy,” Phys. Rev. Lett. 97(16), 168102 (2006). [CrossRef]
9. S. Chowdhury and J. Izatt, “Structured illumination quantitative phase microscopy for enhanced resolution amplitude and phase imaging,” Biomed. Opt. Express 4(10), 1795–1805 (2013). [CrossRef]
10. A. G. York, S. H. Parekh, D. D. Nogare, R. S. Fischer, K. Temprine, M. Mione, A. B. Chitnis, C. A. Combs, and H. Shroff, “Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy,” Nat. Methods 9(7), 749–754 (2012). [CrossRef]
11. G. Zheng, R. Horstmeyer, and C. Yang, “Wide-field, high-resolution Fourier ptychographic microscopy,” Nat. Photonics 7(9), 739–745 (2013). [CrossRef]
12. X. Ou, R. Horstmeyer, C. Yang, and G. Zheng, “Quantitative phase imaging via Fourier ptychographic microscopy,” Opt. Lett. 38(22), 4845–4848 (2013). [CrossRef]
13. L. Tian, Z. Liu, L. H. Yeh, M. Chen, J. Zhong, and L. Waller, “Computational illumination for high-speed in vitro Fourier ptychographic microscopy,” Optica 2(10), 904–911 (2015). [CrossRef]
14. L. Tian, X. Li, K. Ramchandran, and L. Waller, “Multiplexed coded illumination for Fourier ptychography with an LED array microscope,” Biomed. Opt. Express 5(7), 2376–2389 (2014). [CrossRef]
15. C. Zuo, J. Sun, and Q. Chen, “Adaptive step-size strategy for noise-robust Fourier ptychographic microscopy,” Opt. Express 24(18), 20724–20744 (2016). [CrossRef]
16. Z. F. Phillips, M. V. D’Ambrosio, L. Tian, J. J. Rulison, H. S. Patel, N. Sadras, A. V. Gande, N. A. Switz, D. A. Fletcher, and L. Waller, “Multi-contrast imaging and digital refocusing on a mobile microscope with a domed LED array,” PLoS One 10(5), e0124938 (2015). [CrossRef]
17. A. Pan, C. Zuo, Y. Xie, M. Lei, and B. Yao, “Vignetting effect in Fourier ptychographic microscopy,” Opt. Lasers Eng. 120, 40–48 (2019). [CrossRef]
18. A. Pan, Y. Zhang, K. Wen, M. Zhou, J. Min, M. Lei, and B. Yao, “Subwavelength resolution Fourier ptychography with hemispherical digital condensers,” Opt. Express 26(18), 23119–23131 (2018). [CrossRef]
19. J. Sun, C. Zuo, L. Zhang, and Q. Chen, “Resolution-enhanced Fourier ptychographic microscopy based on high-numerical-aperture illuminations,” Sci. Rep. 7(1), 1187 (2017). [CrossRef]
20. S. Pacheco, G. Zheng, and R. Liang, “Reflective Fourier ptychography,” J. Biomed. Opt. 21(2), 026010 (2016). [CrossRef]
21. H. Lee, B. H. Chon, and H. K. Ahn, “Reflective Fourier ptychographic microscopy using a parabolic mirror,” Opt. Express 27(23), 34382–34391 (2019). [CrossRef]
22. S. Pacheco, B. Salahieh, T. Milster, J. J. Rodriguez, and R. Liang, “Transfer function analysis in epi-illumination Fourier ptychography,” Opt. Lett. 40(22), 5343–5346 (2015). [CrossRef]
23. K. Guo, Z. Bian, S. Dong, P. Nanda, Y. M. Wang, and G. Zheng, “Microscopy illumination engineering using a low-cost liquid crystal display,” Biomed. Opt. Express 6(2), 574–579 (2015). [CrossRef]
24. J. Chung, H. Lu, X. Ou, H. Zhou, and C. Yang, “Wide-field Fourier ptychographic microscopy using laser illumination source,” Biomed. Opt. Express 7(11), 4787–4802 (2016). [CrossRef]
25. J. Sun, Q. Chen, Y. Zhang, and C. Zuo, “Efficient positional misalignment correction method for Fourier ptychographic microscopy,” Biomed. Opt. Express 7(4), 1336–1350 (2016). [CrossRef]
26. R. Eckert, Z. F. Phillips, and L. Waller, “Efficient illumination angle self-calibration in Fourier ptychography,” Appl. Opt. 57(19), 5434–5442 (2018). [CrossRef]
27. V. Bianco, B. Mandracchia, J. Běhal, D. Barone, P. Memmolo, and P. Ferraro, “Miscalibration-tolerant Fourier Ptychography,” IEEE J. Sel. Top. Quantum Electron. 27(4), 1–17 (2021). [CrossRef]
28. J. Sun, Q. Chen, J. Zhang, Y. Fan, and C. Zuo, “Single-shot quantitative phase microscopy based on color-multiplexed Fourier ptychography,” Opt. Lett. 43(14), 3365–3368 (2018). [CrossRef]
29. B. Lee, J.-Y. Hong, D. Yoo, J. Cho, Y. Jeong, S. Moon, and B. Lee, “Single-shot phase retrieval via Fourier ptychographic microscopy,” Optica 5(8), 976–983 (2018). [CrossRef]
30. S. Dong, Z. Bian, R. Shiradkar, and G. Zheng, “Sparsely sampled Fourier ptychography,” Opt. Express 22(5), 5455–5464 (2014). [CrossRef]
31. J. Sun, Q. Chen, Y. Zhang, and C. Zuo, “Sampling criteria for Fourier ptychographic microscopy in object space and frequency space,” Opt. Express 24(14), 15765–15781 (2016). [CrossRef]
32. A. Zhou, W. Wang, N. Chen, E. Y. Lam, B. Lee, and G. Situ, “Fast and robust misalignment correction of Fourier ptychographic microscopy for full field of view reconstruction,” Opt. Express 26(18), 23661–23674 (2018). [CrossRef]
33. H. Lee, B. H. Chon, and H. K. Ahn, “Rapid misalignment correction method in reflective fourier ptychographic microscopy for full field of view reconstruction,” Opt. Lasers Eng. 138, 106418 (2021). [CrossRef]
34. G. Zheng, “Breakthroughs in Photonics 2014: Fourier Ptychographic Imaging,” IEEE Photonics J. 6(2), 1–7 (2014). [CrossRef]
35. G. Zheng, Fourier Ptychographic Imaging: A MATLAB Tutorial (Morgan & Claypool Publishers, 2016).
36. X. Chen, Y. Zhu, M. Sun, D. Li, Q. Mu, and L. Xuan, “Apodized coherent transfer function constraint for partially coherent Fourier ptychographic microscopy,” Opt. Express 27(10), 14099–14111 (2019). [CrossRef]
37. S. Dong, R. Shiradkar, P. Nanda, and G. Zheng, “Spectral multiplexing and coherent-state decomposition in Fourier ptychographic imaging,” Biomed. Opt. Express 5(6), 1757–1767 (2014). [CrossRef]
