Open Access
Add to BibSonomy
Abstract
To reduce the long data acquisition time of the common mechanical scanning based Ptychographic Iterative Engine (PIE) technique, the digital micro-mirror device (DMD) is used to form the fast scanning illumination on the sample. Since the transverse mechanical scanning in the common PIE is replaced by the on/off switching of the micro-mirrors, the data acquisition time can be reduced from more than 15 minutes to less than 20 seconds for recording 12 × 10 diffraction patterns to cover the same field of 147.08 mm2. Furthermore, since the precision of DMD fabricated with the optical lithography is always higher than 10 nm (1 μm for the mechanical translation stage), the time consuming position-error-correction procedure is not required in the iterative reconstruction. These two improvements fundamentally speed up both the data acquisition and the reconstruction procedures in PIE, and relax its requirements on the stability of the imaging system, therefore remarkably improve its applicability for many practices. It is demonstrated experimentally with both USAF resolution target and biological sample that, the spatial resolution of 5.52 μm and the field of view of 147.08 mm2 can be reached with the DMD based PIE method. In a word, by using the DMD to replace the translation stage, we can effectively overcome the main shortcomings of common PIE related to the mechanical scanning, while keeping its advantages on both the high resolution and large field of view.
© 2017 Optical Society of America
1. Introduction
Compared to imaging techniques based on the intensity detection, phase imaging can easily obtain higher image contrast especially in imaging the semi-transparent biological samples, and since they can provide another perspective for the sample observation, phase imaging is becoming more and more important for medical examinations and disease diagnoses [1,2]. Qualitative phase imaging techniques including the phase contrast and differential interference contrast are mainly adopted to make the phase retardation of sample qualitatively visible to naked eyes. Quantitative phase imaging techniques can measure the accurate complex amplitude of the light field including the modulus and phase simultaneously and thus are more suitable for the quantitative analysis. In digital holography and other interference based imaging methods, the regular reference beam adopted always makes the setup complicated and sensitive to the outside disturbance [3–5]. Transport of intensity equation (TIE) method, which is a non-interference phase imaging technique, has much simpler setup and lower requirement on the working environment [6–13], and recently developed Fourier ptychographic microscopy [14,15] is another non-interference phase imaging technique that can provide complex amplitude of sample, however like other lens aided imaging methods, the accuracy of these two techniques is dependent to the used optics.
Ptychographic Iterative Engine (PIE), which is a recently developed coherent diffraction imaging (CDI) method, can retrieve both the complex amplitude of the illumination beam and that of the object with two counterpart formulas simultaneously [16]. The accuracy of its reconstruction is not influenced by the aberration of optics used to form the illumination, and the resolution reachable is only limited by the numerical aperture of the detection device [17–20]. During the data acquisition of common PIE, the sample is scanned through a localized illuminating beam to a grid of positions, and all the diffraction patterns formed are recorded in the far field by the detector. To get high quality reconstruction, the two neighboring illuminated regions should properly overlap with each other, and the overlapping ratio is about 70%, in other words the grid spacing is about 30% of the diameter of illuminating region [21]. The field of view (FoV) of PIE can be infinitely large in theory if we scan the sample to numerous positions and record all the diffractions. Unlike traditional CDI, which works well only for samples with simple structure, PIE always can reach the reconstruction accuracy higher than 95% after only 200 rounds of iterations even for complicated samples [22], showing high convergence speed and high reliability. Furthermore, since PIE uses Wegener-filter like updating formulas for reconstruction, the retrieved image is always free of the speckle noise [23,24]. Thus, in comparison with traditional CDI techniques using Gerchberg-Saxton algorithm, Error Reduction algorithm and Hybrid Input Output algorithm, which suffer from shortcomings of small field of view (less than 1mm for most of cases), poor convergence speed (more than 10,000 iterations is needed depending on the field of view and the complexity of the sample) and low reliability especially in imaging samples with complex structures, PIE has obvious advantages of high convergence speed, high reliability, speckle free reconstructions and infinitely large field of view in theory, making PIE an outstanding phase imaging technique. In the past years, various meaningful quantitative observations and measurements were realized with PIE: Zhang et al. realized complex amplitude imaging of the monocotyledon in visible light [25], Rodenburg et al. measured the gold structure in hard X-Ray [26], and Hüe et al. retrieved the particle configuration with the electron beam [27]. In other words, PIE has been widely studied with the visible light, x-ray and high energy electron beam worldwide and got great achievements in the fields of material science, biological research and medical diagnosis.
In our experiments, more than 15 minutes are needed to scan the sample to 12 × 10 positions with the mechanical translation stage of our self-built PIE system, this makes the data acquisition time consuming and leads to high requirement on the stability of the imaging system and working environment. At the same time, since the accuracy of mechanical translation stage is only about 1 μm, to get high quality reconstruction, the position-error should be corrected iteratively in the reconstruction procedure to reduce the position error to sub-micron, seriously slowing down the reconstructing speed. Thus, the mechanical translation stage is the main source for most of the shortcomings of PIE, including the extremely long data acquisition time, high requirements on the stability of the imaging system and time-consuming reconstruction [28–31]. Though PIE can realize very large field of view in theory, but the too long data acquisition time required is unacceptable for most of practical cases, thus the large field of view essentially is difficult to acquire for common mechanical PIE in practices. Single-shot PIE methods [32,33] fitting for real time measurements often suffer from small FoV (less than 5 mm2), limiting their observation capability, moreover, the reconstruction quality especially the signal to noise ratio is still much lower than that of common PIE because of its low data redundancy.
To overcome these shortcomings of common PIE techniques, digital micro-mirror device (DMD) based PIE method is proposed to achieve the large field of view (FoV), fast speed data acquisition and high quality reconstruction simultaneously. DMD has been used in other imaging techniques including the tomographic phase microscopy and Fourier ptychographic microscopy to realize the phase measurement, where it was adopted mainly to avoid the rotating of the sample or to reduce the position-error of the light source by forming multi-directional illumination on the sample [34,35]. Quite different from these two existing techniques, our proposed method uses the on/off switching of the micro-mirrors of DMD to replace the mechanical scanning of the sample in common PIE. The switched on micro-mirrors within a rapidly moving region will reflect the laser light to form a rapidly scanning illumination beam. Since the on/off switching can be operated at 1K Hz or faster, the data acquisition time is mainly occupied by the CCD exposure and its data transfer to storage, the overall data acquisition time for recording 12 × 10 diffractions is successfully reduced to less than 20 seconds covering a FoV of 147.08 mm2 in experiment. Compared to common mechanical scanning based PIE, which needs at least 15 minutes to scan the same area, the data acquisition is speeded up by 45 times. Additionally, since the position error of each micro-mirrors of the DMD fabricated with optical lithography is only at the scale of several nano-meters, its influence to the final reconstruction of PIE is negligible, and the accurate reconstruction can be achieved without using the time consuming iterative position-correct computation, also speeding up the iterative reconstruction distinctively. In a word, the main shortcomings of common PIE were successfully overcome by using a DMD to replace the mechanical translation stage, achieving the large FoV, the high imaging quality and the high imaging speed simultaneously.
2. Principle and methods
The principle of the DMD based PIE can be demonstrated with the experimental setup in Fig. 1(A), where the laser is expanded into a planar laser beam and illuminated on the surface of DMD at the incident angle of 12°. Micro-mirrors within a proper region are switched to the 'on' state to reflect the laser beam to the sample, forming an illuminating probe, and when the position of this circular region is programmed to rapidly move on the DMD sensor, the illuminating probe will quickly scan the sample and form diffraction patterns on the CCD sensor. Unlike the common PIE, where the sample fixed on a translation stage is scanned to many positions relative to the static illuminating beam and static detector, In Fig. 1(A), the sample, the detector and the light beam incident on the DMD all keep stationary during the whole data acquisition, what we do to finish the data acquisition is just to switch on/off micro-mirrors within a moving region to form the scanning illumination on the sample and record the diffractions, and this can be automatically and rapidly realized with a computer program. The maximum FoV reachable is limited by the size of the DMD sensor and the size of the CCD chip. Take our experiment as an example, the DMD used is the DLP Discovery D4100 of Texas Instruments with a sensor size of 14.01 by 10.51 mm2, the CCD used is AVT-PE 1100B with a chip size of 15.16 by 15.16 mm2, so the maximum sample size that can be imaged with the proposed method is 14.01 by 10.51mm2, which is about 45 times that of a 10 × objective (~3.21 mm2) and is large enough for most of researches in biological and material sciences. The numerical aperture of 10 × objective is about 0.25, and the numerical aperture of the above mentioned experimental setup is about 0.7, however since AVT-PE 1100B is only an industry CCD, because of its low sensitivity, lots of weak high frequency components cannot be recorded, the spatial resolution reached is only about 5μm in experiments. If AVT-PE 1100B is replaced by a scientific CCD camera, the resolution can be further improved remarkably.
After all the diffraction patterns are recorded, the sample transmission function and the distribution of the scanning laser beam illuminating on the sample can be reconstructed with the standard PIE algorithm according to the flow chat of Fig. 1(B). The iterative reconstruction can be carried out with the following steps:
- (1) Give two initial guesses to the sample transmission function On(r) and the illuminating probe on the sample Pn(r).
- (2) Calculate the exiting wave of the sample as ψn(r) = On(r) × Pn(r) and propagate it to the detector plane as Ψn(u) = F{ψn(r)}, F denotes the Fresnel propagation from the sample plane to the recording plane with the Fresnel integral [22].
- (3) Replace the modulus of the Ψn(u) by the square root of the recorded intensity I(u) and propagate it back to the sample plane to obtain the new exiting field as ψn'(r) = F−1{Ψn(u)}, F−1denotes the inverse propagation from the recording plane to the sample plane.
- (4) Update the illuminating probe and the sample transmission function according to Eq. (1) and (2):
Where α and β are two constants with nonzero values chosen between 0~1. α is used to avoid the zero denominator, and β is used to adjust the updating speed [16].
- (5) Propagate the updated illuminating probe F{Pn'(r)} to the DMD plane using Eq. (3) and force the value outside the H(r), where the micro-mirrors of 'off' state to zero. Since the deviation of the size of the micro-mirror from the designed size is always less than 10 nm, which is much less than the wavelength used, we can simply assume that all the micro-mirrors are the same.
- (7) Jump to step (2) to begin another round of computation till all the scanning positions are addressed.
3. Experiments
The experiments are carried out with the setup schematically shown in Fig. 1(A). The wavelength used is 632.8 nm. The distance of the sample to the DMD and that to the CCD camera are 20.32 cm and 1.89 cm, respectively. Since the iterative computation is carried out between the sample plane and the recording plane, and the exact distribution of the illumination beam moving on the sample is also reconstructed iteratively, the distance between the DMD and the sample is only used to generate an initial guess to the illumination beam for the iterative computation and need not be known accurately, and accordingly the properties of the reflection taking place at the sharp edge of the micro-mirrors need not be considered also. In experiment, the iterative reconstruction can be carried out with the measured distance of the sample to the CCD firstly, and then by propagating the reconstructed complex amplitude backward or forward a little with the Fresnel integral the most clear image and the corresponding real distance of the sample to the CCD can be obtained [36–38]. The parallel laser beam illuminating on the DMD is planar enough, and micro-mirrors within a circular region of about 3.4 mm in diameter are switched to the 'on' state to reflect part of the laser beam to form a illumination probe vertically falling on the sample, and this circular region containing about 61700 micro-mirrors is moved to 12 × 10 positions within 20 seconds, and the all the diffraction patterns are recorded sequentially.
The sample used is a paraffin sectioning of stem cutting of monocot plant. The size of the observed FoV in the proposed PIE is ~45 time larger than the field of view of the commercial 10 × objective. Figure 2(A) shows the reconstructed modulus image of the whole sample, and the top insets are three recorded diffractions. The fine details of the sample can be clearly observed in the zoomed-in modulus and phase images that are also inserted in Fig. 2(A). Other two biological samples of Corn Stems and Umbrella Plant were also quantitatively imaged and shown in Figs. 2(B) and 2(C), respectively. It is obviously that, in all these images, the fine details of the sample structures are all quite clear in both modulus and phase images, showing a good applicability of this proposed method for the biological observation and quantitative analysis. Moreover, large FoV of 147.08 mm2 is reached within 20 seconds, which is about 45 times faster than common mechanical scanning based PIE.
Fig. 2 (A) Quantitative imaging of stem cutting of monocot plant with a FoV of 147.08 mm2. Top insets are recorded diffractions at corresponding positions, Bottom insets are magnified modulus and phase distributions in red box. White bar in (A) indicates 2 mm. (B) and (C) Retrieved modulus and pahse distributions of Corn Stems and Umbrella Plant with the same FoV as 10 × , respectively. White bar in (B) indicates 1 mm.
To check the resolving capability of the experimental setup quantitatively, a USAF 1951 resolution target was used as the sample, and the reconstructions are shown in Figs. 3(A)-(C), where we can find that the fine details of the forth element of group six can be clearly resolved, corresponding to a resolution of 5.52 μm. Figure 3(D) shows the residual error of the reconstruction changing with the going on iterations, where we can find the reconstruction error reaches about 1% after only five iterations, ideally overcoming the second main shortcoming of common mechanical scanning PIE technique on the time consuming position-error-correction procedure.
Fig. 3 Certification with standard sample. (A)–(C) Retrieved amplitudes of 1951 USAF resolution test chart in different FoVs. Amplitude distributions of line pairs of the sixth group, forth element along the section line in both vertical and horizontal directions are shown in (C). (D) Residual error of the reconstruction changing with different iterations.
4. Conclusions
In this paper, DMD based PIE is proposed to overcome the shortcoming of common mechanical scanning based PIE for accurate quantitative phase imaging. By replacing the mechanical scanning of common PIE with the on/off switching of the micro-mirrors of DMD, the data acquisition can be speeded up at least 45 times, and thus remarkably relaxing the requirement of PIE on the stability of the imaging system and working environment. At the same time, since the position error of the micro-mirror of DMD fabricated with optical lithography is always less than 10nm, its influence on the reconstruction quality is negligible, and then the time consuming position-error-correction procedure is not required in the reconstruction procedure. Since the proposed DMD based PIE method has overcome the main disadvantage of common PIE to realize large FoV with very short data acquisition time, the application range of PIE is distinctively extended, and this DMD based PIE can find many important applications for the living cell and biological tissue culture or other cases requiring large field of view and fast imaging speed.
Acknowledgements
The work is supported by Natural Science Foundation of China (11404207, 11647144), Natural Science Foundation of Jiangsu Province of China (BK2012548, BK20130162), Fundamental Research Funds for the Central Universities (JUSRP115A14) and Local Colleges and Universities Capacity Building Program (15110500900, 14110500900).
References and links
1. M. Mir, B. Bhaduri, R. Wang, R. Zhu, and G. Popescu, “Quantitative Phase Imaging,” Prog. Opt. 57, 133–217 (2012). [CrossRef]
2. H. Majeed, M. E. Kandel, K. Han, Z. Luo, V. Macias, K. Tangella, A. Balla, and G. Popescu, “Breast cancer diagnosis using spatial light interference microscopy,” J. Biomed. Opt. 20(11), 111210 (2015). [CrossRef] [PubMed]
3. V. Bianco, P. Memmolo, M. Paturzo, A. Finizio, B. Javidi, and P. Ferraro, “Quasi noise-free digital holography,” Light Sci. Appl. 5(9), e16142 (2016). [CrossRef]
4. F. Merola, P. Memmolo, L. Miccio, R. Savoia, M. Mugnano, A. Fontana, G. D’Ippolito, A. Sardo, A. Iolascon, A. Gambale, and P. Ferraro, “Tomographic Flow Cytometry by Digital Holography,” Light Sci. Appl. 6(4), e16241 (2017). [CrossRef]
5. A. Ozcan and E. McLeod, “Lensless Imaging and Sensing,” Annu. Rev. Biomed. Eng. 18(18), 77–102 (2016). [CrossRef] [PubMed]
6. X. Tian, W. Yu, X. Meng, A. Sun, L. Xue, C. Liu, and S. Wang, “Real-time quantitative phase imaging based on transport of intensity equation with dual simultaneously recorded field of view,” Opt. Lett. 41(7), 1427–1430 (2016). [CrossRef] [PubMed]
7. X. Tian, X. Meng, W. Yu, X. Song, L. Xue, C. Liu, and S. Wang, “In–focus quantitative intensity and phase imaging with numerical focusing transport of intensity equation method,” J. Opt. 18(10), 105302 (2016). [CrossRef]
8. W. Yu, X. Tian, X. He, X. Song, L. Xue, C. Liu, and S. Wang, “Real time quantitative phase microscopy based on single–shot transport of intensity equation (ssTIE) method,” Appl. Phys. Lett. 109(7), 071112 (2016). [CrossRef]
9. X. Meng, H. Huang, K. Yan, X. Tian, W. Yu, H. Cui, Y. Kong, L. Xue, C. Liu, and S. Wang, “Smartphone based hand-held quantitative phase microscope using the transport of intensity equation method,” Lab Chip 17(1), 104–109 (2017). [CrossRef] [PubMed]
10. L. Waller, L. Tian, and G. Barbastathis, “Transport of Intensity phase-amplitude imaging with higher order intensity derivatives,” Opt. Express 18(12), 12552–12561 (2010). [CrossRef] [PubMed]
11. L. Tian, J. C. Petruccelli, and G. Barbastathis, “Nonlinear diffusion regularization for transport of intensity phase imaging,” Opt. Lett. 37(19), 4131–4133 (2012). [CrossRef] [PubMed]
12. S. S. Kou, L. Waller, G. Barbastathis, and C. J. Sheppard, “Transport-of-intensity approach to differential interference contrast (TI-DIC) microscopy for quantitative phase imaging,” Opt. Lett. 35(3), 447–449 (2010). [CrossRef] [PubMed]
13. L. Waller, Y. Luo, S. Y. Yang, and G. Barbastathis, “Transport of intensity phase imaging in a volume holographic microscope,” Opt. Lett. 35(17), 2961–2963 (2010). [CrossRef] [PubMed]
14. G. Zheng, R. Horstmeyer, and C. Yang, “Wide-field, high-resolution Fourier ptychographic microscopy,” Nat. Photonics 7(9), 739–745 (2013). [CrossRef] [PubMed]
15. Y. Zhang, W. Jiang, and Q. Dai, “Nonlinear optimization approach for Fourier ptychographic microscopy,” Opt. Express 23(26), 33822–33835 (2015). [CrossRef] [PubMed]
16. A. M. Maiden and J. M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109(10), 1256–1262 (2009). [CrossRef] [PubMed]
17. H. M. Faulkner and J. M. Rodenburg, “Movable Aperture Lensless Transmission Microscopy: A Novel Phase Retrieval Algorithm,” Phys. Rev. Lett. 93(2), 023903 (2004). [CrossRef] [PubMed]
18. H. M. Faulkner and J. M. Rodenburg, “Error tolerance of an iterative phase retrieval algorithm for moveable illumination microscopy,” Ultramicroscopy 103(2), 153–164 (2005). [CrossRef] [PubMed]
19. R. Gerchberg and W. Saxton, “A practical algorithm for the determination of the phase from image and diffraction plane pictures,” Optik (Stuttg.) 35, 237–250 (1972).
20. R. Gerchberg, “Super-resolution through error energy reduction,” J. Mod. Opt. 21(9), 709–720 (1974).
21. J. Rodenburg and H. Faulkner, “A phase retrieval algorithm for shifting illumination,” Appl. Phys. Lett. 85(20), 4795–4797 (2004). [CrossRef]
22. J. R. Fienup, “Reconstruction of an object from the modulus of its Fourier transform,” Opt. Lett. 3(1), 27–29 (1978). [CrossRef] [PubMed]
23. J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21(15), 2758–2769 (1982). [CrossRef] [PubMed]
24. J. M. Rodenburg, A. C. Hurst, and A. G. Cullis, “Transmission microscopy without lenses for objects of unlimited size,” Ultramicroscopy 107(2-3), 227–231 (2007). [CrossRef] [PubMed]
25. F. Zhang and J. Rodenburg, “Phase retrieval based on wave-front relay and modulation,” Phys. Rev. B 82(12), 121104 (2010). [CrossRef]
26. J. M. Rodenburg, A. C. Hurst, A. G. Cullis, B. R. Dobson, F. Pfeiffer, O. Bunk, C. David, K. Jefimovs, and I. Johnson, “Hard-x-ray lensless imaging of extended objects,” Phys. Rev. Lett. 98(3), 034801 (2007). [CrossRef] [PubMed]
27. F. Hüe, J. Rodenburg, A. Maiden, F. Sweeney, and P. Midgley, “Wave-front phase retrieval in transmission electron microscopy via ptychography,” Phys. Rev. B 82(82), 7171 (2010).
28. W. Yu, S. Wang, S. Veetil, S. Gao, C. Liu, and J. Zhu, “High-quality image reconstruction method for ptychography with partially coherent illumination,” Phys. Rev. B 93(24), 241105 (2016). [CrossRef]
29. F. Zhang, I. Peterson, J. Vila-Comamala, A. Diaz, F. Berenguer, R. Bean, B. Chen, A. Menzel, I. K. Robinson, and J. M. Rodenburg, “Translation position determination in ptychographic coherent diffraction imaging,” Opt. Express 21(11), 13592–13606 (2013). [CrossRef] [PubMed]
30. A. M. Maiden, M. J. Humphry, M. C. Sarahan, B. Kraus, and J. M. Rodenburg, “An annealing algorithm to correct positioning errors in ptychography,” Ultramicroscopy 120(5), 64–72 (2012). [CrossRef] [PubMed]
31. D. Claus and J. M. Rodenburg, “Pixel size adjustment in coherent diffractive imaging within the Rayleigh-Sommerfeld regime,” Appl. Opt. 54(8), 1936–1944 (2015). [CrossRef] [PubMed]
32. P. Sidorenko and O. Cohen, “Single-shot ptychography,” Optica 3(1), 9–14 (2016). [CrossRef]
33. X. Pan, C. Liu, and J. Zhu, “Single shot ptychographical iterative engine based on multi-beam illumination,” Appl. Phys. Lett. 103(17), 171105 (2013). [CrossRef]
34. S. Shin, K. Kim, J. Yoon, and Y. Park, “Active illumination using a digital micromirror device for quantitative phase imaging,” Opt. Lett. 40(22), 5407–5410 (2015). [CrossRef] [PubMed]
35. C. Kuang, Y. Ma, R. Zhou, J. Lee, G. Barbastathis, R. R. Dasari, Z. Yaqoob, and P. T. So, “Digital micromirror device-based laser-illumination Fourier ptychographic microscopy,” Opt. Express 23(21), 26999–27010 (2015). [CrossRef] [PubMed]
36. P. Ferraro, G. Coppola, S. De Nicola, A. Finizio, and G. Pierattini, “Digital holographic microscope with automatic focus tracking by detecting sample displacement in real time,” Opt. Lett. 28(14), 1257–1259 (2003). [CrossRef] [PubMed]
37. P. Memmolo, C. Distante, M. Paturzo, A. Finizio, P. Ferraro, and B. Javidi, “Automatic focusing in digital holography and its application to stretched holograms,” Opt. Lett. 36(10), 1945–1947 (2011). [CrossRef] [PubMed]
38. A. Pelagotti, M. Paturzo, M. Locatelli, A. Geltrude, R. Meucci, A. Finizio, and P. Ferraro, “An automatic method for assembling a large synthetic aperture digital hologram,” Opt. Express 20(5), 4830–4839 (2012). [CrossRef] [PubMed]
