Abstract
In structured illumination microscopy (SIM), the imaging speed is expected to be increased to observe living cells. The conventional 2D SIM reconstruction algorithm (RA) requires nine raw images to reconstruct a super-resolution image. Here, we develop a partial-frequency-spectrum (PFS) reconstruction algorithm, based on the subtraction of frequency spectrum, which can reconstruct a super-resolution image by using six raw SIM images (two SIM images for each orientation). Our experiments of actin filament in bovine pulmonary artery endothelial (BPAE) cell imaging indicate that by the PFS algorithm, the frame rate increases. The PFS algorithm can resolve 120 nm in our experiment, which is equivalent to the reconstruction result of conventional 9-frame SIM. The PFS algorithm only requires the phase estimation of the three images. The reconstruction speed is about 5 times faster that of the conventional nine-images SIM method.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
In the past few decades, a variety of super-resolution fluorescence microscopy has been developed [1–5]. Structured illumination microscopy is one of the notable techniques because of the low excitation intensity, the high speed and a twofold resolution gain in linear form [2,6,7]. SIM is achieved by employing the modulated illumination of a sinusoidal pattern. The high-frequency information is shifted into the optical transfer function (OTF), separated from superimposed information components, then shifted back to the correct position to construct a super-resolution image [7].
Dynamic cells observation is one of the advantages of SIM, but the up-to-date imaging speed misses the details of cellular activities and produce undesired results [8–10]. In order to increase the frame rate, the optical system have been greatly improved, such as high-speed imaging devices [11], and the system synchronization optimization [11–13]. In the aspect of reconstruction algorithms (RA), conventional 2D SIM techniques need nine raw images [13–18], the nine exposures for one super-resolution image limit the time resolution of SIM to follow the activities of living cells. So, reducing the number of raw images required is a promising way to increase the SIM imaging speed. Recently, many 4-frame algorithms are developed to improve the frame rate [19–21]. These algorithms require longer calculation time (parallel computing) than the conventional 9-frame method. Lal et al. [22] demonstrate a reconstruction theory in the frequency domain and operate it on numerical simulations successfully. However, it shows little resolution enhancement in the reconstructed images from their experimental SIM images. It is a challenge to have a better trade-off between the spatial resolution and the reconstruction speed.
In this paper, we develop an imaging method, termed partial-frequency-spectrum (PFS) RA, which employs the similarity between wide-field images and raw SIM images to separate the high-frequency components from raw SIM images. PFS algorithm can reconstruct a super-resolution image using six raw SIM images in three orientations, two images with the phase shift of π in each orientation. We simulate the performance of our PFS RA and validate it with experimental raw data. The super-resolution images of bovine pulmonary artery endothelial (BPAE) cells indicate that the PFS algorithm can improve the frame rate. The reconstruction speed is five times faster than that of the conventional 9-frame method. The PFS image can resolve about 120 nm in the experiment, much better than Wiener-filtered images.
2. Reconstruction in the PFS algorithm
In a linear system without aberration, the recorded image D(r) can be written as
where H(r) is the point spread function (PSF) of the optical system, * is the convolution, S(r) is the sample fluorophore density, and I(r) is the illumination pattern. The sinusoidal structured light I(r)is expressed by Thus, the Fourier transform of Eq. (1) can be written asA schematic diagram of the reconstruction algorithm is shown in Fig. 1. A sample is illuminated by a sinusoidal pattern in three orientations. Two images (D1, D2) with the phase shift of π are caught in each orientation. D1 and D2 are able to synthesize a wide-field image Dw. The process in the Fourier domain can be expressed as
The specific method of the process II, the subtraction and shifting in frequency domain, is shown in Fig. 2 and Fig. 3. The ways of obtaining the high-frequency component in CH1 and CH2 are the same, so we take CH2 for example.
First, remove CL from ${\tilde{D}_1}(k)$, as shown in Fig. 3(a). The wide-field image ${\tilde{D}_w}(k)$ is used for CL, as the following equation
Next, we calculate the low frequency in CH1, which can be obtained by shifting ${\tilde{D}_{Wiener}}(k)$ (Wiener-filter of wide-field image) in frequency domain. Following the approach in [14], ${\tilde{D}_{Wiener}}(k)$ can be expressed as follows
Then, subtract the low-frequency spectrum ${\tilde{S}_{shift}}(k)$ from ${\tilde{D}_{1,2}}(k)$, as shown in Fig. 3(b). The process can be expressed as follows:
when the low frequency spectrum in CH1 is totally removed (2ξ=m/2), $|{\tilde{D}_{1.3}}(k)|$ corresponds to the minimum value. ξ can be obtained as follows:At last, the separated high-frequency components in RA need to deconvolute and move back to the original position. $R_{A} =\{ |k| < {K_{OTF}} \cap |k - p| < {K_{OTF}} \cap k \cdot p > 0\} $, consists of the high-frequency spectrum in CH2 and the overlapped part of Wiener-filtered image. This guarantees the continuity between the high and the low frequency spectrum. The spectrum of ${\tilde{S}_{shift}}(k)$ is distorted after deconvolution, as shown in Fig. 3(b). So, after the step above, residual spectrum appears at the position of Fig. 3(c) (the black dotted line), and a part of the residual spectrum is in RA. In order to suppress the influence of the residual spectrum, we use an estimated noise model Nb=ND3+ Nr in Wiener filter. ND3 is the average noise of ${\tilde{D}_{1.3}}(k)$. Nr can be written as
3. Simulation
We simulate the performance of the PFS algorithm. First, we analyze the performance of the PFS algorithm in different Gaussian noise level and use a circle pattern as a test sample, as shown in Fig. 4(a). Here are the simulation parameters, the excitation wavelength λex=488 nm, the emission wavelength λem=515 nm, the objective (100×, NA = 1.4), and the pixel size is 40 nm. According to the Abbe’s diffraction limit theory, the maximum spatial frequency of the simulation is 184nm−1. We set the illumination pattern frequency at 260nm−1, corresponding to 70% of the maximum frequency supported by OTF. Figure 4(b2) shows the simulation performance of PFS with different Gaussian noise level added. For comparison, Wiener-filtered images and conventional SIM are also shown in Fig. 4(b1) and Fig. 4(b3). The red line box in the low left corner of Fig. 4(a1) shows the original enlarged part without noise level.
The reconstructed images are shown in Fig. 4(b1)–4(b3). They indicate that for the noise level less than 10%, the resolution of the images reconstructed by the PFS algorithm is better than that of images by Wiener filtering. The PFS algorithm can achieve similar reconstruction results to the conventional 9-frames SIM. In the simulations, we use peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) to evaluate the reconstructed images, as shown in Fig. 4(c1)–(c2). As the noise ratio rises, the quality of PFS images deteriorate more obviously than that of conventional 9 images SIM method, but the PFS algorithm still performs better than Wiener filter.
We also calculated the reconstruction time in our computer (CPU Intel i5) and reconstruct the raw images (image size: 512*512) in Matlab. The PFS algorithm reconstruction time is about 15-17s, which is much faster than the 75-78s required by the conventional 9-frame SIM. Reducing the raw image size to 200*200, the reconstruction time of the PFS algorithm and the conventional 9-frame SIM are 3-3.5s and 13-14s, respectively. Under different datasets, the reconstruction speed is about 5 times faster than conventional SIM methods. Other algorithms that reduce the number of SIM images, require parallel computing and iterative recovery. Those take more time to reconstruct a super-resolution image. For example, the ordinary least squares algorithm, it takes nearly 12 minutes to reconstruct a super-resolution image (150*150) in our computer.
4. Experiments and results
In order to prove the feasibility of PFS algorithm in experimental raw SIM images, we prepared BPAE cells as the samples. The experimental conditions are as follows. The excitation wavelength λex=488 nm, the emission wavelength λem=515 nm, the numerical aperture NA = 1.49, and the pixel size is 60 nm. For comparison, we also reconstructed the experimental images using Wiener filter method. In order to ensure the validity of the data, we controlled the fringe frequency less than 86% of maximum frequency supported by the OTF.
Figure 5(a) is the reconstructed image by the PFS algorithm. The yellow line box in the low left corner shows the enlarged part. In order to show the details of the images, we enlarged the blue dotted box as shown in Fig. 5(b)–5(d). The wide-field image of Wiener filter method is shown in Fig. 5(b). Figure 5(c) shows the results of PFS algorithm. Figure 5(d) shows the results of conventional SIM. The details of the enlarged parts are mainly microtubules. Obviously, the reconstructed resolution of PFS algorithm is better than that of Wiener-filtered image, as shown in Fig. 5(b) and 5(c). The resolution of the image reconstructed by the PFS algorithm is closer to that of the conventional SIM. For a more intuitive view of the resolution enhancement, we plotted a section intensity of the red line in Fig. 5(a). Wiener-filtered only shows one peak, as depicted in the green line in Fig. 5(e). However, at the same position, the PFS algorithm and conventional SIM have two peaks. the PFS image can resolve 120 nm.
5. Conclusion
In this paper, we develop PFS algorithm for SIM to improve the frame rate. This algorithm can reconstruct a super-resolution image by using six raw low-resolution images instead of conventional nine images. In the PFS algorithm, the reconstruction is mainly completed by the subtraction in the frequency domain. The reconstruction process does not require multiple iterative recovery that mainly used in the determination of parameters. The simulations show that the reconstruction speed of PFS algorithm is about 5 times faster than that of conventional SIM methods. In the experiment, we employ PFS algorithm to reconstruct the super-resolution images of BPAE cells. In the experiments, we employ PFS algorithm to reconstruct the super-resolution images of BPAE cells. The reconstruction quality of PFS algorithm is close to the 9-frame algorithm, but the speed is much faster. Because there is a step in the PFS algorithm to estimate the spectrum by using Wiener filtering, the larger the noise ratio of the SIM image is, the more severe the distortion of the spectrum is. This is the characteristic of Wiener filtering. So, better signal-to-noise ratio of SIM images can bring better reconstruction effect. It is a promising algorithm for fast SIM imaging.
Funding
National Natural Science Foundation of China (61505143); National Key Scientific Instrument and Equipment Development Projects of China (2014YQ510403); National Basic Research Program of China (973 Program) (2013CB933804).
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. S. W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. 19(11), 780–782 (1994). [CrossRef]
2. R. Heintzmann and C. G. Cremer, “Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating,” Proc. SPIE 3568, 185–196 (1999). [CrossRef]
3. M. Hofmann, C. Eggeling, S. Jakobs, and S. W. Hell, “Breaking the diffraction barrier in fluorescence microscopy at low light intensities by using reversibly photoswitchable proteins,” Proc. Natl. Acad. Sci. U. S. A. 102(49), 17565–17569 (2005). [CrossRef]
4. S. T. Hess, T. P. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006). [CrossRef]
5. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006). [CrossRef]
6. M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Doubling the lateral resolution of wide-field fluorescence microscopy using structured illumination,” Proc. SPIE 3919, 141–150 (2000). [CrossRef]
7. M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198(2), 82–87 (2000). [CrossRef]
8. R. Förster, K. Wicker, W. Müller, A. Jost, and R. Heintzmann, “Motion artefact detection in structured illumination microscopy for live cell imaging,” Opt. Express 24(19), 22121–22134 (2016). [CrossRef]
9. X. Huang, J. Fan, L. Li, H. Liu, R. Wu, Y. Wu, L. Wei, H. Mao, A. Lal, and P. Xi, “Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy,” Nat. Biotechnol. 36(5), 451–459 (2018). [CrossRef]
10. J. Demmerle, C. Innocent, A. J. North, G. Ball, M. Muller, E. Miron, A. Matsuda, I. M. Dobbie, Y. Markaki, and L. Schermelleh, “Strategic and practical guidelines for successful structured illumination microscopy,” Nat. Protoc. 12(5), 988–1010 (2017). [CrossRef]
11. P. J. Keller, A. D. Schmidt, A. Santella, K. Khairy, Z. Bao, J. Wittbrodt, and E. H. Stelzer, “Fast, high-contrast imaging of animal development with scanned light sheet–based structured-illumination microscopy,” Nat. Methods 7(8), 637–642 (2010). [CrossRef]
12. P. Kner, B. B. Chhun, E. R. Griffis, L. Winoto, and M. G. Gustafsson, “Super-resolution video microscopy of live cells by structured illumination,” Nat. Methods 6(5), 339–342 (2009). [CrossRef]
13. X. Zhou, M. Lei, D. Dan, B. Yao, Y. Yang, J. Qian, G. Chen, and P. R. Bianco, “Image recombination transform algorithm for superresolution structured illumination microscopy,” J. Biomed. Opt. 21(9), 096009 (2016). [CrossRef]
14. S. A. Shroff, J. R. Fienup, and D. R. Williams, “Lateral superresolution using a posteriori phase shift estimation for a moving object: experimental results,” J. Opt. Soc. Am. A 27(8), 1770 (2010). [CrossRef]
15. A. Lal, C. Shan, and X. Peng, “Structured illumination microscopy image reconstruction algorithm,” IEEE J. Sel. Top. Quantum Electron. 22(4), 50–63 (2016). [CrossRef]
16. W. Kai, M. Ondrej, B. Gerrit, F. Reto, and H. Rainer, “Phase optimisation for structured illumination microscopy,” Opt. Express 21(2), 2032–2049 (2013). [CrossRef]
17. K. Wicker, “Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in Fourier space,” Opt. Express 21(21), 24692–24701 (2013). [CrossRef]
18. D. Li, L. Shao, B. C. Chen, X. Zhang, M. Zhang, B. Moses, D. E. Milkie, J. R. Beach, J. A. Hammer 3rd, M. Pasham, T. Kirchhausen, M. A. Baird, M. W. Davidson, P. Xu, and E. Betzig, “ADVANCED IMAGING. Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics,” Science 349(6251), aab3500 (2015). [CrossRef]
19. A. Lal, C. Shan, K. Zhao, W. Liu, X. Huang, W. Zong, L. Chen, and P. Xi, “A frequency domain SIM reconstruction algorithm using reduced number of images,” IEEE Trans. Image Process. 27(9), 4555–4570 (2018). [CrossRef]
20. F. Orieux, E. Sepulveda, V. Loriette, B. Dubertret, and J. C. Olivo-Marin, “Bayesian estimation for optimized structured illumination microscopy,” IEEE Trans. Image Process. 21(2), 601–614 (2012). [CrossRef]
21. S. Dong, P. Nanda, R. Shiradkar, K. Guo, and G. Zheng, “High-resolution fluorescence imaging via pattern-illuminated Fourier ptychography,” Opt. Express 22(17), 20856–20870 (2014). [CrossRef]
22. A. Lal, X. S. Huang, and P. Xi, “A frequency domain reconstruction of SIM image using four raw algorithm,” Proc. SPIE 27(9), 4555–4570 (2018). [CrossRef]