Open Access
Add to BibSonomy
Abstract
A wide-field fluorescence microscope with a double-helix point spread function (PSF) is constructed to obtain the specimen’s three-dimensional distribution with a single snapshot. Spiral-phase-based computer-generated holograms (CGHs) are adopted to make the depth-of-field of the microscope adjustable. The impact of system aberrations on the double-helix PSF at high numerical aperture is analyzed to reveal the necessity of the aberration correction. A modified cepstrum-based reconstruction scheme is promoted in accordance with properties of the new double-helix PSF. The extended depth-of-field images and the corresponding depth maps for both a simulated sample and a tilted section slice of bovine pulmonary artery endothelial (BPAE) cells are recovered, respectively, verifying that the depth-of-field is properly extended and the depth of the specimen can be estimated at a precision of 23.4nm. This three-dimensional fluorescence microscope with a framerate-rank time resolution is suitable for studying the fast developing process of thin and sparsely distributed micron-scale cells in extended depth-of-field.
© 2017 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Three-dimensional (3D) optical microscopy capable of distinguishing depth discrepancies of the concerned components is drawing increasing interest in modern bio-medical imaging [1–4]. Conventional 3D imaging solutions are mostly implemented by successively scanning the focus of the imaging system through the interested axial region of the specimen. The simplest and most representative technique is the focal-stack (FS) method [5, 6], which needs no hardware modifications in a routine microscope. The extended depth-of-field (DOF) image and its depth information at every transverse position can be recovered with focus-recognition algorithms. To obtain higher sectioning power and signal-to-noise ratio (SNR), a variety of novel technologies have been developed, including laser scanning confocal microscopy (LSCM) [7, 8], structured illumination microscopy (SIM) [9–11], and light-sheet fluorescence microscopy (LSFM) [3, 12], etc. These technologies differentiate the in-focus and the out-of-focus components by introducing special illumination schemes, which ease the recovery process and eliminate the algorithm-related errors to extract the all-in-focus image and its depth map. However, the setups of these methods are complex and generally require elaborate calibration. In addition, the requirement of axial scanning in these systems severely decreases the time resolution, restraining their application in the dynamic situation.
In contrast to the active illumination methods to scan the focal plane throughout the specimen, the point spread function (PSF) engineering methods attempt to obtain an all-in-focus image within single snapshot by producing an elongated PSF, which was firstly implemented by Dowski et al. [13]. By generating a depth-invariant PSF, the extended DOF image of the object could be directly obtained via image deblurring algorithms. However, the recovered image suffers from abundant artifacts and contains subtle transverse translations [14] for 3D structures, and the object’s depth information is abandoned. Although several improved reconstruction algorithms [15, 16] have been reported to restrain the deconvolution artifacts, none of them can solve the problem fundamentally. Until several years ago, Zammit et al. [17] promoted an ingenious complementary kernel matching imaging method to recover the depth information from the transverse translation and to cancel the transverse translation in return. Unfortunately, this method requires for switching the complementary two phase masks, which sacrifices either the imaging speed or the system simplicity [18]. Recently, a single shot 3D imaging system was developed by Berlich et al. [19] in which a single thin, custom-built phase element was used to generate a spatially rotating PSF (termed as double-helix PSF, DH-PSF), which, incorporating with the cepstrum-based reconstruction algorithm they proposed, was able to realize fast 3D imaging. This setup was built on a macro-scale imaging system with limited magnification and the depth-of-field of the system was fixed.
In this paper, a wide-field fluorescence microscope with high numerical aperture (100 × , NA = 1.25) is constructed, extending the single shot, three-dimensional imaging concept to high numerical aperture fluorescence microscopy. A variable computer-generated hologram (CGH) is employed to generate the double-helix PSF, making the depth-of-field of the system adjustable. The impact of system aberrations on the double-helix PSF at high numerical aperture is analyzed to reveal the necessity of the aberration correction. A modified cepstrum-based reconstruction scheme is also presented to accommodate the new double-helix PSFs, significantly improving the precision of the depth estimation while simultaneously restraining the artifact of deconvolution. The imaging performance is verified by respectively observing the fluorescent beads and the bovine pulmonary artery endothelial (BPAE) cells.
2. Experimental setup
Figure 1 illustrates the optical configuration of the single shot, three-dimensional fluorescence microscope. The illumination beam from a solid-state laser (λ = 491nm, Calypso 491, Cobolt AB Inc., Sweden) is expanded by a factor of twenty-five with a telescope system consisting of Lens1 (f = 10mm) and Lens2 (f = 250mm). The collimated beam is then focused towards the back focal plane of the infinity-corrected objective lens (100 × , NA = 1.25, Nikon Inc., Japan) to provide an epi-illumination at the focal plane. The fluorescence emission light from the illuminated specimen is collected by the identical objective. The pupil phase of the emission light can be modulated by simply loading specific CHGs onto the spatial light modulator (SLM, 1920 × 1080pixels, Pluto ΙΙ, HoloEye Photonics AG, Germany) by relaying the back focal plane of the objective on the SLM with a 1:1 4f system composed of Lens4 and Lens5. In addition, a linear polarizer is placed before the 4f system to block the vertical polarization component rejected by the SLM. The modulated light is then filtered by a bandpass emission filter (520 ± 22nm, Semrock Inc., USA) and collected by the zoom lens, and the specimen is eventually imaged onto the cooled charge-coupled device (CCD) camera (3296 × 2472pixels, 14bit, 8051M-GE-TE, Thorlabs Inc., USA).
Fig. 1 Schematic of the single shot, three-dimensional fluorescence microscope. The expanded and collimated laser beam is projected in parallel onto the specimen as an epi-illumination source. The wavefront of the fluorescence beam emitted from the specimen is modulated by the spatial light modulator, which is later focused onto the CCD camera. The inset shows an example of the loaded CGH on the spatial light modulator. DM: longpass dichroic mirror (λc = 500nm), SLM: spatial light modulator, M1-M8: mirrors.
3. Methods
3.1 Generation of the double-helix PSF
The PSF engineering, achieved by placing a phase mask at the pupil plane of the imaging lens to encode the wavefront emerging from an imaging system, has been a routine technology in extended DOF imaging during the past decades [20–23]. The performance of the PSF-engineered system is mostly dominated by the adopted CGH and the aberration in the optical system. Since Greengard et al. [20] promoted the rotating PSF in 2006, the double-helix PSF has enjoyed great attention in multiple imaging technologies [24, 25] due to its high performance in depth estimation. The mainstream CGHs to generate the double-helix PSFs in these applications are generally derived from Gaussian-Laguerre (GL) modes, which were proved to be energy-efficient [26]. Unfortunately, the extended range of the PSF generated with this method is limited and cannot be fully controlled by varying the superimposed GL modes.
Superior to the GL-mode-based approach, a novel approach based on spiral phase (SP) to generating rotating PSFs [27–29] was claimed to be capable of providing more compact main lobes, while the rate of rotation can also be easily adjusted by changing the number of Fresnel zones. The outstanding feature of this approach makes it possible to get higher lateral resolution as well as adjustable range and sensitivity of depth estimation by simply changing the loaded CGHs on the SLM. The simplified form of the CGHs to generate double-helix PSFs is shown in the equation below:
where PN(ρ,φ) denotes the phase modulation at the polar coordinates (ρ,φ), R presents the maximal radius of the selected aperture, and N is the total number of the Fresnel zones. The phase mask is composed of a sequence of radial sampling of the spiral phase into the Fresnel zones. The rotation rate of the rotating PSF can be easily controlled by tuning the number of azimuthal phase sections [29].To demonstrate the effects of CGH type on the double-helix PSF under high numerical aperture, the intensity distributions of the 3D PSFs were numerically calculated using the Fresnel diffraction theory. Without loss of generality, we selected a tailored CGH superimposed by five GL modes with respective indices of (2,2), (4,6), (6,10), (8,14) and (10,18) being an example of the GL-mode-based approach (GL CGH), similar to that used in Ref [19]. To display the changing tendency of the PSFs generated by the spiral-phase-based approach (SP CGH), the total number of the Fresnel zones of 4, 6 and 8 are adopted. In keeping with the parameters of the single shot, three-dimensional fluorescence microscope shown in Fig. 1, the numerical aperture of the objective is set as 1.25 and the central wavelength is selected as 515 nm that approximates to the emission wavelength.
The simulative results are presented in Fig. 2. As expected, the rotation rate of the double-helix PSF with varied amounts of defocus decreases with the number of the Fresnel zones, which leads to an opposite change of the PSF’s extended range (Fig. 2(a)). Quantitative analysis of two key features of the PSFs, namely the rotation rate and the distance between the two main lobes, further demonstrates the superiority of the spiral-phase-based CGHs (Fig. 2(b) and 2(c)). When the rotation rate is analyzed as a function of axial position, the GL-mode-based PSFs deviate from linearity at high and low positions. In contrast, the rotation rate of the spiral-phase-based PSFs is linear at all axial positions, revealing a more convenient and precise conversion of the rotation angle and the depth. In addition, the inter-lobe distance for spiral-phase-based PSF is unaffected by defocusing, whereas for the GL-mode-based PSFs this distance varies by as much as 400nm, especially at high and low positions (Fig. 2(c)).
Fig. 2 The effects of CGH type on the double-helix PSF. Properties of the double-helix PSF generated by the GL-mode-based CGH (GL CGH) and the spiral-phase-based CGHs (SP CGH) for NA = 1.25, λ = 515nm. (a) Intensity distribution with varying amounts of defocus. (b) and (c) Variation of the inter-lobe distance and the rotation angle with varying amounts of defocus. (d) Radial and angular FWHM of the main lobes for the spiral-phase-based double-helix PSFs with different number of Fresnel zones. The right inset illustrates how the radial FWHM and the angular FWHM of the main lobe are determined from the PSF’s intensity distribution.
It is also worth mentioning that although the extended range of the PSF increases with the number of the Fresnel zones, the sizes of the main lobes enlarge, which leads to resolution loss of the imaging system. This is mainly due to the linear increase in radial FWHM with increasing number of Fresnel zones, even though the angular FWHM keeps invariant (Fig. 2(d)). As a hint, it is more reasonable to approximate the main lobes of the PSF with the elliptical Gaussian function than the circular Gaussian function in the restoration process, which will be discussed below.
3.2 Image formation and restoration
For the designed wide-field microscope, the image captured by the camera can be modeled as
where i(ξ,η) is the collected image, o(x,y;z) presents the object’s luminance distribution in three dimension, h(x,y;ξ,η;z) indicates the PSF of the system at the object point (x,y,z), and n(ξ,η) denotes the additive noise. The symbol * remarks the lateral convolution operation of the object and the corresponding PSF. If the system satisfies the isoplanatic condition, h(x,y;ξ,η;z) will remain invariant with (ξ,η). For the modulated double-helix PSF in our system, the shape and the distance of the main lobes can be treated as constant. Thus, the intensity distribution generated by a point-like object at the depth z can be approximated bywhere h0(x,y;0) presents one of the main lobes of the PSF at the focal plane, kθ(z) describes the linear dependency between the defocus distance z and the rotation angle θ, Δy indicates the distance of the two main lobes, and rot(h,θ) is the function to rotate the PSF h by an angle of θ around the center. Under the elliptical Gaussian distribution approximation of the main lobes, h0(x,y;0) can be further estimated as followingwhere σa and σr are dimensional parameters that could be respectively derived from the angular FWHM and radial FWHM of the main lobes.Different from the image obtained by using a standard Gaussian PSF which contains only a single image, the coded image obtained by the double-helix PSF consists of twin blurred images that are located with depth-related orientations. By decoding the orientation of each of the twin images at each transverse position, the object can be recovered using deconvolution algorithms with the corresponding PSF estimated with the decoded depth.
The image restoration process in this paper is based on the framework proposed by Berlich [19]. Some modifications are introduced to accommodate the new double-helix PSFs. The flowchart of the process is illustrated in Fig. 3. Before restoration, the key parameters of the corresponding double-helix PSF are experimentally measured, including the rotation rate, the inter-lobe distance, and the radial and angular dimensions of the main lobes. Besides, some preprocessing manners such as background removing and denoising are taken to minimize the impact of the white noise. To guarantee a seamless recombined image and minimize the ring artifacts caused by deconvolution of sharp edges, sliding two-dimensional Hann-windows are adopted to divide the field-of-view into sub-windows. It is assumed that all objects within a sub-window are located at uniform depth. Each sub-window will be processed separately and the final image is recovered by putting all the sub-windows together. The restoration process of each sub-window includes depth estimation and image reconstruction.
Fig. 3 Flowchart of the extended depth-of-field image recovery and the depth estimation process. The captured raw image is divided into a series of sub-windows which are processed in parallel. The restoration process of each sub-window includes depth estimation and image reconstruction. The depth information is estimated by determining the maximal polar angle corresponding to the most densely populated cepstrum in a specific ring-shaped window. Meanwhile, the PSF corresponding to this sub-window can be estimated with the depth. The image is recovered by deconvolution with the estimated PSF. Depth map and image of the whole field-of-view can be recovered with ordered recombination of the results from all sub-windows.
As demonstrated in Ref [19], the cepstrum-based algorithm is suitable to decode the orientation of the depth-related-oriented twin images. The calculated cepstrum of the windowed image shows two centrosymmetric peaks at a ring-shaped window specified by the polar radius [0.8Δy,1.2Δy], and the polar angle of the two peaks exactly equals the orientation of the double-helix PSF at the corresponding sub-window. However, these two peaks are often submerged by the noise if the current sub-window is short of object features. This significantly impacts the recognition precision of the polar angle.
To address this issue, a threshold (half of the maximum) was employed to wipe out the majority of noises which mostly remains low-level in the cepstrum. Afterwards, the cepstrum’s distribution density varying with the polar angle is calculated to portray the concentration of the cepstrum. Supposing the high-level noise distributes randomly, the polar angle of the two peaks can be found at the maximum node of the distribution density. The depth of the object is easily calculated via dividing the obtained polar angle θmax by the advance-measured parameter kθ(z).
Once the depth is determined, the corresponding PSF of the sub-window can be obtained by using the measured parameters of the PSF, as described in Eq. (3) and Eq. (4). Afterwards, the image of each sub-window could be recovered by deconvolution. It’s worth noting that an improper deconvolution algorithm will lead to significant artifacts in the recovered image. In our experiment, we found the Richardson-Lucy algorithm with suitable number of iterations performs much better than the Wiener-type filter used in Ref [19]. in suppression of the artifact for the non-negativity prior. A simple performance comparison of these two algorithms will be shown in the experimental results. In the end, after all the sub-windows are processed, a depth map and an extended DOF image can be recovered by recombining the results in specific order.
In the following, we simulate the procedure of the image formation and restoration to reveal the effect of the promoted algorithm. The simulated object is shown in Fig. 4(a). The six lines of texts are located at different depths so that they become increasingly out-of-focus toward the bottom. The simulated images with the conventional Gaussian PSF and the double-helix PSF (N = 6) are respectively illustrated in Fig. 4(b) and Fig. 4(c). In contrast with the conventional image, the defocused components of the DH-PSF-blurred image still remain bright. The extended depth-of-field image and its depth map are both recovered (see Fig. 4(d) and 4(e)) by applying the promoted algorithm to the simulative image blurred with the double-helix PSF. As expected, the contents in recovered image are consistent with that of the original object in Fig. 4(a) and the recovered depth map also agrees with the pre-set depths.
Fig. 4 Simulative results of image formation and restoration. (a) Simulated object composed of six lines of texts located at different depths. (b) and (c) Images formed with the conventional Gaussian PSF and the double-helix PSF (N = 6), respectively. (d) and (e) The recovered extended depth-of-field image and depth map from (c). The numeric aperture of the objective is set as 1.25 to keep in consistence with the experiment.
3.3 Influence of the aberrations
Maintaining an ideal shape of the engineered PSF in the image formation procedure is of great importance. Unfortunately, the defects of the system often produce aberrations, which will distort the PSF, especially when high numerical aperture objectives are used. To ascertain the influence of the different types of aberrations [30, 31] on the double-helix PSF, we calculated the intensity distribution of the double-helix PSF with N = 6 after imposing different types of aberrations shown in Fig. 5. The results show that the spherical aberration causes slight focal plane shift and generates a non-uniform rotation rate above and below the focal plane, while the coma aberration unbalances the energy of the main lobes, and the astigmatism aberration mainly brings about varying inter-lobe distance at different depths. All these deformations will affect the accuracy of depth estimation and moreover, deteriorate the deconvolution results.
Fig. 5 The impact of different aberrations on the double-helix PSF (simulated with N = 6, NA = 1.25, λ = 515nm). (a) The aberration-free PSF. (b)-(d) The distorted PSFs respectively calculated with primary spherical aberration, primary coma, and primary astigmatism. The top-right colorbar maps the intensity in the calculated images. To provide a quantitative impression of the distorted PSFs, the related data are attached on the bottom-right corner of each panel in (b), (c), and (d) respectively, which represent the rotation angle, the maximal intensity ratio of the main lobes, and the inter-lobe distance.
Generally, provided that the objective is aberration-free, coma and astigmatism of the system can be eliminated by careful alignment of the optical path, and spherical aberration can be minimized by matching the refractive index of the immersion medium (oil or water), coverslip, to that of the specimen. Nevertheless, due to the defect of the SLM’s production process, there often exist sorts of surface curvature on the reflective panel, which inevitably causes a deviation of the wavefront and leads to unwanted distortions of the PSF [32]. Therefore, whatever its source, aberration correction is an essential procedure, which will be discussed below.
4. Experimental results and discussion
Before examining a specimen, the engineered PSF of the microscope system was measured. A single, commercial fluorescent nanoparticle (F8803, Thermo Fisher Scientific Inc., USA) with a diameter of 100 nm is used as a probe. This particle is labeled with multiple, randomly oriented fluorescent dye molecules, which gives rise to its insensitivity to the polarization state of the excitation beam. To minimize the spherical aberration caused by the refractive index mismatch, the particle is firstly dried on the coverslip and then submerged in immersion oil.
To correct the aberrations of the system (mainly SLM introduced), a simple but efficient method is employed by loading an additional phase mask opposite to the phase error caused by the surface curvature of SLM. The method is to apply a single image of a focused doughnut PSF created by the SLM to calculate the corresponding distortion phase hologram by using the Gerchberg-Saxton (GS) algorithm [32]. Here, the doughnut PSF generated with a helical charge l = 2 is adopted for its high sensitivity to the aberration and the enough sampling rate for the hologram at the Fourier plane (Fig. 6(a)). The results show that the original doughnut PSF is distorted into two lobes (Fig. 6(b)), while the corrected one more closely resembles the simulation results (Fig. 6(c)). Analysis of the calculated hologram opposite to the phase error caused by the SLM shows that as expected, flatness deviation of the SLM panel mostly contributes to the astigmatism (inset in Fig. 6(c)).To demonstrate the effects of aberration correction, the double-helix PSF of N = 8 was measured at different depths. The unstable inter-lobe distance in the uncorrected PSF presents the astigmatism in the system (Fig. 6(e)), which will significantly hinder the image recovery process. With our correction, the symmetry and the orderliness of the PSF are significantly improved (Fig. 6(f)). The depth-of-field provided by the double-helix PSF with N = 8 is measured to be 3.6μm. Similarly, the depth-of-fields by the double-helix PSFs with N = 4, N = 6, N = 10 are respectively 2.1μm, 2.9μm, 4.6μm, verifying a 4~10 times extension of the depth-of-field compared with a standard microscope (typically 500nm).
Fig. 6 Experimental results of aberration correction. (a) Calculated intensity distribution of the adopted doughnut PSF with l = 2. (b) and (c) Measured intensity distribution of the doughnut PSF before and after aberration correction. The bottom-left inset of (c) shows the calculated hologram opposite to the aberration. (d) The vertical (blue) and horizontal (red) profile of the corrected doughnut PSF. (e) and (f) Measured intensity distribution of the double-helix PSF (N = 8) without and with aberration correction. The characteristic parameters including the rotation angle, the maximal intensity ratio of the main lobes, and the inter-lobe distance are respectively indicated at the top-right, top-left, and bottom-left corners of each panel.
As illustrated in Fig. 6(f), the diffraction zero-order remains near the focal plane (Δz = 0), which is mostly caused by the discrete structure of the SLM. For our system, we choose not to isolate the zero-order from the first-order with the frequently-used blazed grating phase. One reason is that the relatively wide spectrum of the fluorescence will give rise to serious chromatic aberration in the collected image if a blazed grating phase is appended. In addition, even if we ignore the chromatic aberration, blocking zero order also sacrifices the field-of-view of the system. Fortunately, the zero-order on the PSF is overall minor, as shown in Fig. 6(f), which brings little influence on the image reconstruction.
To demonstrate the extended depth-of-field provided by the double-helix PSF, we first observed a tilted slide densely covered with fluorescent beads of a diameter of 100nm (F8803, Thermo Fisher Scientific Inc., USA). In contrast to the image acquired with the conventional Gaussian PSF (Fig. 7(a)), the image recovered with the double-helix PSF (Fig. 7(b)) obviously provides more bright and clear image for the defocused beads, revealing the extended depth-of-field of the system.
Fig. 7 Experimental observation results of a tilted slide of fluorescent beads with a diameter of 100nm. (a) and (b) are respectively the raw images acquired with the conventional Gaussian PSF and the double-helix PSF (N = 10). (c) and (d) are the reconstructed images of (b) with the Wiener-type filter and the Richardson-Lucy algorithm, respectively. For comparison, the magnified views of the dotted boxed regions marked with ROI1 and ROI2 in (a)-(d) are respectively shown in (a1)-(d1), and (a2)-(d2).
In addition, to testify the effect of the deconvolution algorithm on the result, we recovered the image of Fig. 7(b) with both the Wiener-type filter (Fig. 7(c)) and the Richardson-Lucy algorithm (Fig. 7(d)). The only difference between the two recovery processes is the type of deconvolution algorithms. Obviously, the image recovered with the Richardson-Lucy algorithm contains much less artifact in our system than that recovered with the Wiener-type filter. Because the Richardson-Lucy algorithm needs iteration process compared with the Wiener-type filter, this will sacrifice the computing speed of the recovery process. However, if the number of iterations is set as 8~12, the cost time for the image recovery will only increase by 1~2 times considering the computing expense of other processes, which is acceptable for most cases.
Next, a section of bovine pulmonary artery endothelial (BPAE) cells (F36924, Thermo Fisher Scientific Inc., USA) was tested, which is tilted to provide an appreciable range of defocus. The F-actin of the cell labeled with Alexa Fluor 488 phalloidin is efficiently excited by the laser to emit green fluorescence and imaged by the camera. For the conventional Gaussian PSF, only the in-focus part of the specimen can be imaged clearly while the out-of-focus regions appear dark and blurry (Fig. 8(a) and 8(b)). In contrast, the image obtained with the double-helix PSF (N = 6) homogeneously bright in the entire FOV (Fig. 8(c)). Furthermore, the recovered object in Fig. 8(d) offers a sharp and bright view of the entire tilted specimen. When the Gaussian- and double-helix PSF-obtained images are compared in the magnified views, the results are even more striking. For the sub-regions (ROI1 and ROI2 in Fig. 8(a)), the specimen is only detectable in in-focus Gaussian PSF images (Fig. 8(a1) and 8(b2)), whereas in the double-helix images, the specimens in both sub-regions are visible and the F-actin filaments are clearly discernible (Fig. 8(d1) and 8(d2), respectively). Collectively, these results show that the recovered object contains more useful information than any single image with a Gaussian PSF, and in addition, the depth-of-field of the image system is extended properly.
Fig. 8 The double-helix PSF produces the extended depth-of-field recovered image, which is verified by the observation results of F-actin in BPAE cells. (a) and (b) are images captured by using the conventional Gaussian PSF as comparison. The two images are captured at different depths (1500nm apart). (c) The raw image acquired with the double-helix PSF (N = 6). (d) The recovered object image shown in (c). The magnified views of the dashed regions of interest marked with ROI1 (blue) and ROI2 (yellow) in (a)-(d) are respectively shown in (a1)–(d1), and (a2)–(d2), i.e., (a1), (b1), (c1) and (d1) are respectively zoomed in ROI1 from (a), (b), (c) and (d), while (a2), (b2), (c2) and (d2) are respectively zoomed in ROI2 from (a), (b), (c) and (d).
As described in Fig. 3, achieving a high quality recovered image mostly relies on the precise estimation of the PSF. In our system, the PSF is calculated with the estimated depth of the sub-window. Thus, the precision of the depth estimation is a decisive factor of the image quality. The corresponding depth map of the BPAE specimen is shown in Fig. 9. As expected, the whole slide is tilted with a specific slope. To provide a calibration on the reliability of the depth estimation, the depth map of the identical FOV is obtained again by axially moving the specimen downward 300nm (see Fig. 9(b)). The difference between the two measured maps is plotted as a histogram in Fig. 9(c), which presents a normal distribution. Gaussian fit of the statistical data gives a population standard deviation of 33nm. Considering the additive property of the statistical variance from the subtraction operation, the precision, namely the standard deviation of single measurement is estimated to be (33/)nm≈23.4nm.
Fig. 9 Estimated depth map corresponding to the FOV of Fig. 8(a) and 8(b) are estimated depth maps of an identical FOV, where map (b) is obtained after moving the specimen 300nm downward axially relative to that in (a). The patches marked with darkest blue in the depth map represent regions with no in-focus specimen. (c) presents the statistical histogram of the difference of the above two depth maps. Gaussian fit of the statistical data is calculated, implying a population standard deviation of 33nm. The precision of single measurement is thus calculated to be 23.4nm.
According to the linear equation z = θmax/kθ(z), the precision of the estimated depth primarily depends on the linear factor kθ(z) and the recognition precision of the polar angle. The linear factor kθ(z) is uniquely dominated by the total number of the Fresnel zones. Supposing the recognition precision of the polar angle is free from the total number of the Fresnel zones, increasing the total number of the Fresnel zones will on one hand increase the range of extended depth-of-field, yet on the other hand decrease the precision of the estimated depth.
Systematically, the recognition precision of the polar angle is sensitive to a number of factors, including the amount of the specimen’s features, the SNR of the captured image and the size of the sub-window. The influence of the former two is readily comprehensible, for both of them directly affect the SNR of the cepstrum. The more features the specimen contains, the more remarkable the two centrosymmetric peaks shows in the cepstrum. So the system is more suitable for specimens with rich sharp structures. Meanwhile, the SNR of the captured image is affected by the illumination power and the performance of the detector. Thus, the devices used in the system should be carefully selected. For the third factor, things get more complicated. On one hand, compressing dimension of the sub-window will improve the lateral resolution of the depth estimation. On the other hand, excessive small sub-window will be short of object features and end up with low fidelity of the cepstrum algorithm. To balance the lateral resolution and the estimation fidelity, we choose the size of the sub-window to be 5-15 times of the PSF distribution area. In our experiment, the PSF spreads nearly 15 pixels, therefore choosing the sub-window size as 150 pixels is reasonable. Meanwhile, the step size of the sliding sub-window is set as 50 pixels. Even all the relevant parameters are carefully chosen, small amount of artifacts still occur in the recovered results, as shown in Fig. 8. This is mainly attributed to the initial assumption that the object in single sub-window distributes at a uniform depth. In the future, we will adjust the recovery framework by optimizing the segment strategy and introducing the feedback mechanisms between the depth estimation and the deconvolution to solve this problem.
As demonstrated above, this imaging method is capable of obtaining the three-dimensional information of the specimen in a single snapshot and the depth range can be adjusted by simply changing the loaded CGH on the SLM. With the modified algorithm, an extended DOF image and its corresponding depth map can be obtained with single snapshot, which imply a time resolution equivalent of the camera’s maximal frame rate can be achieved while preserving the three dimensional information of the specimen.
5. Summary
We have built up a wide-field fluorescence microscope capable of obtaining three-dimensional information of the specimen in a single snapshot. The depth range can be adjusted by simply changing the loaded CGH on the SLM. To provide a guideline for aberration correction, the impact of the aberrations on the double-helix PSF at high numerical aperture were analyzed. By employing a modified cepstrum-based reconstruction scheme, we have achieved extended depth-of-field images and estimated depth maps, demonstrated by both simulation and imaging a tilted section of BPAE cells. This three-dimensional microscope with a framerate-rank time resolution is suitable for studying the fast developing process of thin and sparsely distributed micron-scale cells in extended depth-of-field.
Funding
This research was supported by the Natural Science Foundation of China (NSFC) (61522511, 81427802, 11474352, 61377008), the Natural Science Basic Research Plan in Shaanxi Province of China (Program No: 2016JZ020), and the National Institutes of Health Grant GM100156 to P. R. B.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References and links
1. H.-U. Dodt, U. Leischner, A. Schierloh, N. Jährling, C. P. Mauch, K. Deininger, J. M. Deussing, M. Eder, W. Zieglgänsberger, and K. Becker, “Ultramicroscopy: three-dimensional visualization of neuronal networks in the whole mouse brain,” Nat. Methods 4(4), 331–336 (2007). [PubMed]
2. W. R. Legant, L. Shao, J. B. Grimm, T. A. Brown, D. E. Milkie, B. B. Avants, L. D. Lavis, and E. Betzig, “High-density three-dimensional localization microscopy across large volumes,” Nat. Methods 13(4), 359–365 (2016). [PubMed]
3. P. J. Keller, A. D. Schmidt, A. Santella, K. Khairy, Z. Bao, J. Wittbrodt, and E. H. Stelzer, “Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy,” Nat. Methods 7(8), 637–642 (2010). [PubMed]
4. B. L. Scott and A. D. Hoppe, “Three-dimensional reconstruction of three-way fret microscopy improves imaging of multiple protein-protein interactions,” PLoS One 11(3), e0152401 (2016). [PubMed]
5. S. K. Nayar and Y. Nakagawa, “Shape from focus,” IEEE Trans. Pattern Anal. Mach. Intell. 16(8), 824–831 (1994).
6. X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, “Coded focal stack photography,” in Computational Photography (ICCP),2013IEEE International Conference on, (IEEE, 2013), 1–9.
7. T. Wilson, “Optical sectioning in confocal fluorescence microscopes,” J. Microsc. 154, 143–156 (1989).
8. H. J. Jeong, H. Yoo, and D. Gweon, “High-speed 3-D measurement with a large field of view based on direct-view confocal microscope with an electrically tunable lens,” Opt. Express 24(4), 3806–3816 (2016). [PubMed]
9. D. Dan, M. Lei, B. Yao, W. Wang, M. Winterhalder, A. Zumbusch, Y. Qi, L. Xia, S. Yan, Y. Yang, P. Gao, T. Ye, and W. Zhao, “DMD-based LED-illumination super-resolution and optical sectioning microscopy,” Sci. Rep. 3, 1116 (2013). [PubMed]
10. M. A. Neil, R. Juškaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22(24), 1905–1907 (1997). [PubMed]
11. X. Zhou, M. Lei, D. Dan, B. Yao, J. Qian, S. Yan, Y. Yang, J. Min, T. Peng, T. Ye, and G. Chen, “Double-exposure optical sectioning structured illumination microscopy based on Hilbert transform reconstruction,” PLoS One 10(3), e0120892 (2015). [PubMed]
12. B. C. Chen, W. R. Legant, K. Wang, L. Shao, D. E. Milkie, M. W. Davidson, C. Janetopoulos, X. S. Wu, J. A. Hammer 3rd, Z. Liu, B. P. English, Y. Mimori-Kiyosue, D. P. Romero, A. T. Ritter, J. Lippincott-Schwartz, L. Fritz-Laylin, R. D. Mullins, D. M. Mitchell, J. N. Bembenek, A. C. Reymann, R. Böhme, S. W. Grill, J. T. Wang, G. Seydoux, U. S. Tulu, D. P. Kiehart, and E. Betzig, “Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution,” Science 346(6208), 1257998 (2014). [PubMed]
13. E. R. Dowski Jr and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34(11), 1859–1866 (1995). [PubMed]
14. M. Demenikov and A. R. Harvey, “Image artifacts in hybrid imaging systems with a cubic phase mask,” Opt. Express 18(8), 8207–8212 (2010). [PubMed]
15. J. A. Conchello and M. E. Dresser, “Extended depth-of-focus microscopy via constrained deconvolution,” J. Biomed. Opt. 12(6), 064026 (2007). [PubMed]
16. S. Kim, S. Jun, E. Lee, J. Shin, and J. Paik, “Real-time bayer-domain image restoration for an extended depth of field (EDoF) camera,” IEEE Trans. Consum. Electron. 55(4), 1756–1764 (2009).
17. P. Zammit, A. R. Harvey, and G. Carles, “Extended depth-of-field imaging and ranging in a snapshot,” Optica 1(4), 209–216 (2014).
18. P. Zammit, A. R. Harvey, and G. Carles, “Practical single snapshot 3D imaging method with an extended depth of field,” in Computational Optical Sensing and Imaging, (Optical Society of America, 2015), CT2E. 2.
19. R. Berlich, A. Bräuer, and S. Stallinga, “Single shot three-dimensional imaging using an engineered point spread function,” Opt. Express 24(6), 5946–5960 (2016). [PubMed]
20. A. Greengard, Y. Y. Schechner, and R. Piestun, “Depth from diffracted rotation,” Opt. Lett. 31(2), 181–183 (2006). [PubMed]
21. F. Zhou, R. Ye, G. Li, H. Zhang, and D. Wang, “Optimized circularly symmetric phase mask to extend the depth of focus,” J. Opt. Soc. Am. A 26(8), 1889–1895 (2009). [PubMed]
22. Z. Fan and Q. D. Duong, “To extend the depth of field by using the asymmetrical phase mask and its conjugation phase mask in wavefront coding imaging systems,” Appl. Opt. 54(12), 3630–3634 (2015).
23. K. Chu, N. George, and W. Chi, “Extending the depth of field through unbalanced optical path difference,” Appl. Opt. 47(36), 6895–6903 (2008). [PubMed]
24. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. U.S.A. 106(9), 2995–2999 (2009). [PubMed]
25. D. Chen, B. Yu, H. Li, Y. Huo, B. Cao, G. Xu, and H. Niu, “Approach to multiparticle parallel tracking in thick samples with three-dimensional nanoresolution,” Opt. Lett. 38(19), 3712–3715 (2013). [PubMed]
26. S. R. P. Pavani and R. Piestun, “High-efficiency rotating point spread functions,” Opt. Express 16(5), 3484–3489 (2008). [PubMed]
27. R. Piestun and J. Shamir, “Generalized propagation-invariant wave fields,” J. Opt. Soc. Am. A 15(12), 3039–3044 (1998).
28. S. Prasad, “Rotating point spread function via pupil-phase engineering,” Opt. Lett. 38(4), 585–587 (2013). [PubMed]
29. M. Baránek and Z. Bouchal, “Optimizing the rotating point spread function by SLM aided spiral phase modulation,” in XIX Polish-Slovak-Czech Optical Conference on Wave and Quantum Aspects of Contemporary Optics, (International Society for Optics and Photonics, 2014), 94410N.
30. S. Ghosh, G. Grover, R. Piestun, and C. Preza, “Effect of double-helix point-spread functions on 3D imaging in the presence of spherical aberrations, ” in SPIE BiOS, (International Society for Optics and Photonics, 2011), 79041D–79041D–79049.
31. S. Ghosh and C. Preza, “Characterization of a three-dimensional double-helix point-spread function for fluorescence microscopy in the presence of spherical aberration,” J. Biomed. Opt. 18(3), 036010 (2013). [PubMed]
32. A. Jesacher, A. Schwaighofer, S. Fürhapter, C. Maurer, S. Bernet, and M. Ritsch-Marte, “Wavefront correction of spatial light modulators using an optical vortex image,” Opt. Express 15(9), 5801–5808 (2007). [PubMed]
