Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

One-step deconvolution for multi-angle TIRF microscopy with enhanced resolution

Open Access Open Access

Abstract

Total internal reflection fluorescence microscopy (TIRF microscopy) uses a rapid decay of evanescent waves to excite fluorophores within several hundred nanometers (nm) beneath the plasma membrane, which can effectively suppress excitation of fluorescence signals in the deep layers. From image stacks obtained with a plurality of different incident angles, a three-dimensional spatial structure of the observed sample can be reconstructed by a Multi-Angle-TIRF (MA-TIRF) algorithm that provides an axial resolution of ~50 nm. Taking into account the point spread function (PSF) of the TIRF microscopes, we further increase its lateral resolution by introducing a fast deconvolution algorithm into the reconstruction of MA-TIRF data (DMA-TIRF), which is approached in just one step of minimizing the reconstruction function. We also introduce a TV regularization term in the deconvolution algorithm to suppress artifacts induced by the excessive noise. Therefore, based on the hardware of existing MA-TIRF microscopes, the proposed DMA-TIRF algorithm has achieved lateral and axial resolutions of ~200 and ~50 nm, respectively.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Fluorescence microscopy is instrumental for the visualization of intricate structures and dynamics in cells and animals. Besides resolution, the contrast of the fluorophore excitation determines the ability of microscopy to resolve densely packed structures. As compared to other types of fluorescence microscopy, total internal reflection fluorescence microscopy (TIRF) only excites labeled fluorescent signal close to the glass coverslip, thus provides images of high signal-noise contrast. However, TIRF microscopy only captures two-dimensional images and provides no axial information. On the other hand, fluorescence microscopy techniques such as wide-field microscopy, confocal microscopy and most of super-resolution (SR) microscopy exhibited axial resolution inferior to the lateral axis. By rapidly switching among multiple incident angles of the TIRF illumination (MA-TIRF), it has been shown that an axial resolution of ~50 nm is possible in live cells [1,2]. However, given that raw images are always convolution of fluorescence emission image with the point spread function (PSF) of the microscopy, the lateral resolution in previous studies did not reach the diffraction limit yet, as neither method deblurred the raw image with a deconvolution step. Although a recent paper tried to combine deconvolution into the reconstruction pipeline [3], the authors separated the lateral deconvolution and the axial reconstruction into two steps. The first step is the initial lateral deconvolution of raw images, and then a structure extraction procedure to generate a binary mask. The second step is the multiplication of the binary mask with the axial reconstruction of MA-TIRF to achieve the final MA-TIRF reconstruction. However, given that the binary mask is generated by thresholding the deconvolved images, the final reconstruction results depends on properly adjusting thresholds during the structure extraction procedure. Therefore, this pipeline may suffer from reconstruction artifacts.

Here we have seamlessly incorporated a deconvolution algorithm into the axial reconstruction by including the PSF information in the optimization of the axial information assignment. Equipped also with a Wiener inverse filter and the related regularization terms in our pipeline, our microscopy is able to process raw images of low signal-to-noise ratio but still achieves lateral and axial resolutions of ~200 nm and ~50 nm with few artifacts, respectively. By maximally exploring the potential of graphic processing unit (GPU) using MATLAB, 3D results can be reconstructed from raw images in several seconds, accelerated 5 times compared to running on the CPU alone.

2. Theory and reconstruction of DMA-TIRF

2.1 Optical setup

The schematic of DMA-TIRF microscope is shown in Fig. 1, which was based on a commercial microscope (IX81, Olympus) equipped with a TIRF objective (UAPON 100XOTIRF, Olympus). Laser (Sapphire 488LP, coherent) was used as light source. An acoustic optical tunable filter (AOTF) was used to control the intensity of the laser. A collimating lens was used to couple the laser to the single-mode fiber. The output laser beam was collimated using an off-axis parabolic mirror (L1). Then the beam was reflected from a two-axis MEMS (A1B2.2, Mirrorcle Tech), in which the ring illumination TIRF microscopy was generated by two sinusoidal input voltages with a phase difference of 90 degrees. Next the light beam was expanded by Lens 2 (WHN10X, Olympus) and Lens 3 successively. Lens 2 is an eyepiece that can reduce the field curvature by 2 times compared with achromatic lens. The angular dispersion of the incident angle will be as low as ± 0.3° for the incident angle range. The tube lens (L4) focuses the light at the back focus plane of the objective lens (Apo N 100X/1.7 HI Oil, Olympus). The emitted fluorescence was captured by an electron-magnifying charge coupled device (EMCCD iXon 897U, Andor). The whole system was controlled by the software written in LABVIEW.

 figure: Fig. 1

Fig. 1 Schematic illustration of the MA-TIRF setup. AOTF: acoustic optical tunable filter, M: mirror, L1-L5: Lens 1-Lens 5, MEMS: Microelectromechanical systems, DM: dichroic mirrors, Obj: objective lens.

Download Full Size | PDF

2.2 Forward model with deconvolution

The intensity of evanescent waves I(z, φ, θ) along the illumination depth z, the azimuth φ, and the incident angle θ decays exponentially with the distance from the interface in TIRF microscopy [4]:

I(z,φ,θ)=I0(0,φ,θ)ezd
where I0(0, φ, θ) is the illumination intensity at the depth z = 0 [1,4]:
I0(0,φ,θ)=12cos2φ4cos2θ(2sin2θnr2)nr4cos2θ+sin2θnr2+12(1+sin2φ)4cos2θ1-nr2
where nr is the index ratio nr = n2/n1, n1 is the reflection index of glass, and n2 is the reflection index of medium. The parameter d is denoted as:
d=λ4π1n12sin2θn22,
where λ is the excitation wave length.

The emission distribution E(x, y, θ) results from the integration of an object f ′(x, y, θ) excited by the illumination of the evanescent wave:

E(x,y,θ)=02π0I(z,φ,α)ρ(αθΩ/cosθ)f(x,y,z)dαdzdφ,
where ρ is a Gaussian distribution of the incident angle α, taking in account the divergence Ω of the Gaussian beam around the angle θ. I(z, φ, α) is the intensity of evanescent wave, considering that incident angle α is a Gaussian distribution around the angle θ. f’(x, y, z) is the sample density of the fluorophores corresponding to the coordinate (x, y, z).

Finally, the observed image g′(x, y, θ) captured by the microscopy [1] is a result of the convolution of the emission distribution E(x, y, θ) with the emission PSF according to the coordinate (x, y) and incident angle θ:

g(x,y,θ)=psfE(x,y,θ),
where g′(x, y, θ) are images captured by the camera with the different incidence angles θ, psf is the emission point spread function of the TIRF microscopy, and the * is the convolution operator. By capturing TIRF images with m different incidence angles, raw data with different θ can be used to reconstruct axial information [1].

Although fluorophores deeper than 300 nm beneath the coverslip were illuminated in our MA-TIRF microscope, we only reconstructed structures within 300 nm away from the coverslip. Given that full width half maximum (FWHM) of the PSF in axial direction is about 500 nm, fluorescent puncta at different depths within the 300 nm plane would not be significantly different in fluorescence profiles. Therefore, 2D PSF is a proper approximation for the deconvolution proposed here. Then the Eq. (5) is shown below:

g(x,y,θ)=psf(x,y)02π0I(z,φ,α)ρ(αθΩ/cosθ)f(x,y,z)dαdzdφ.

Equation (6) could be discretized along the z direction as follows:

gθ=psfz=z1znhz,θfz,
where hz,θ is the reconstruction parameter that is computed as the integral of the I(z,φ,α)ρ(αθΩ/cosθ) according to the different depth z1~zn, and n is the number of axial layers. For a simplified representation of the above equation in m different incident angles θ, the Eq. (7) was represented as follows:
g=psf(Hf),
where g=[g1,g2,...,gm]T is the set of gθ with m different incidence angles θ, f=[f1,f2,...,fn]T is the set of fz at different depths, and H is the n columns and m rows matrix of the parameter hz,θ.

2.2.1 Reconstruction algorithm

To suppress reconstruction artifacts, we introduced the TV regularization and nonnegative constraint in the object function, defined as follows:

minfζ2gpsfHf22+D(f)+(f),
where ζ is the parameter of fidelity term, D(f) is the TV term D(f) = ||fx||1 + ||fy||1, and ∏ is the indicator function to impose the non-negativity constraint [5], which is denoted as follows:

(f)={0,f0+,f<0.

To efficiently optimize Eq. (9), we proposed an optimization method based on the variable splitting technique (half quadratic splitting) [6]. By introducing an auxiliary variable w, Eq. (9) can be reformulated as follow:

minf,wζ2gpsfw22+μ2wHf22+D(f)+(f),
where μ is a penalty parameter that was fixed in iterations for each individual sample in our experiments, but varied between different samples. We also introduced two auxiliary variables v and t to decouple the TV regularization and the nonnegative constraint:

minf,w,v,tζ2gpsfw22+μ2wHf22+D(t)+λ2ft22+(v)+β2fv22.

To optimize Eq. (12), we defined a Bregman distance BQp associated to a function Q about the variable v and t [7,8]:

BQp(v,v(k),t,t(k))=Q(v,t)Q(v(k),t(k))pv(k),vv(k)pu(k),uu(k),
where
Q(v,t)=D(t)+(v),
and
pt(k+1)=pt(k)λ(t(k+1)f(k+1))pv(k+1)=pv(k)λ(v(k+1)f(k+1)),
where the superscript k is the iteration number.

Equation (12), in which D(t) + ∏(v) is replaced by the function Q(v, t), is reformulated as follow:

minf,w,v,tQ(v,t)+ζ2gpsfw22+μ2wHf22+λ2ft22+β2fv22,
in which Q(v, t) can be optimized as a series of simpler minimization problems in the form of optimizing Bregman distance BQp:

minf,w,v,tBQp+ζ2gpsfw22+μ2wHf22+λ2ft22+β2fv22.

According to the split Bregman method [9], Eq. (17) is equivalent to:

minf,w,v,tζ2gpsfw22+μ2wHf22+D(t)+(v)+λ2ft+bt(k)22+β2fv+bv(k)22bt(k+1)=bt(k)+f(k+1)t(k+1)bv(k+1)=bv(k)+f(k+1)v(k+1).

Solved from the above Eq. (18), the global scheme of the reconstruction is presented as following iterative scheme:

minwζ2gpsfw22+μ2wHf(k)22,
minfμ2wHf22+λ2ft(k)+bt(k)22+β2fv(k)+bv(k)22,
minv(v)+β2f(k+1)v+bv(k)22,
mintD(t)+λ2f(k+1)t+bt(k)22,

The minimization of the iterative scheme for Eq. (19.1) to Eq. (19.4) is divided as four steps.

The first step is to minimize the objective function of variable w in Eq. (19.1), which can be solved directly as a Wiener filter:

w(k+1)=ifft(ζOTFfft(Y)+μfft(Hf(k))ζOTFOTF+μ+c),
where OTF is the discrete Fourier transform of psf, the optical transfer function of the MA-TIRF microscopy; † is the conjugate operator, fft represents the Fast Fourier transform, ifft is the inverse Fast Fourier transform, and c is the parameter of Wiener filter, which was set empirically [10].

The second step is to minimize the objective function of variable f in Eq. (19.2), which is solved directly as:

f(k+1)=[μHTH+(γ+λ)I]1[μHTw(k+1)+λ(t(k)bt(k))+γ(v(k)bv(k))].

The third step is to minimize the objective function of variable v in Eq. (19.3), which is solved independently using a shrinkage formula via minimization of following function:

minv(v)+β2f(k+1)v+bv(k)22,
of which the minimizer is v(k+1)=f(k+1)+bv(k) when f(k+1)+bv(k)0, and v(k+1) = 0 when f(k+1)+bv(k)<0:

v(k+1)=max(f(k+1)+bv(k),0).

The last step is to minimize the objective function about TV regularization term in Eq. (19.4), which is not a necessary procedure in the reconstruction. If the noise is low in the raw images, the reconstruction of Eq. (20) to Eq. (23) by Wiener DMA-TIRF is an effective way to suppress artifacts. Otherwise, the TV DMA-TIRF method, of which the objective function contains an additional variable t in Eq. (19.4), which can be solved as a typical optimization of denoising process with TV penalty [9].

We substitute D(t) = ||tx||1 + ||ty||1 into this function, and rewrite it into the equivalent form:

minttx1+ty1+λ2f(k+1)t+bt(k)22.

We use the split Bregman method to optimize this function by introducing a new variable l to approximate the partial derivatives of the TV penalty, and obtained the following unconstraint problem:

mintlx1+ly1+λ2f(k+1)t+bt(k)22+γ2lxtx22+γ2lyty22,
where γ is the parameter of penalty function. Using the split Bregman method, Eq. (25) can also be rewritten as follows:
mintlx1+ly1+λ2f(k+1)t+bt(k)22+γ2lxtxbx(k,i)22+γ2lytyby(k,i)22,
where i is the sequence of the iterative minimization of the variable t, and the first order partial derivative bx(k,i) and by(k,i) in different axis directions are used to reduce the computational complexity in the iteration and could be described as:

bx(k,i+1)=bx(k,i)+(tx(k,i+1)lx(k,i+1))by(k,i+1)=by(k,i)+(ty(k,i+1)ly(k,i+1)).

Through the Split-Bregman method, t could be directly solved as:

t(k,i+1)=ifft(fft((x)T(lx(k,i)bx(k,i))+(y)T(ly(k,i)by(k,i)))+λγ(f(k+1)+bt(k))|fft(x)|2+|fft(y)|2+λγ),
where x and y is the first order derivations operator matrix in x and y direction and was written as x=[1,1] and y=[1,1]T.

Variables lx and ly represent first order partial derivatives in x and y directions of the variable l in Eq. (26), and can be solved by:

lx(k,i+1)=shrink(tx(k,i+1)+bx(k,i),1γ):={(tx(k,i+1)+bx(k,i))1γ,tx(k,i+1)+bx(k,i)(1γ,)0,tx(k,i+1)+bx(k,i)(1γ,1γ)1γ+(tx(k,i+1)+bx(k,i)),tx(k,i+1)+bx(k,i)(,1γ),
and

ly(k,i+1)=shrink(ty(k,i+1)+by(k,i),1γ),

2.2.2 Pseudo-code chart of DMA-TIRF

In our algorithm (Table 1), the initial T1 is the tolerance and Nk is the maximum number of iterations in the optimization. The reconstruction took ~3 s by Wiener DMA-TIRF method without TV penalty (Wiener DMA-TIRF) and ~328 s by DMA-TIRF method with TV penalty (TV DMA-TIRF) for a desktop PC equipped with a 3.6-GHz i7 processor and a 1060Ti NVidia GPU to reconstruct an DMA-TIRF image stack of 512 × 512 × 10 pixels using the MATLAB procedures provided here.

Tables Icon

Table 1. The implementation procedure of our DMA-TIRF algorithm is summarized below.

3. Experiments

3.1 Simulation experiment

First, we benchmarked our algorithm with MA-TIRF algorithm using simulation with designed ground-truth structures. For simulation, data were excited by 10 different incident angles, convolved with the actual PSF of our TIRF microscope, and then captured by the camera. We added a background value of 99 a.u. to all pixels, in combination with Gaussian noise of a standard deviation at 50 and 500 a.u. to simulate images with low or high noise levels. These images were then reconstructed without (MA-TIRF) and with deconvolution (DMA-TIRF) to verify the validity of reconstruction algorithm on samples with ground-truth. We simulated two types of structures to test the fidelity of the proposed deconvolution pipeline. The first set included an overlay of vesicles, bars and point structures. We simulated a vesicle with lateral diameter of 360 nm, and only the outer membrane was labeled with fluorescence. As shown in Fig. 2(a), although both algorithms were able to reconstruct images of the vesicle at different planes, images obtained with DMA-TIRF exhibited much improved lateral resolution. The second ROI encircled with the solid square is two points, of which the points were separated in the lateral axis by 183 nm (Fig. 2(b)). These points were blurred along lateral axis in the TIRF image and also were blurred in the reconstruction with MA-TIRF, but became separated by the DMA-TIRF (Fig. 2(b) and 2(c)), highlighting the benefit of improving lateral resolution with deconvolution. Finally, we tested the ability of DMA-TIRF in discriminating axial-spaced structures. Three paralleled bars that were separated laterally by 260 nm (Fig. 2(d)), were repeatedly placed at 30 nm and 70 nm depths, and were also detected axially after DMA-TIRF reconstruction (Fig. 2(e)). Similarly, two points placed axially (50 nm in axial distance, red point in Fig. 2(b)) were reconstruct at their correct depths as compared to the ground truth (Fig. 2(e)), validating the method in correctly reconstructing axial information.

 figure: Fig. 2

Fig. 2 Reconstruction of simulated data with ground-truth known. (a) Simulations of the overlay of vesicles. The 1st column is the TIRF image with the minimum incident angle. The 2nd to 4th columns are ground-truth (the 2nd column), reconstructions by the MA-TIRF method (the 3rd column), and the DMA-TIRF method (the 4th column). Each row in the 2nd~4th columns presented along the axial distribution at 10 nm depth (the 1st row), 30 nm depth (the 2nd row), 50 nm depth (the 3rd row), 70 nm depth (the 4th row), 90 nm depth (the 5th row), and 110 nm depth (the 6th row). Scale bar: 0.5 μm. (b) The ground-truth of point structures and their reconstructions via MA-TIRF and DMA-TIRF methods. (c) The unitary lateral intensities from the ground-truth image and reconstructions from MA-TIRF and DMA-TIRF methods along the yellow line in (b). (d) The ground-truth, MA-TIRF and DMA-TIRF reconstructions of three paralleled bars. (e) The unitary intensities of two ROIs along the axial direction. The max value of ROI encircled within the yellow rectangle box in (d) is normalized to 1 a.u., while the max value of ROI marked with red point in (b) is normalized to 2 a.u..

Download Full Size | PDF

In the second experiment, we simulated a curved structure in the three-dimensional space, which mimicked the actin or tubulin filaments in cells (Fig. 3(a)). To simulate the noise in biologic sample, Gaussian noises of a standard deviation at 50 and 500 a.u. were added to the simulation data. Again, DMA-TIRF reconstruction generated an image with better lateral resolution than that reconstructed with the MA-TIRF method (Fig. 3(b) and 3(c)). DMA-TIRF method without TV penalty was able to generate high quality reconstructions in raw image with high SNR (the 2nd column, Fig. 3(b)), but produced a lot of artifacts in reconstructing the raw image of low SNR (iii in Fig. 3(d)). The previous two-step method (deconvolution and structure extraction procedure followed by the axial reconstruction with TV penalty) [3] could provide a similar extension in lateral resolution (Fig. 3(c)). However, in reconstructing raw images with low SNR, it was prone to reconstruction artifacts and exhibited a slightly lower structural similarity index (SSIM) (ii in Fig. 3(d), and Fig. 3(e)). In contrast, using a TV penalty in the DMA-TIRF reconstruction algorithm could suppress the artifacts and improve the reconstruction quality (Fig. 3(d) and 3(e)).

 figure: Fig. 3

Fig. 3 Reconstruction of a simulated curved line structure. (a) The ground-truth of the curved line structure in 3D. (b) Reconstructions of MA-TIRF (left) and DMA-TIRF (right) from images corrupted with Gaussian noise of a standard deviation at 50 a.u.. (c) Lateral distributions of the ground-truth, MA-TIRF, two-step method and Wiener DMA-TIRF reconstructions along the yellow line in (a). (d) Reconstructions of MA-TIRF (i), two-step method (ii), DMA-TIRF (iii) and TV DMA-TIRF (iv) from images corrupted with Gaussian noise of a standard deviation at 500 a.u.. Scale bar: 0.5 μm. (e) SSIMs of five different ROIs reconstructed with MA-TIRF, two-step, DMA-TIRF and TV DMA-TIRF methods as compared with the ground-truth.

Download Full Size | PDF

3.2. Experiments with biological samples

Actin filaments in live HUVECs were labeled with Lifeact-EGFP, imaged by TIRF microscope of multiple incident angles (Fig. 1), and reconstructed by MA-TIRF and DMA-TIRF methods without penalty. Because actin is mainly distributed in the cytosol, we used a refractive index of 1.36 in the reconstruction [11]. Similar to the simulation, TIRF images were blurred due to convolve with the PSF and structures overlaid in axial direction (Fig. 4(a)). MA-TIRF reconstruction provided an axial resolution as compared to the TIRF image, however, significant increase in lateral resolution manifested only in the image reconstructed with DMA-TIRF, in which two laterally intersecting actin filaments were clearly separated (Fig. 4(b) and 4(c)). Reconstruction without deconvolution (MA-TIRF) yielded an axial resolution similar to that obtained with DMA-TIRF, as shown by the images in which different depths (from 0 to 300 nm) were coded with different colors (Fig. 4(d) and 4(e)).

 figure: Fig. 4

Fig. 4 Reconstructions of filament actin structures labeled by lifeact-EGFP in the HUVEC cell. (a) One example of multiple-angles TIRF raw images, scale bar 2 μm. (b) The zoom-ins of MA-TIRF and DMA-TIRF reconstructions at the square box in (a), scale bar 0.2 μm. (c) Lateral distributions of MA-TIRF and DMA-TIRF reconstructions along the yellow line in (b). The distance of two peaks of double-Gaussian fitting is 204 nm. (d) The color-map of the reconstruction with the MA-TIRF method. (e) A color-map to highlight different axial distributions of filament actin structure obtained with the DMA-TIRF method without TV penalty.

Download Full Size | PDF

Next, we examined clathrin coated pits (CCPs) labeled by clathrin-EGFP in live insulin-secreting INS-1 cells [12]. Due to the excessive noise in the raw image, reconstruction using the MA-TIRF method was corrupted with artifacts (Fig. 5(a)), which could be relieved by the incorporation of TV penalty (Fig. 5(b)). However, overlapped two CCPs could not be clearly resolved by MA-TIRF related methods. On the other hand, deconvolution without TV penalty (Fig. 5(c)) and with TV penalty (Fig. 5(d)) significantly improved the lateral resolution from ~291 nm to ~195 nm (N = 8) (Fig. 5(e) and 5(g)). Although all methods provided a ~52 nm (N = 10) axial resolution (Fig. 5(f) and 5(g)), only DMA-TIRF method with TV penalty generated high-resolution 3D reconstruction with minimal artifacts.

 figure: Fig. 5

Fig. 5 Reconstructions of CCPs. (a)-(d) Reconstruction results with different methods. (a) The reconstruction by MA-TIRF method without TV penalty, scale bar: 2 μm. The large square box is the corresponding zoom-in of the small ROI, scale bar: 0.2 μm; (b) The reconstruction by MA-TIRF method with TV penalty; (c) The reconstruction by the DMA-TIRF method without TV penalty; (d) The reconstruction by the DMA-TIRF method with TV penalty. (e) Gaussian fittings to measure lateral resolutions yielded by the DMA-TIRF and the MA-TIRF methods with TV penalty. (f) Gaussian fittings to measure the axial resolution yielded by the DMA-TIRF and the MA-TIRF methods with TV penalty. (g) Lateral and axial resolutions computed from Gaussian fittings.

Download Full Size | PDF

4. Method

The HUVECs were isolated and cultured in M199 medium (Thermo Fisher Scientific, 31100035) supplemented with fibroblast growth factor, heparin, and 20% fetal bovine serum (FBS) or in ECM medium containing endothelial cell growth supplement (ECGS) and 10% FBS. The cells were infected with a retrovirus system to express Lifeact-EGFP. The transfected cells were cultured for 24 h, detached using trypsin-EDTA, seeded onto poly-L-lysine-coated coverslips, and cultured in an incubator at 37°C with 5% CO2 for an additional 20-28 h before the experiments.

INS-1 cells were cultured as described previously [12] and transfected with Clathrin-EGFP using the Lipofectamine 2000 reagent (Thermo Fisher Scientific, 11668019) according to the manufacturer’s instructions. After transfection, the cells were detached using trypsin-EDTA, seeded onto poly-L-lysine-coated coverslips, and cultured in an incubator at 37°C with 5% CO2 for an additional 20-28 h before the experiments.

5. Discussion

TIRF microscopes can only contain the information of biological sample without axial information because the TIRF raw data is an integration of fluorophore at different depths. By introducing the MA-TIRF microscopy and corresponding reconstruction algorithm, the axial information of the biological sample can be reconstructed. Furthermore, we find that MA-TIRF algorithm sometimes slightly enhance the lateral resolution due to the separation of fluorescent emitters along axial axis. For example, a hollow 3D spherical structure with a 360 nm lateral diameter, which simulated a biological sample labeled with an outer membrane, can only be observed as a solid 2D disc structure by TIRF microscopy (the 1st column of Fig. 2(a)). However, a blurred ring structure manifested in the reconstructed images by MA-TIRF (the 3rd column 3rd row of Fig. 2(a)). By incorporating the deconvolution procedure, our DMA-TIRF method could further improve their lateral resolution (the 4th column of Fig. 2(a)).

Noise seriously affect the fidelity of reconstruction (Fig. 5(a)). At larger incident angle, the relative signal to noise ratio decreases, which caused reconstructions with many artifacts (Fig. 5(a)). Under such circumstance, we recommended to adjust the parameter c in the Wiener inverse filter in Eq. (20) and incorporate with the TV penalty to remove some of the noise (Fig. 4(e) and Fig. 5(d)).

DMA-TIRF require no hardware changes to the existing MA-TIRF setups [1,13]. Because the imaging speed of the TIRF microscopy can be further improved [14], 100 Hz volumetric imaging shall be possible in the future. With the reduction of exposure time, the signal-to-noise-ratio of the raw images may degrade further. The revised penalty term could be used to suppress the artifacts generated due to excessive noise. The Hessian penalty, a second-order partial derivative penalty, uses a piecewise-approximation of boundaries between regions of different intensities and thus enables globally smooth transitions. In the future, the NLTV [15,16] or deep learning method [17] could also be used to suppress the reconstruction artifacts.

6. Conclusion

We proposed a one-step reconstruction algorithm DMA-TIRF by taking into account the PSF of the microscopy. The reconstruction of DMA-TIRF provide a higher lateral resolution compared with the state-of-the-art MA-TIRF algorithm [1], which could rapidly reconstruct actin filaments with a 200 nm lateral resolution with minimal artifacts. By including the TV penalty into the objective function to suppress the artifacts generated due to noise, we achieved ~200 nm lateral resolution and ~50 nm axial resolution in reconstructing low signal-to-noise ratio images containing the CCPs.

Funding

National Science and Technology Major Project Program (2016YFA0500400); National Natural Science Foundation of China (31327901, 31521062, 31570839, 31428004, 61375018, 61672253, 91750203); the Major State Basic Research Program of China (2013CB531200); Beijing Natural Science Foundation (L172003); National Postdoctoral Program for Innovative Talents (BX201800008).

Disclosures

The authors declare that there are no conflicts of interest related to this article.

References

1. J. Boulanger, C. Gueudry, D. Münch, B. Cinquin, P. Paul-Gilloteaux, S. Bardin, C. Guérin, F. Senger, L. Blanchoin, and J. Salamero, “Fast high-resolution 3D total internal reflection fluorescence microscopy by incidence angle scanning and azimuthal averaging,” Proc. Natl. Acad. Sci. U.S.A. 111(48), 17164–17169 (2014). [CrossRef]   [PubMed]  

2. S. Pendharker, S. Shende, W. Newman, S. Ogg, N. Nazemifard, and Z. Jacob, “Axial super-resolution evanescent wave tomography,” Opt. Lett. 41(23), 5499–5502 (2016). [CrossRef]   [PubMed]  

3. C. Zheng, G. Zhao, W. Liu, Y. Chen, Z. Zhang, L. Jin, Y. Xu, C. Kuang, and X. Liu, “Three-dimensional super-resolved live cell imaging through polarized multi-angle TIRF,” Opt. Lett. 43(7), 1423–1426 (2018). [CrossRef]   [PubMed]  

4. D. Axelrod, T. P. Burghardt, and N. L. Thompson, “Total internal reflection fluorescence,” Annu. Rev. Biophys. Bioeng. 13(1), 247–268 (1984). [CrossRef]   [PubMed]  

5. J. Lim, K. Lee, K. H. Jin, S. Shin, S. Lee, Y. Park, and J. C. Ye, “Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography,” Opt. Express 23(13), 16933–16948 (2015). [CrossRef]   [PubMed]  

6. D. Geman and C. Yang, “Nonlinear image recovery with half-quadratic regularization,” IEEE Trans. Image Process. 4(7), 932–946 (1995). [CrossRef]   [PubMed]  

7. J. F. Abascal, J. Chamorro-Servent, J. Aguirre, S. Arridge, T. Correia, J. Ripoll, J. J. Vaquero, and M. Desco, “Fluorescence diffuse optical tomography using the split Bregman method,” Med. Phys. 38(11), 6275–6284 (2011). [CrossRef]   [PubMed]  

8. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, “An iterative regularization method for total variation-based image restoration,” Siam Journal on Multiscale Modeling & Simulation 4(2), 460–489 (2005). [CrossRef]  

9. T. Goldstein and S. Osher, “The split Bregman method for L1-regularized problems,” SIAM J. Imaging Sci. 2(2), 323–343 (2009). [CrossRef]  

10. M. G. Gustafsson, L. Shao, P. M. Carlton, C. J. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat, “Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J. 94(12), 4957–4970 (2008). [CrossRef]   [PubMed]  

11. P. Y. Liu, L. K. Chin, W. Ser, H. F. Chen, C. M. Hsieh, C. H. Lee, K. B. Sung, T. C. Ayi, P. H. Yap, B. Liedberg, K. Wang, T. Bourouina, and Y. Leprince-Wang, “Cell refractive index for cell biology and disease diagnosis: past, present and future,” Lab Chip 16(4), 634–644 (2016). [CrossRef]   [PubMed]  

12. T. Yuan, L. Liu, Y. Zhang, L. Wei, S. Zhao, X. Zheng, X. Huang, J. Boulanger, C. Gueudry, J. Lu, L. Xie, W. Du, W. Zong, L. Yang, J. Salamero, Y. Liu, and L. Chen, “Diacylglycerol Guides the Hopping of Clathrin-Coated Pits along Microtubules for Exo-Endocytosis Coupling,” Dev. Cell 35(1), 120–130 (2015). [CrossRef]   [PubMed]  

13. W. Zong, X. Huang, C. Zhang, T. Yuan, L. L. Zhu, M. Fan, and L. Chen, “Shadowless-illuminated variable-angle TIRF (siva-TIRF) microscopy for the observation of spatial-temporal dynamics in live cells,” Biomed. Opt. Express 5(5), 1530–1540 (2014). [CrossRef]   [PubMed]  

14. X. Huang, J. Fan, L. Li, H. Liu, R. Wu, Y. Wu, L. Wei, H. Mao, A. Lal, P. Xi, L. Tang, Y. Zhang, Y. Liu, S. Tan, and L. Chen, “Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy,” Nat. Biotechnol. 36(5), 451–459 (2018). [CrossRef]   [PubMed]  

15. G. Gilboa and S. Osher, “Nonlocal operators with applications to image processing,” Multiscale Model. Simul. 7(3), 1005–1028 (2009). [CrossRef]  

16. G. Chierchia, N. Pustelnik, B. Pesquet-Popescu, and J. C. Pesquet, “A nonlocal structure tensor-based approach for multicomponent image recovery problems,” IEEE Trans. Image Process. 23(12), 5531–5544 (2014). [CrossRef]   [PubMed]  

17. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising,” IEEE Trans. Image Process. 26(7), 3142–3155 (2017). [CrossRef]   [PubMed]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1 Schematic illustration of the MA-TIRF setup. AOTF: acoustic optical tunable filter, M: mirror, L1-L5: Lens 1-Lens 5, MEMS: Microelectromechanical systems, DM: dichroic mirrors, Obj: objective lens.
Fig. 2
Fig. 2 Reconstruction of simulated data with ground-truth known. (a) Simulations of the overlay of vesicles. The 1st column is the TIRF image with the minimum incident angle. The 2nd to 4th columns are ground-truth (the 2nd column), reconstructions by the MA-TIRF method (the 3rd column), and the DMA-TIRF method (the 4th column). Each row in the 2nd~4th columns presented along the axial distribution at 10 nm depth (the 1st row), 30 nm depth (the 2nd row), 50 nm depth (the 3rd row), 70 nm depth (the 4th row), 90 nm depth (the 5th row), and 110 nm depth (the 6th row). Scale bar: 0.5 μm. (b) The ground-truth of point structures and their reconstructions via MA-TIRF and DMA-TIRF methods. (c) The unitary lateral intensities from the ground-truth image and reconstructions from MA-TIRF and DMA-TIRF methods along the yellow line in (b). (d) The ground-truth, MA-TIRF and DMA-TIRF reconstructions of three paralleled bars. (e) The unitary intensities of two ROIs along the axial direction. The max value of ROI encircled within the yellow rectangle box in (d) is normalized to 1 a.u., while the max value of ROI marked with red point in (b) is normalized to 2 a.u..
Fig. 3
Fig. 3 Reconstruction of a simulated curved line structure. (a) The ground-truth of the curved line structure in 3D. (b) Reconstructions of MA-TIRF (left) and DMA-TIRF (right) from images corrupted with Gaussian noise of a standard deviation at 50 a.u.. (c) Lateral distributions of the ground-truth, MA-TIRF, two-step method and Wiener DMA-TIRF reconstructions along the yellow line in (a). (d) Reconstructions of MA-TIRF (i), two-step method (ii), DMA-TIRF (iii) and TV DMA-TIRF (iv) from images corrupted with Gaussian noise of a standard deviation at 500 a.u.. Scale bar: 0.5 μm. (e) SSIMs of five different ROIs reconstructed with MA-TIRF, two-step, DMA-TIRF and TV DMA-TIRF methods as compared with the ground-truth.
Fig. 4
Fig. 4 Reconstructions of filament actin structures labeled by lifeact-EGFP in the HUVEC cell. (a) One example of multiple-angles TIRF raw images, scale bar 2 μm. (b) The zoom-ins of MA-TIRF and DMA-TIRF reconstructions at the square box in (a), scale bar 0.2 μm. (c) Lateral distributions of MA-TIRF and DMA-TIRF reconstructions along the yellow line in (b). The distance of two peaks of double-Gaussian fitting is 204 nm. (d) The color-map of the reconstruction with the MA-TIRF method. (e) A color-map to highlight different axial distributions of filament actin structure obtained with the DMA-TIRF method without TV penalty.
Fig. 5
Fig. 5 Reconstructions of CCPs. (a)-(d) Reconstruction results with different methods. (a) The reconstruction by MA-TIRF method without TV penalty, scale bar: 2 μm. The large square box is the corresponding zoom-in of the small ROI, scale bar: 0.2 μm; (b) The reconstruction by MA-TIRF method with TV penalty; (c) The reconstruction by the DMA-TIRF method without TV penalty; (d) The reconstruction by the DMA-TIRF method with TV penalty. (e) Gaussian fittings to measure lateral resolutions yielded by the DMA-TIRF and the MA-TIRF methods with TV penalty. (f) Gaussian fittings to measure the axial resolution yielded by the DMA-TIRF and the MA-TIRF methods with TV penalty. (g) Lateral and axial resolutions computed from Gaussian fittings.

Tables (1)

Tables Icon

Table 1 The implementation procedure of our DMA-TIRF algorithm is summarized below.

Equations (33)

Equations on this page are rendered with MathJax. Learn more.

I ( z , φ , θ ) = I 0 ( 0 , φ , θ ) e z d
I 0 ( 0 , φ , θ ) = 1 2 cos 2 φ 4 cos 2 θ ( 2 sin 2 θ n r 2 ) n r 4 cos 2 θ + sin 2 θ n r 2 + 1 2 ( 1 + sin 2 φ ) 4 cos 2 θ 1 - n r 2
d = λ 4 π 1 n 1 2 sin 2 θ n 2 2 ,
E ( x , y , θ ) = 0 2 π 0 I ( z , φ , α ) ρ ( α θ Ω / cos θ ) f ( x , y , z ) d α d z d φ ,
g ( x , y , θ ) = p s f E ( x , y , θ ) ,
g ( x , y , θ ) = p s f ( x , y ) 0 2 π 0 I ( z , φ , α ) ρ ( α θ Ω / cos θ ) f ( x , y , z ) d α d z d φ .
g θ = p s f z = z 1 z n h z , θ f z ,
g = p s f ( H f ) ,
min f ζ 2 g p s f H f 2 2 + D ( f ) + ( f ) ,
( f ) = { 0 , f 0 + , f < 0 .
min f , w ζ 2 g p s f w 2 2 + μ 2 w H f 2 2 + D ( f ) + ( f ) ,
min f , w , v , t ζ 2 g p s f w 2 2 + μ 2 w H f 2 2 + D ( t ) + λ 2 f t 2 2 + ( v ) + β 2 f v 2 2 .
B Q p ( v , v ( k ) , t , t ( k ) ) = Q ( v , t ) Q ( v ( k ) , t ( k ) ) p v ( k ) , v v ( k ) p u ( k ) , u u ( k ) ,
Q ( v , t ) = D ( t ) + ( v ) ,
p t ( k + 1 ) = p t ( k ) λ ( t ( k + 1 ) f ( k + 1 ) ) p v ( k + 1 ) = p v ( k ) λ ( v ( k + 1 ) f ( k + 1 ) ) ,
min f , w , v , t Q ( v , t ) + ζ 2 g p s f w 2 2 + μ 2 w H f 2 2 + λ 2 f t 2 2 + β 2 f v 2 2 ,
min f , w , v , t B Q p + ζ 2 g p s f w 2 2 + μ 2 w H f 2 2 + λ 2 f t 2 2 + β 2 f v 2 2 .
min f , w , v , t ζ 2 g p s f w 2 2 + μ 2 w H f 2 2 + D ( t ) + ( v ) + λ 2 f t + b t ( k ) 2 2 + β 2 f v + b v ( k ) 2 2 b t ( k + 1 ) = b t ( k ) + f ( k + 1 ) t ( k + 1 ) b v ( k + 1 ) = b v ( k ) + f ( k + 1 ) v ( k + 1 ) .
min w ζ 2 g p s f w 2 2 + μ 2 w H f ( k ) 2 2 ,
min f μ 2 w H f 2 2 + λ 2 f t ( k ) + b t ( k ) 2 2 + β 2 f v ( k ) + b v ( k ) 2 2 ,
min v ( v ) + β 2 f ( k + 1 ) v + b v ( k ) 2 2 ,
min t D ( t ) + λ 2 f ( k + 1 ) t + b t ( k ) 2 2 ,
w ( k + 1 ) = i f f t ( ζ O T F f f t ( Y ) + μ f f t ( H f ( k ) ) ζ O T F O T F + μ + c ) ,
f ( k + 1 ) = [ μ H T H + ( γ + λ ) I ] 1 [ μ H T w ( k + 1 ) + λ ( t ( k ) b t ( k ) ) + γ ( v ( k ) b v ( k ) ) ] .
min v ( v ) + β 2 f ( k + 1 ) v + b v ( k ) 2 2 ,
v ( k + 1 ) = max ( f ( k + 1 ) + b v ( k ) , 0 ) .
min t t x 1 + t y 1 + λ 2 f ( k + 1 ) t + b t ( k ) 2 2 .
min t l x 1 + l y 1 + λ 2 f ( k + 1 ) t + b t ( k ) 2 2 + γ 2 l x t x 2 2 + γ 2 l y t y 2 2 ,
min t l x 1 + l y 1 + λ 2 f ( k + 1 ) t + b t ( k ) 2 2 + γ 2 l x t x b x ( k , i ) 2 2 + γ 2 l y t y b y ( k , i ) 2 2 ,
b x ( k , i + 1 ) = b x ( k , i ) + ( t x ( k , i + 1 ) l x ( k , i + 1 ) ) b y ( k , i + 1 ) = b y ( k , i ) + ( t y ( k , i + 1 ) l y ( k , i + 1 ) ) .
t ( k , i + 1 ) = i f f t ( f f t ( ( x ) T ( l x ( k , i ) b x ( k , i ) ) + ( y ) T ( l y ( k , i ) b y ( k , i ) ) ) + λ γ ( f ( k + 1 ) + b t ( k ) ) | f f t ( x ) | 2 + | f f t ( y ) | 2 + λ γ ) ,
l x ( k , i + 1 ) = s h r i n k ( t x ( k , i + 1 ) + b x ( k , i ) , 1 γ ) : = { ( t x ( k , i + 1 ) + b x ( k , i ) ) 1 γ , t x ( k , i + 1 ) + b x ( k , i ) ( 1 γ , ) 0 , t x ( k , i + 1 ) + b x ( k , i ) ( 1 γ , 1 γ ) 1 γ + ( t x ( k , i + 1 ) + b x ( k , i ) ) , t x ( k , i + 1 ) + b x ( k , i ) ( , 1 γ ) ,
l y ( k , i + 1 ) = s h r i n k ( t y ( k , i + 1 ) + b y ( k , i ) , 1 γ ) ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.