1. Introduction

Multiphoton microscopy provides high spatiotemporal information of living cells and associated biological processes volumetrically in a native microenvironment deep within the living host [1,2]. Non-uniform and spatially varying aberrations, also known as field-dependent spatial aberrations, place a fundamental limit on the amount of spatiotemporal information, which is defined by the optical space bandwidth product (SBP) [3] received by the imaging system at a given time interval [4–6]. In laser scanning microscopes (LSM), the optical SBP is calculated by dividing the field of view (FOV) over the lateral point spread function (PSF) in a two-dimensional raster scan [3,6]. Hence, low SBP is attributed to non-uniform distribution of spatial aberrations across the entire FOV arising from either optical imperfections of optical elements across the entire system (system aberrations (SA)), imperfections towards the edges of the imaging FOV of a standard imaging objective lens (lens aberrations (LA)) [5,7,8] and/or sample-derived refractive index inhomogeneity (sample aberrations (SAA)) [9,10]. While customized objectives or scan lenses have extended the imaging FOV [3,6,11] to increase their SBP, they account only for optical distortion, which stem from field curvature of the lens, but not distortion caused by the sample. To counter sample-derived distortion, adaptive optics techniques are used [6]. Existing pupil-based adaptive optics (AO) in LSM use either time or spatial multiplexing to measure and correct spatially varying aberration masks across the imaging FOV. Spatial multiplexing (SM) refers to the spatial tiling of complex aberration wavefront masks across the imaging FOV using image splitters [12,13], while time multiplexing (TM) [14–16] refers to the sequential illumination of tiled wavefronts to cover the total imaging FOV. Existing TM approaches were demonstrated with a fraction (<20%) of the entire FOV at video rate [14] or in a large FOV with slow correction speeds [17], and cannot achieve video rates over the entire FOV. On the other hand, SM approaches achieve a large FOV at video rate but require specialized elements i.e. multi-angle prism [12] or non-pupil AO [13] arrangements, where the number of correctable field-dependent aberrations in a single acquisition is limited by either the size of a sub-element or number of actuators. A key advantage of TM over SM is that TM provides a higher number of sampling points using only a low-resolution deformable mirror (DM) [18]. In other words, TM can easily scale the number of aberration measurements and corrections across any FOV, albeit at the cost of increased acquisition time.

In this paper, we describe a high-speed aberration removal method for raster laser scanning systems that not only achieves a large FOV correction at video rate but can be directly implemented into any standard LSM microscope. We term this technique Raster Adaptive Optics (RAO). RAO is a TM approach that first arbitrarily segments a region of interest within a given FOV through digital synchronization with laser scanning mirrors that are then subjected to iterative image-based aberration measurements to detect isoplanatic aberration patches [13] (common aberrations) in real time, whilst scanning. To achieve video rate correction, the DM is synchronized to laser scanning mirrors so as to project the appropriate correction wavefront mask for each segment. The implementation of RAO was achieved in a triple laser scanning mirror system where one of the high-speed scanning mirrors, in this case, a polygon mirror, makes it possible to achieve video rate AO correction with minimal loss of acquisition time over the entire FOV. Since our RAO TM approach only requires a low-resolution deformable element to be placed at the pupil plane of the microscope objective lens, it can be directly integrated into any existing LSM system.

2. Methods

RAO demonstrates significant improvements over two previously reported approaches for rapid SM AO correction, namely conjugate AO [13] and multi-pupil AO [12]. Firstly, RAO can correct for spatially varying aberrations with different dominant aberration layers. Conjugate AO, on the other hand, can only operate well for samples with a single dominant layer of phase aberrations at a well-defined distance. Secondly, RAO can tailor flexible isoplanatic patches with samples that possess multiple axial aberrations for thick biological tissue using a single element, whereas multi-pupil AO is limited to fixed segments of isoplanatic patches, defined by the customized prism-pairs [12]. RAO shares similarity with spatial wavefront sampling methods that are used in TM scanning aperture phase retrieval [16] and Fourier Ptychography pupil recovery [15], where digital image segments are used to retrieve wavefront variations. Instead of post processing operations, RAO combines raster scanning and digital sampling to retrieve aberration wavefront masks from corresponding image segments. In doing so, it is able to isolate specific isoplanatic patches of aberrations from chosen image segments and converge towards optimal real time imaging. The simplicity of wavefront retrieval in RAO makes it possible to improve the imaging performance (e.g. PSF, SBP and fluorescence intensity) without optical splitting of the detection signal. Hence, compared to other approaches [12,14,18], RAO is not limited by the sensitivity of wavefront detectors or the number of image splitters.

Figure 1 describes the general concept and implementation of RAO along the x- and y- directions. Here, we first define a digital image constructed from raster scanning patterns (x,y), that rely on scan time (t) of the laser beam. For ease of reference, each raster scanned segment is labelled as TM along the x and y directions of the scan. Figure 1(a) shows five spatially separated TM segments along the direction of the scan that are first identified for RAO. Each of the segments, labelled as TMy, are generated by grouping a set of raster lines along the y-axis [2]. For example, Fig. 1(a) shows a series of scanning lines (e.g. y₁-y₁₀₂ for TMy₁ and y₄₀₉-y₅₁₂ for TMy₅) that are first digitally identified and grouped. Since each raster line (y₁-y₁₀₂) in TMy₁ has its unique synchronizing signals based on a pixel clock defined by the scan time in the y-axis (y(t)) [2], it is possible to isolate TMy₁ through digital synchronization. Once each TM segment is selected, it is then subjected to an iterative image-based wavefront sensing loop through a hill climbing algorithm [Fig. 1(b), i)] to identify the Zernike modes that make up the final wavefront masks. Once the final wavefront masks (e.g. W1, W3, W5) are determined, they are updated onto their corresponding TM segments [Fig. 1(b), ii)] by synchronizing with an internal software clock (not shown). An additional synchronization clock ensures that the DM update rate of the wavefront masks match the selected TMy segments. This sequential overlaying of the wavefront masks along TM segments allows complex wavefront masks to be displayed from a single deformable mirror [19]. Using this basic RAO concept along the x-axis, one can extend the imaging FOV by shifting multiple TMx segments along the x-axis of a given objective lens, as shown in Fig. 1(c) [3].

 

Fig. 1. Raster Adaptive Optics (RAO). The basic framework of Raster Adaptive Optics (RAO) lies in image segmentation, wavefront retrieval and sequential update of wavefront masks (W) for video rate imaging. a) Digital selection of raster lines along the y-axis for each TM- time multiplexing segment. b) i) describes the image-based AO metric (a gradient search algorithm) used to iterate towards optimizing the appropriate wavefront mask. b) ii) All the retrieved wavefront mask (W1-W5) are saved in memory before the scanning mirror resumes to scan the whole area along with multiple wavefront masks updated by the DM. c) illustrates that the segmentation can be extended to fill the back aperture of the objective to correct aberrations, especially those in the off-axis FOV, which is subject to higher amounts of LA.

Download Full Size | PDF

Next, we describe the time required to achieve RAO in 2 distinct steps for each TMy segment: optimization time for aberration retrieval [Eq. (1)] and imaging time after optimization [Eq. (2)], illustrated in Fig. 1(b), i and ii, respectively.

To achieve video rate imaging of 20 Hz with 512 × 512 pixels, the line rate is set to be around ∼0.1 ms. Based on this line rate and Fig. 1(a) i), each given TMy segment will take around 10.2 ms (0.1 ms * 102 lines). For a TMy optimization loop where n_Zernike = 12 Zernike modes and m = 5 steps on average, the number of measurements will be 61. Therefore, the optimization time for each TMy segment is around 622 ms for the optimization loop in Fig. 1(b) i). So five different TMy segments will take up to 3.11 sec to optimize.

The TMy segments can be selected either optically or digitally. An optical selection of a TMy segment would require a DM with an update time of 10.2 ms by limiting the scan range of the slow axis scanning galvo [2]. Digital selection, on the other hand, does not limit the scan range of the galvanometer, thus involving scanning of the entire frame. For this approach, a DM update rate of only 50 ms is needed so as to match the video imaging rate of 20 Hz. The digital selection removes the needs for an additional step to control the galvanometer mirror but increases the optimization time. After optimization is achieved, we can set n_w in Eq. (1) to just 1. Since the DM update rate already equals the time for each TMy segment, this fulfils the video rate imaging speed set at the start. We refer this as imaging time below

To test the concept of RAO, we construct a standard pupil AO setup that is conjugated to a triple conjugate scanning microscope (single polygon mirror scanning mirror and two galvanometer mirrors) as shown in Fig. 2(a). A 36-facets polygon mirror (PM, Lincoln Laser, MA, USA: DT-36-250-025, 4.8 mm × 5.0 mm) provides a fast line scan across the x-axis with a unidirectional line rate of 10.8 kHz (540 lines at 20 frames per second (fps)). A deformable mirror (DM, Thorlabs, NJ, USA: DMP40, 10 mm x 10 mm) is conjugated to an orthogonal pair of galvanometer mirrors (GM, Cambridge Technology, MA, USA: 6220H, 7 mm × 5 mm), which are all conjugated to the back focal plane of a long working distance objective lens (MO, Zeiss, Germany: W Plan 20× Apochromatic, 1.00 NA/water, Item No. 421452-9800-000). The pair of galvanometer mirrors provide the ability to transverse a fast scanning line across the full FOV of the objective lens. Only one of the galvanometric mirrors (GM-y(t)) is used for the slow axis. The other additional galvanometer mirror (GM-TMx) centers along the same fast axis direction as the polygon mirror and remains stationary. Movement of the second galvanometer mirror shifts the FOV to allow wider access along the fast axis (x-direction). The excitation laser beam uses an ultrafast femtosecond laser (Titanium-Sapphire pulsed laser, Spectra Physics: Tsunami, CA, USA) that is relayed by multiple sets of telescope lens arrangements (L1-L6) before entering the objective lens. A ∼50% filling factor of the excitation beam is projected onto the back aperture of the objective lens to ensure maximum power transmission and minimal light scattering, resulting in an effective excitation numerical aperture (NA) of 0.5 [20] for deep tissue imaging. With this NA, the theoretical lateral and axial resolutions can be approximated to 0.61 µm and 5.21 µm, respectively, by calculating the illumination point spread function [21]. A dichroic long-pass filter (M) (Semrock, NJ, USA: FF665-Di02) is used to reflect the emitted fluorescence signal onto photomultiplier tubes (PMT, Hamamatsu, Japan: R3896 or H7422-40) with corresponding spectral filters for the desired fluorescence. In video rate multiphoton microscopy, a single frame takes around tens of milliseconds to complete [22], which equals to ∼15–20 fps. Here, we fix a single frame acquisition to ∼50 ms, or 20 fps for 512 scanning lines (y-axis). Figure 2(b) shows how the FOV (256 µm X 320 µm) is digitally segmented into a series of TMy segments. Each TMy segment is digitally cropped and optimized in real-time so as to obtain a unique pupil Zernike wavefront by a simplified hill climbing algorithm. The hill climbing method used in RAO is achieved by moving through the first 12 orders of Zernike modes (not including tip, tilt and power), from low (Z4) to high (Z15) to identify the optimal set of Zernike mask for each correction; each step, a 0.05 step value of the Zernike amplitude. The optimization accounts for measured intensity (higher being better) but does not consider the gradient between each point (difference between each measurement). Figure 2(c) shows the retrieved wavefront mask (W1-W10) from the corresponding segments TMy₁-y₁₀.

 

Fig. 2. Implementing RAO using a triple scanning mirrors multiphoton microscope and removing SA. a) A deformable mirror (DM), a polygon scanning mirror (PM) and 2 galvo scanning mirrors (GM-y(t) for the slow axis and GM-TMx for FOV x-axis extension) are placed on the pupil plane, which is conjugated to the back focal plane of the objective. b) The acquired image frame of a fluorescent sample is segmented and c) the wavefront pattern projected by the DM is changed over time, corresponding to the SA at different time sequences of each segment (TM) during the scan. d) The degree of improvement in fluorescence intensity under low light (3–7% of full signal) after applying 10-segment RAO is assessed from images of 540 nm quantum dots titrated and dried on a microscope slide. e) Measurement of intensity improvement with increasing RAO segmentation of a rhodamine-stained kidney section off-axis (more than 100 µm from the center of the FOV) or on-axis (less than 100 µm from the center of the FOV), where the FOV size is 256 µm x 275 µm. Average aberration optimization time: b) 10 TMy – 220 sec, e) 1 TMy segment- 22 sec, 2 TMy segments- 44 sec, 5 TMy segments- 110 sec, 10 TMy segments- 220 sec. Total imaging time: b, e) 50 ms.

Download Full Size | PDF

In our experiments, individual TMy segments were digitally selected. Hence, each TM_y optimization is conducted by taking a full imaging frame at 50 ms. The experimentally measured total time taken to converge on an optimal wavefront mask for each TM segment is around 22 sec, which is 19 sec longer than the calculated time of 3 sec. This extra time taken during optimization is attributed to the software image acquisition and processing time (MATLAB and Matrox MIL). This means that for 10 TMy segments, the optimization time takes around 220 sec to retrieve all 10 wavefront masks. After optimization, the DM is updated at 5 ms [Eq. (2)] for each TMy segment that still achieves video rate aberration removal over the entire FOV. A software reference clock is used to synchronize the scanning mirrors to the DM, which has a rise time of up to 250 µs. The DM is then programmed to refresh at a rate of 20 Hz to 200 Hz, to project individual phase masks independently onto 1 to 10 different TMy segments.

RAO collates the intensity at video rate speed (20 fps ∼ dwell time of 5 µs) for wavefront optimization of a selected TM segment without slowing down the scanning speed or reducing the region of interest [10]. However, RAO uses image-based metrics [10], which makes it highly susceptible to inaccuracy under low light conditions (i.e. weak fluorescence signal). As such, it is necessary to calibrate the overall accuracy of the wavefront sensing and quantify the degree of enhancement possible. For this, we used a thin layer of stationary fluorescence particles (quantum dots) and performed an analysis of 10 TMy segments, as shown in Fig. 2(b), to retrieve a selection of wavefront masks for each TMy segment in Fig. 2(c). The uniform emission properties of a quantum dot (each quantum dot has a quantum yield of >50%) and their sharp emission spectrum allowed us to build upon reliable statistical information and thereby identify a positive correlation between intensity prior to correction and the corresponding gain in intensity after correction. Figure 2(d) shows the ratio of summed total fluorescence signal intensities in a single TMy segment over the maximum intensity (ideal) that is detectable before and after RAO. This fluorescence signal is deliberately captured at >90% lower intensity than the total maximum detectable intensity in the FOV in order to quantify limitations of the RAO’s digital retrieval at low light levels, especially at video rate raster scanning speeds. From Fig. 2(d), RAO is observed to require at least 3.5% of the maximum detectable intensity before any visible enhancement can be observed, with enhancement levelling off at a measured/ideal ratio of ∼7.5%. Instead of quantum dots as a reliable source of fluorescence signals with utility as intensity signals, one can also use other exogenous fluorescence markers [10]. To further demonstrate that RAO correction works across the FOV, we measured the improvement of randomly selected regions off-axis (more than 100 µm from the center of the FOV) and on-axis (within 100 µm from the center of the FOV) after applying 0-10 RAO segment corrections [Fig. 2(e)] on fixed biological tissue samples [10]. Improvements in signal intensity correlated with an increase in RAO segments implemented, with off-axis improvements seen after 5 RAO segment correction, indicating that increased sectioning enhanced the resolution of RAO correction across the entire FOV (256 µm X 275 µm).

3. Results

We perform a series of imaging tests to measure the efficiency of the RAO method on retrieving and correcting axial (z) and lateral (x, y) spatially varying aberrations so as to expand the imaging depth and FOV in a single frame [3,12,13]. To mimic SAA heterogeneity in refractive indices in tissue, we purposely introduce several layers of mismatched refractive indexes (glycerol, glass and cured transparent silicone – polydimethylsiloxane, PDMS) of up to ∼ 1.1 mm depth for the microscope’s water immersion objective lens. In addition, 3 µm diameter Fluoresbrite yellow-green microspheres (Polyscience, Inc) embedded in a silicone polymer block are used as fluorescence intensity signals to enable RAO measurement and quantification of the spatial varying aberrations [Fig. 3(a)]. Figures 3(b) and 3(c) show representative lateral and axial fluorescence profiles of the FOV and individual beads, respectively, taken at different axial depths (25 µm, 500 µm, 900 µm and 1000 µm) before and after RAO correction across a single imaging FOV. For ease of comparison and to maintain a consistent LA, we selected beads in the same region of the TMy2 section and tracked the difference between intensity and resolution before and after RAO [Fig. 3(b), ii-ix]. Figure 3(d) shows the cross-section line plots of the axial intensity distribution and the corresponding retrieved TMy₂ section wavefront masks after background subtraction from W3 and W4 masks in Fig. 2(c) to reveal SA. We further analyzed the signal and axial full width half maximum (FWHM) of 3 µm spheres (n = 3), for quantitative comparison [Fig. 3(e)]. Our results show that RAO affords up to 1.36-fold and 1.74-fold improvement in the FWHM and fluorescence intensity respectively, at an imaging depth of 900 µm. Thus, without applying any digital deconvolution, RAO is shown to achieve improvements in both measured intensity and imaging resolution in all three dimensions. Whilst there is significant improvement in imaging performance across the entire imaging FOV, we observe some artifacts in the x-z plot in Fig. 3(c). These image artifacts are likely due to slight mismatches during the time interval (rise time of DM) before each wavefront mask is displayed and scanning time (y). Still, the overall results clearly demonstrate the effectiveness of RAO in imaging samples of varying distortion dependent on their axial refractive indexes with video rate correction speed (20 fps), a rate that has not been achieved with previous TM methods.

 

Fig. 3. Removing SAA in refractive index inhomogeneity ∼1.1 mm thick sample. a) Diagram of the experimental setup: 3 µm yellow-green fluorescent beads in solidified PDMS overlaid by a glass coverslip were imaged at various depths (d) with a 20x objective with an effective excitation NA of 0.5 immersed in glycerol. Inhomogeneity of the refractive indexes (n) within the PDMS sample, and at the glass/glycerol interface scatter emitted photons, inducing aberration that increase with axial depth of the imaging plane. b) shows maximum intensity Z-projections of (i) a representative whole FOV and time-multiplexing sections (TM) at 1 mm z-depth and (ii-ix) TMy2 sections taken at various sample depths before and after 5-segment RAO. Images displayed are normalized to the maximum intensity of the RAO images for visualization. c) Axial maximum intensity projections of individual beads (colored boxes in (b)) at various sample depths along the axial direction before and after 5-segment RAO and d) the corresponding cross section plots and wavefront masks of the beads in (c) subtracted from the SA masks from Fig. 2c. e) Improvements in the axial FWHM and fluorescence intensity normalized to the RAO intensity maxima as shown in (d) are measured. FWHM is measured by applying Gaussian fitting (R² > 0.9) to the cross-section plots. Data are the means and standard deviations of n = 3 beads and scale bars shown are b) i) 50 µm, ii-ix) 25 µm and c) 5 µm. Average aberration optimization time: a-c) 110 sec. Total imaging time: a-c) 50 ms.

Download Full Size | PDF

Next, we move to demonstrate RAO in retrieving lateral spatially varying aberrations dominated by LA to increase the current FOV. For this, we constructed samples exhibiting heterogeneously distributed aberrations using two solutions of gelatin: the first encased between two coverslips positioned above a second solution containing embedded 1 µm Tetraspeck beads (Invitrogen), as shown in Fig. 4(a). Using the same 5 TMy segments as in Fig. 3, we carry out the RAO retrieval and correction process. In Fig. 4(b), the retrieved wavefront mask varies significantly from TM segments 1 to 5 as shown, with more complex aberrations identified in TMy₅. By plotting the post RAO results of the fluorescence amplitude and axial FWHM of beads selected in TMy₁ and TMy₅ segments [Fig. 4(c)], RAO is shown to achieve a ∼2-fold improvement in resolution and intensity gains at the imaging field with higher aberration as opposed to the field with lower aberration. Thus, RAO provides marginal improvement on the axial resolution of beads in TMy₁, which approaches the axial diffraction limit of 5.2 µm for the system (e.g. FWHM before RAO: 6.3 µm and FWHM after RAO: 5.9 µm), whereas in TMy₅, the axial resolution was improved from 10.76 µm to 4.63 µm.

 

Fig. 4. Removing field-dependent LA and extending FOV. a) Diagram of the experimental setup: 1 µm Tetraspeck beads embedded in gelatin and overlaid by a layer of gelatin sandwiched between two glass coverslips are imaged under an objective in PBS immersion medium. Photons emitted from beads at the periphery of the FOV exhibit increased scattering from spherical LA differences in sample refractive indexes. b) Maximum intensity Z-projection of the Tetraspeck beads before and after 5-segment RAO correction and the corresponding Zernike masks (W) applied in each TMy segment. Insets: 9.5X magnification showing changes in fluorescence intensity before and after RAO correction. c) Axial FWHM and normalized fluorescence intensity measurements of beads taken from the first and last (Fig. 4(b), TMy₁ and TMy₅, respectively) segments before and after RAO correction. d) and e) Imaging of fixed and permeabilized human fibroblast cells treated with SYTOX Green nucleic acid stain. d) The imaging FOV is shifted across the x-axis (∼100 µm per FOV or TMx) of the sample from the current FOV (TMx₀) by control of the galvo mirror until the limits of the extended FOV (800 µm) of the objective are reached. 5-segment RAO correction is performed at each FOV to obtain a map of (d, i) Zernike modes implemented on each segment and maximum intensity projection of the overlaid and aligned FOV (d, ii) before, (d, iii) after RAO correction and (d, iv) the fold change in fluorescence intensity. Insets show a 17X magnification of selected TM(x,y) segments in the (d, v and vi) left, (d, vii and viii) center and (d, ix and x) right of the full FOV, before and after RAO correction. e) 10X magnification of a cell in late anaphase (e, i) before and (e, ii) after RAO correction. Data are the means and standard deviations of n = 3 beads. The summed fluorescence intensity of images were increased by (b) 1.25X, (d) 2X or (e) 1.7X above the raw image for visualization and scale bars are (b) 25 µm, (b, inset) 10 µm, (d, ii and iii) 50 µm, (d, insets and e) 5 µm. Average aberration optimization time: b, c), e) 110 sec, d) 1100 sec. Total imaging time: b, c), e) 50 ms, d) 500 ms.

Download Full Size | PDF

To fully expand the FOV from the current FOV (TMx₀), we make use of the triplet mirror scanning conjugation [Fig. 2(a)] to create a series of TMx,y segments, as illustrated in Fig. 1(d). Instead of fluorescence beads, we used fixed and fluorescently-labeled fibroblasts so as to demonstrate the imaging quality of RAO. Fibroblasts play an important role in producing the structural framework (stroma) of biological tissue and actively divide and participate in wound healing. Hence, using the nucleic acid stain SYTOX Green (Invitrogen), we can visualize the variety of stages of mitosis in fibroblasts grown on a coverslip. Figure 4(d) i) shows that RAO can retrieve and display a total of 50 wavefront masks at 0.5 sec across an imaging FOV of up to 0.8 × 0.3 mm² for the fixed fibroblast sample. This equates to a 2.34-fold increase in the space-bandwidth product (SBP) of the imaging system. Figure 4(d) (ii, iii and iv) show the full tiled FOV extended to regions where the sample is not visible any longer due to defocusing, with all three scanning mirrors working in tandem. We selected regions of interest to provide qualitative comparison of the improvement in image quality across the full FOV in Fig. 4(d) (ii, iii and iv) which are taken at x-y positions of (TM(x_-4, y_3-4); TM(x₀, y₃); TM(x₄, y₂) [Fig. 4(d) v, vii and ix], respectively. We observe a marked increase in fluorescence recovery in cells residing within the extended FOV, with an up to 1.6-fold increase in fluorescence in a fibroblast 335 µm away from the center of the FOV [Fig. (4), ix and x]. Figure 4(d), iv demonstrates comparable improvements in fluorescence in both the current and extended FOV, despite the differences and heterogeneity in retrieved wavefront aberration masks. This suggests that other aberrations than field curvature are also being corrected for, although we note the degree of improvement could equally be skewed by the number of cells and fluorescence staining efficiency within the particular TMx,y segment. Using RAO, we greatly enhanced the signal of some of the cells undergoing mitosis as revealed by chromosomal condensation and separation to opposite poles exemplified in Fig. 4(e). Therefore, our results demonstrate the potential of overcoming spatially varying SAA and LA in depth and over a large FOV in laser scanning multiphoton imaging systems using only a single DM.

4. Conclusion

In conclusion, through digital segmentation and independent wavefront measurements, we demonstrate that RAO supports video rate imaging (20 fps) across the current imaging FOV whilst removing optical and sample aberrations caused by objective lenses or thick samples with heterogenous refractive indexes. RAO uses an image intensity-based aberration retrieval approach, also referred as sensorless AO, to obtain the optimized Zernike mask for each segment, prior to video rate imaging. Because RAO draws upon TM segmentation, it supports imaging at video rate, is highly flexible and can, unlike existing pupil AO methods, correct arbitrary isoplanatic patches with digital sectioning at a fixed phase retrieval time per segment, with symmetry of the TMx,y segment having only minor impact on the overall improvement of the fluorescence intensity [Appendix Fig. 5(i)-(iii)]. Further, RAO achieves the entire accessible imaging FOV of the scanning system which is calculated to be approximately 0.83 mm by generating over 50 distinct wavefront masks, which is 5.5-fold more wavefront masks than previous pupil AO (9 wavefront masks in [12,18]) at only a 0.5 sec update rate. The image enhancement and the modular nature of RAO will be a key enabler for existing multiphoton microscopy and other LSM to overcome heterogenous spatial aberrations in real time to visualize sub-cellular and cellular activities in vivo. The current RAO uses a DM with modest stroke (6.5 µm) and actuation speed (4 kHz) to converge toward optimal aberration removal which results in modest improvements (maximum 167%) at an imaging speed of 20 fps for the current FOV with 5–10 TMy segments. The critical factors limiting RAO is directly related to the optimization time and imaging time. The critical limit for the speed of optimization time [Eq. (1)] is dependent upon the number of steps (n_w) to retrieve the optimal aberration mask. As for the limit for imaging time, it is dependent on the temporal bandwidth of the DM. Since the current DM has a maximum update rate of 4 kHz, the transition time between each actuation or TMy segment is ∼250 µs. During this transition time, the wavefront mask can be updated inaccurately, and so result in imaging artifacts akin to image aliasing. Hence, a higher number of TMy segments would require a faster DM to keep up with the real time imaging speed. Hence, a tradeoff exists between aberration correction improvements and image artifacts incurred with increasing TMy segments. With these improvements considered, it would then be possible to extend RAO towards volumetric aberration correction [6].

Appendix A

 

Fig. 5. The effect of segment symmetry on RAO segment correction. A section of the cardiac tissue was imaged (i) before and after (ii) 5 TMy segments or (iii) 2 TMx with 5 TMy segments. The Zernike modes retrieved for each TMx,y segment is displayed next to the RAO images and the degree of improvement in fluorescence intensity is mentioned below each RAO image. Average aberration optimization time: ii) 110 sec and iii) 220 sec. Total imaging time: 50 ms.

Download Full Size | PDF

Funding

Australian Research Council (CE140100011, DE160100843, DP190100039).

Acknowledgments

We thank Mr Tienan Xu for assistance in sample preparation, initial advice from the adaptive optics team at Stromlo Observatory and Dr Kate Hannan for providing cell culture samples.

Disclosures

The technique and findings in this paper have been submitted for a provisional patent application, Application No. 2019904929.

References

1. R. Niesner, V. Andresen, J. Neumann, H. Spiecker, and M. Gunzer, “The power of single and multibeam two-photon microscopy for high-resolution and high-speed deep tissue and intravital imaging,” Biophys. J. 93(7), 2519–2529 (2007). [CrossRef]  

2. Y. X. Li, V. Gautam, A. Brüstle, I. A. Cockburn, V. R. Daria, C. Gillespie, K. Gaus, C. Alt, and W. M. Lee, “Flexible polygon-mirror based laser scanning microscope platform for multiphoton in-vivo imaging,” J. Biophotonics 10(11), 1526–1537 (2017). [CrossRef]  

3. J. R. Bumstead, J. J. Park, I. A. Rosen, A. W. Kraft, P. W. Wright, M. D. Reisman, D. C. Côté, and J. P. Culver, “Designing a large field-of-view two-photon microscope using optical invariant analysis,” Neurophotonics 5(02), 1–20 (2018). [CrossRef]  

4. G. Follain, L. Mercier, N. Osmani, S. Harlepp, and J. G. Goetz, “Seeing is believing – multi-scale spatio-temporal imaging towards in vivo cell biology,” J. Cell Sci. 130(1), 23–38 (2017). [CrossRef]  

5. A. V. Diezmann, M. Y. Lee, M. D. Lew, and W. E. Moerner, “Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy,” Optica 2(11), 985–993 (2015). [CrossRef]  

6. N. Ji, J. Freeman, and S. L. Smith, “Technologies for imaging neural activity in large volumes,” Nat. Neurosci. 19(9), 1154–1164 (2016). [CrossRef]  

7. G. Zheng, X. Ou, R. Horstmeyer, and C. Yang, “Characterization of spatially varying aberrations for wide field-of-view microscopy,” Opt. Express 21(13), 15131–15143 (2013). [CrossRef]  

8. T. Kamal, L. Yang, and W. M. Lee, “In situ retrieval and correction of aberrations in moldless lenses using Fourier ptychography,” Opt. Express 26(3), 2708–2719 (2018). [CrossRef]  

9. M. J. Booth, M. A. A. Neil, R. Juškaitis, and T. Wilson, “Adaptive aberration correction in a confocal microscope,” Proc. Natl. Acad. Sci. U. S. A. 99(9), 5788–5792 (2002). [CrossRef]  

10. J. Zeng, P. Mahou, M.-C. Schanne-Klein, E. Beaurepaire, and D. Débarre, “3D resolved mapping of optical aberrations in thick tissues,” Biomed. Opt. Express 3(8), 1898–1913 (2012). [CrossRef]  

11. N. J. Sofroniew, D. Flickinger, J. King, and K. Svoboda, “A large field of view two-photon mesoscope with subcellular resolution for in vivo imaging,” eLife 5, e14472 (2016). [CrossRef]  

12. J.-H. Park, L. Kong, Y. Zhou, and M. Cui, “Large-field-of-view imaging by multi-pupil adaptive optics,” Nat. Methods 14(6), 581–583 (2017). [CrossRef]  

13. J. Li, D. R. Beaulieu, H. Paudel, R. Barankov, T. G. Bifano, and J. Mertz, “Conjugate adaptive optics in widefield microscopy with an extended-source wavefront sensor,” Optica 2(8), 682–688 (2015). [CrossRef]  

14. K. Wang, D. E. Milkie, A. Saxena, P. Engerer, T. Misgeld, M. E. Bronner, J. Mumm, and E. Betzig, “Rapid adaptive optical recovery of optimal resolution over large volumes,” Nat. Methods 11(6), 625–628 (2014). [CrossRef]  

15. J. Chung, G. W. Martinez, K. C. Lencioni, S. R. Sadda, and C. Yang, “Computational aberration compensation by coded-aperture-based correction of aberration obtained from optical Fourier coding and blur estimation,” Optica 6(5), 647–661 (2019). [CrossRef]  

16. F. Soldevila, V. Durán, P. Clemente, J. Lancis, and E. Tajahuerce, “Phase imaging by spatial wavefront sampling,” Optica 5(2), 164–174 (2018). [CrossRef]  

17. N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7(2), 141–147 (2010). [CrossRef]  

18. S. J. Yang, W. E. Allen, I. Kauvar, A. S. Andalman, N. P. Young, C. K. Kim, J. H. Marshel, G. Wetzstein, and K. Deisseroth, “Extended field-of-view and increased-signal 3D holographic illumination with time-division multiplexing,” Opt. Express 23(25), 32573–32581 (2015). [CrossRef]  

19. K. Visscher, S. P. Gross, and S. M. Block, “Construction of multiple-beam optical traps with nanometer-resolution position sensing,” IEEE J. Sel. Top. Quantum Electron. 2(4), 1066–1076 (1996). [CrossRef]  

20. F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2(12), 932–940 (2005). [CrossRef]  

21. W. R. Zipfel, R. M. Williams, and W. W. Webb, “Nonlinear magic: multiphoton microscopy in the biosciences,” Nat. Biotechnol. 21(11), 1369–1377 (2003). [CrossRef]  

22. Mikael J. Pittet and R. Weissleder, “Intravital Imaging,” Cell 147(5), 983–991 (2011). [CrossRef]