Open Access
Add to BibSonomy
Abstract
Laser scanning microscopy requires beam steering through relay and focusing optics at sub-micron precision. In light-weight mobile systems, such as head mounted multiphoton microscopes, distortion and imaging plane curvature management is unpractical due to the complexity of required optic compensation. Thus, the resulting scan pattern limits anatomical fidelity and decreases analysis algorithm efficiency. Here, we present a technique that reconstructs the three-dimensional scan path only requiring translation of a simple fluorescent test probe. Our method is applicable to any type of scanning instrument with sectioning capabilities without prior assumptions regarding origin of imaging deviations. Further, we demonstrate that the obtained scan pattern allows analysis of these errors, and allows to restore anatomical accuracy relevant for complementary methods such as motion correction, further enhancing spatial registration and feature extraction.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Laser scanning microscopy, especially multi-photon microscopy [1], is the only currently available method allowing accurate imaging in vivo of functionally [2] and non-functionally [3] labelled structures deep in scattering tissue such as the brain. This allows detection of anatomical structures and measurement of dynamic properties, such as activity in fluorescently-labelled neurons with defined spatio-temporal patterns.
Ideally, such spatio-temporal patterns are acquired in a way that maps the instantaneously emitted photon flux originating from a spatially confined excited volume (a voxel) to an orthogonal, planar and equidistant grid, to form a time-series of distortion-free images for the purpose of display and computation. Most actual scanning systems, especially open-loop configurations [4], exhibit prominent beam path- and timing deviations from this ideal pattern. Additionally, the imaging optics may introduce spatial deviations of the object plane in lateral and axial directions. All these errors result in corresponding deformation of the recorded structure in addition to possible movement artifacts in the output image.
Recent advances in the development of miniaturized multi-photon microscopes [5,6] motivate a detailed analysis of the relationship between the intended and actual scan path. These systems combine miniature scanning devices and necessary optics in a lightweight headpiece that is connected to the rest of the setup via optical fibers and electrical conduits. These microscopes allow quantification of neuronal activity in animals free to interact with the environment in a self-determined way [7] by removing the experimental constraint of head-fixation.
Due to size and weight constraints, these microscopes contain comparatively simple imaging optics, and the control of their scanners, such as piezo-based fiber scanners [8] or microelectromechanical systems (MEMS) mirror scanners [6], is limited due to the lack of positional feed-back signals or tailored feed-forward control. At the same time, the scanners often are driven in or near the non-linear regime close to resonance in order to achieve high imaging frame rates, a prerequisite for accurate quantification of the neuronal dynamics (e.g. [9]).
To date there is no readily-usable method available for measuring the three-dimensional scan pattern produced by these miniaturized systems, and thus the differences between intended and actual sampling positions for these scanning systems are not known. However, this knowledge is crucial for unambiguous (Fig. 1(b) spatial reconstructions of the recordings in order to allow a subsequent correction of motion (e.g. [10]) or matching of the spatial organization to histological images or volumes (e.g. [11]).
Fig. 1. Comparison of ideal and actual scan pattern to assess deviations and stability. (a) Sub-sampled sketch of ideal bi-directional in-plane equidistant and orthogonal grid scan (single frame), dots mark center of signal sampling. (b) Two-dimensional (2d) projection of actual fast-scanning beam trajectory (128 lines) measured with a PSD averaged over ten full frames; scanner was a MEMS mirror driven near resonance.
In this publication, we present a technique to measure the three-dimensional path of the scanning focus with accuracies even below optical spatial resolution in stationary and mobile multi-photon microscopes, by imaging a constructed fluorescent volume with well-defined geometric properties. Our method applies, without modifications, to all laser scanning devices with optical sectioning capabilities, and only requires actuation accuracy of the scan location near the optical resolution of the instrument under test. We show that the measurements obtained can be used to correct for significant imaging errors, while also revealing local properties of the point spread function (PSF).
2. Measurement of the scan pattern
Laser scanning microscopes emit series of sequentially recorded samples forming images where each pixel encodes the detected signal amplitude along the scan path
with three-dimensional coordinates ${{\boldsymbol S}_{ik}}\; $ and timing ${t_{ik}}$ per scanned focal volume $ik$ (note that this representation assumes that the scan path is stable over time). Under ideal conditions, the measured signal relates directly back to a planar, orthogonal and equidistant grid, so that the image intensity per pixel isCommon scan-modes are uni- or bi-directional with near constant speed through a specimen (Fig. 1(a); note that uni- and bi-directional scanning refers to the fast scan axis where, depending on scan modalities, the signal is only detected in forward motion of the scanner or in both directions, respectively, as depicted in the figure). To evaluate the deviations of the actual scan path in a miniature microscope equipped with a MEMS mirror in open-loop configuration, we initially measured the beam spot position using a position sensitive diode (PSD, Hamamatsu S5991-01 with custom-built 4-channel transimpedance amplifier). As PSDs are not strictly linear, we recorded a calibration pattern with the beam stationary and moving the PSD perpendicular to the beam with a motorized micro-manipulator (Sutter MP-285). After the MEMS mirror, the beam was passed through the scan (Leica TCS SP2) and tube lens (Nikon MXA 22018) of a standard two-photon microscope system [1] and was focused onto the PSD using an achromat (f = 40 mm, Thorlabs AC254-040-B), creating a magnified scan pattern resolvable with the PSD. The PSD-signal after calibration provided spatial coordinates of the beam trajectory in the sensor plane (Fig. 1(b).
The average of ten measured actual scan paths projected onto the two-dimensional PSD is shown in Fig. 1(b). During the experiment, the scanner scanned 128 lines (bi-directional) per frame at a frame rate of 17.96 Hz. The fast axis was driven just below its first resonance. Compared to the idealized pattern in Fig. 1(a), the measured scan path shows prominent errors particularly Lissajous-type oscillations at the start of the frame, which we attributed primarily to the fast fly-back on the slow scan axis (Fig. 1(a), gray diagonal line).
To infer the 3d scan path pattern in situ we established a measurement principle based on the z-scan technique [12] as illustrated in Fig. 2 (see also supplemental document). The laser beam is focused into a homogeneous fluorescent medium with a plane interface to a non-fluorescent region, here the bottom of the vessel. By moving the interface axially (i.e. along z, the optical axis) the integral fluorescent signal recorded provides information of the focal volume which is characteristic for the PSF in this case axial, Fig. 2(b), see supplemental document Eq. (S1).
Fig. 2. Illustration of the implemented z-scan measurement principle. (a) Multi-photon excitation by focusing a pulsed laser into fluorescent material and detection of fluorescence; interface between fluorescent and non-fluorescent volume (here bottom of vessel) can be shifted through focal region using a translation stage (TS). DM: dichroic mirror, L: lens, MO: microscope objective, PMT: photomultiplier tube. (b) Shifting the interface through the focus volume gives a characteristic signal. Inset shows focal region contributing to multi-photon excitation for the shift position marked in the main graph; the transition point (cross) gives the center of the laser focus.
As laser scanning microscopes usually employ laser light sources with quasi-Gaussian transverse mode profile we assumed that the PSF can be well described by a three-dimensional Gaussian, as the axial intensity distribution is similar in shape to a Gaussian (see supplemental document). For such a Gaussian excitation volume, the integral fluorescence signal (Eq. (S1) can be described by the error function (Fig. 2(b) for any scan direction and interface orientation through the focal volume or PSF, respectively:
Fig. 3. Measurement probe and recording of interface transition through microscope’s field of view. (a) Conceptual design and (b) implemented probe. (c) Microscope recordings (20 frames average) with Alexa 488 solution at intermediate positions depicted in panels I–IV. (d) Measured fluorescence signal (dots, blue) for single pixel marked in (c) over shift distance s, numerical fit (line, green) yields transition point ${s_0}$ (corresponds to panel III in (c)).
We should note, that this method can be extended to a tomographic scan (cf. [13]) revealing not only the scan path but in addition a more rigorous estimation of the geometric shape of the focal volume (for details see supplemental document).
Taking optical properties into account, i.e. refraction and reflection specifically at the scan interface – both may interfere with the objective of recovering the scan path under general conditions, the simplest geometric object allowing recovery of the 3d scan path, requiring three interfaces, is an inverse pyramid which can also be used as a container for a suitable fluorescent solution.
The conceptual design and realization of such a probe only using three planar glass surfaces is shown in Fig. 3(a) and b, respectively. The three triangular glass pieces were mounted in dental cement (Paladur, Kulzer GmbH, Hanau, Germany) within a metal ring which provided a plane reference surface for a coverslip. For the scan pattern measurements the vessel is filled with a homogeneous fluorescent solution (here: Alexa 488, 20 µM, Thermo Fisher Scientific, Germany). The probe, mounted on a translation stage unless built in to the microscope itself, is imaged with the microscope and optics while running the desired scan program.
3. Correction of spatial distortion based on scan pattern measurement
We next measured the three-dimensional scan pattern of a standard multiphoton microscope (as described above) where the scan unit was supplemented by a MEMS mirror (A7M20.1-2000AU-LCC20-C2TP, Mirrorcle, CA, USA; see [6]) focused onto the standard galvanometric scanners (6215H, Cambridge technology, MA, USA) through a 4-f lens system (two scan lenses as above, see Fig. 4). The microscope was controlled by ScanImage (version 3.8).
Fig. 4. Schematic of experimental setup, inset shows photo of 2d MEMS mirror scanner (MMS). DM: dichroic mirror, GS: galvo scanners, L: lens, M: mirror, MO: microscope objective, PMT: photomultiplier tube, TS: translation stage.
To demonstrate the capabilities of the scan pattern correction, the software was configured to output an uneven aspect ratio. The generated triangular (fast axis, see bi-directional scanning above) and saw-tooth (slow axis) driving signals were applied to the scanner driver of the MEMS, which had a built-in low-pass filter (corner frequency below resonance). The closed-loop driven galvanometer scanners remained static at central position. For measuring the scan pattern the probe was mounted on a translation stage (Sutter Instrument MP-285) and a full set of transition measurements was recorded. For demonstration of the scan pattern correction we also recorded a sequence of images from a sample of green fluorescent spheres (Thermo Fischer F8836, Ø 9.9 µm) under the same imaging conditions (Fig. 5(b) and a standard scan using the galvanometer scanners (Fig. 5(d). Data was then analyzed as described above to obtain the 3d coordinates for every recorded sampling position (Fig. 5(a).
In addition to the anticipated uneven aspect ratio, the acquired data showed the oscillations along the slow axis after the fly-back of the MEMS mirror described above, as well as a slight skew. The optical system is expected to be well corrected, exemplified by the low deviation on the z-axis from the focal plane in the scan pattern measurement (standard deviation 0.44 µm over the full FOV), below the typical size of the axial PSF of a conventional multi-photon microscope (0.45 µm using an objective lens with a numerical aperture of 1.2 [14]).
Fig. 5. Scan pattern based correction of two-photon data (spheres Ø 9.9 µm). (a) Measured 3d scan pattern (every 10th sampling position) when scanning with MEMS mirror only; z-axis in side projections has same scale as fast x- and slow y-axis. (b) Direct microscope output scanning with MEMS mirror according to Fig. 1(a). (c) Scan pattern corrected data from (b) using natural nearest neighbor. (d) Overview scan with galvanometer scanners. (e) Cutout marked in overview scan (d) corresponding to the region shown in (c). Scale bars are 25 µm.
Distortions in the direct image output of the ScanImage software, based on an idealized scan pattern (Fig. 1(a), are clearly apparent as stretching effects, offsets between even and odd lines in the bidirectional scan and significant line-wise distortions at the top of the frame (Fig. 5(b). To correct these distortions we resampled the two-photon data onto the scan pattern plane (fitted to the data points) with a natural nearest neighbor resampling (Fig. 5(c). For comparison we recorded an image sequence with a larger FOV (Fig. 5(d) containing the area recorded with the MEMS mirrors (Fig. 5(e).
To quantify the accuracy of the scan pattern correction, we fitted a 2d Gaussian function to the fluorescent spheres in the raw and the corrected data. Quantification of the ratios between the resulting minor and major axes show an increase in ratio towards the expected ratio of 1 for a perfect sphere and the measured values match those from multiple reference recordings imaged with the closed-loop galvanometer scanners (Fig. 6).
Fig. 6. Quantifying the accuracy of the scan pattern corrected images by comparing the ratios of minor and major axis in 2d Gaussian fits of fluorescent spheres in the raw (Fig. 5(b)), corrected (Fig. 5(c)), and reference (Fig. 5(d)) data.
4. Application to mobile two-photon microscopy
The scan pattern measurement and correction was applied to a previously published miniature microscope developed for two-photon imaging in vivo in a freely moving rat [5]. The central piece is a so-called piezo-lever fiber scanner connected to a single-mode fiber (about 12 mm length inside scanner) transmitting the excitation pulses in a way, that amplifies lateral motion of the fiber end, imaged through tube lens (f = 12.7 mm, AC064-013-B, Thorlabs) and objective (f = 3 mm, custom design, Throl optics GmbH) into the tissue (Fig. 7(a). For measurement of the scan pattern, the microscope was mounted on a translation stage (MP-285, Sutter Instruments) and moved during the transition recordings while the sample stayed fixed on the table. We scanned bi-directionally at a frame rate of 10.02 Hz, 128 lines per frame with 128 sampling positions (pixels) per line. Setting up and execution of the procedure took about one hour.
Fig. 7. Scan pattern measurement of miniature microscope with fiber scanner. (a) Schematic of mobile two-photon microscope. DM: dichroic mirror, FL: fiber launch, FS: piezo-lever fiber scanner, L: lens, M: mirror, MO: microscope objective, PMT: photomultiplier tube, POF: plastic optical fiber, SMF: single-mode fiber. (b) Measured 3d scan pattern of fiberscope. (c) Unprocessed and (d) natural nearest neighbor corrected in vivo two-photon recordings (2d projection) of cortical neuronal population (GCaMP6s) in a rat.
The recorded interface transitions were analyzed (taking only few minutes to compute) as described above and the resulting three-dimensional distribution of the scan pattern positions is shown in Fig. 7(b). The coordinate system was chosen such that the x-y-plane is a two-dimensional linear fit to all points and the scan lines are aligned parallel to the x-axis. Again we observe oscillations, a slightly stretched aspect ratio and skew in the two scanning directions, which results from the non-linear behavior of and mechanical deviations inside the scanner. As the tip of the fiber describes an arc when deflected from its central location, the measured scan pattern is curved correspondingly, with a range of 12.1 µm in axial direction (the geometry of the scanner and optical magnification of the lens system predicts a range of about 8 µm).
The scan pattern correction was applied to in vivo recordings of cortical neuronal populations in a rat expressing the fluorescent activity indicator GCaMP6s [5]. Comparing the uncorrected and resampled (natural nearest neighbor) sequence averages (Fig. 7(c) and d, respectively) it is apparent that the distortions due to phase mismatches between the lines in the bi-directional scan are significantly reduced and the spatial dimensions, most notably at the sides, are corrected.
5. Summary
We presented a new method to measure three-dimensional coordinates of individual sampling positions in laser scanning microscopes by imaging a constructed volume with well-defined fluorescence properties. This technique can be applied directly to any existing laser scanning device with optical sectioning capabilities and does not rely on assumptions regarding the spatial relationship between individual samples nor the origin of occurring scanning errors.
Given the reliance of accurate spatial reconstructions on the knowledge of the individual sampling positions, such a characterization is particularly relevant for mobile miniature multi-photon microscopes as their scanner units, so far, are open-loop systems without a feedback or error signal. Using a MEMS mirror scanner and a miniature microscope with piezo-lever based fiber scanner, we demonstrated that the obtained scan pattern measurement can be used to analyze scan path distortions and correct corresponding recordings. Here, already a rather simple natural nearest neighbor resampling based on the measured scan pattern was shown to increase the spatial accuracy of recorded images. Such a correction is crucial in order to allow a subsequent matching of a recording to histological images obtained, for example, with electron microscopy.
The observed significant variability of the axial focus position along the scan path in the miniature microscope underlines the particular importance of our three-dimensional analysis. In such a case only the combination of fluorescence recordings with the three dimensional distribution of their sampling positions provides the full information required for applying adequate processing or analysis tools like, for example, motion correction or specific feature extraction.
Acknowledgments
We thank Kay-Michael Voit, Paul Stahr and Giacomo Bassetto for comments and suggestions for the manuscript.
Disclosures
The authors report no conflicts of interest.
Data availability
Data and software underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. W. Denk, J. Strickler, and W. Webb, “Two-photon laser scanning fluorescence microscopy,” Science 248(4951), 73–76 (1990). [CrossRef]
2. J. T. Russell, “Imaging calcium signals in vivo: a powerful tool in physiology and pharmacology,” Br J Pharmacol 163(8), 1605–1625 (2011). [CrossRef]
3. T. Euler, P. B. Detwiler, and W. Denk, “Directionally selective calcium signals in dendrites of starburst amacrine cells,” Nature 418(6900), 845–852 (2002). [CrossRef]
4. B. Borovic, A. Q. Liu, D. Popa, H. Cai, and F. L. Lewis, “Open-loop versus closed-loop control of MEMS devices: choices and issues,” J. Micromech. Microeng. 15(10), 1917–1924 (2005). [CrossRef]
5. J. Sawinski, D. J. Wallace, D. S. Greenberg, S. Grossmann, W. Denk, and J. N. D. Kerr, “Visually evoked activity in cortical cells imaged in freely moving animals,” Proc. Natl. Acad. Sci. 106(46), 19557–19562 (2009). [CrossRef]
6. A. Klioutchnikov, “Three-photon head-mounted microscope for imaging deep cortical layers in freely moving rats,” Nat. Methods 17(5), 509–513 (2020). [CrossRef]
7. F. Helmchen, M. S. Fee, D. W. Tank, and W. Denk, “A miniature head-mounted two-photon microscope: high-resolution brain imaging in freely moving animals,” Neuron 31(6), 903–912 (2001). [CrossRef]
8. J. Sawinski and W. Denk, “Miniature random-access fiber scanner for in vivo multiphoton imaging,” J. Appl. Phys. 102(3), 034701 (2007). [CrossRef]
9. V. Rahmati, K. Kirmse, D. Marković, K. Holthoff, and S. J. Kiebel, “Inferring neuronal dynamics from calcium imaging data using biophysical models and Bayesian inference,” PLoS Comput Biol 12(2), e1004736 (2016). [CrossRef]
10. D. S. Greenberg and J. N. D. Kerr, “Automated correction of fast motion artifacts for two-photon imaging of awake animals,” J. Neurosci. Methods 176(1), 1–15 (2009). [CrossRef]
11. K. A. Fulton and K. L. Briggman, “Permeabilization-free en bloc immunohistochemistry for correlative microscopy,” eLife 10, e63392 (2021). [CrossRef]
12. M. Sheik-Bahae, A. A. Said, T.-H. Wei, D. J. Hagan, and E. W. Van Stryland, “Sensitive measurement of optical nonlinearities using a single beam,” IEEE J. Quantum Electron. 26(4), 760–769 (1990). [CrossRef]
13. R. M. Mersereau, “Digital reconstruction of multidimensional signals from their projections,” Proc. IEEE 62(10), 1319–1338 (1974). [CrossRef]
14. W. R. Zipfel, R. M. Williams, and W. W. Webb, “Nonlinear magic: multiphoton microscopy in the biosciences,” Nat Biotechnol 21(11), 1369–1377 (2003). [CrossRef]
