1. Introduction

In confocal microscopy, an excitation focal point is raster scanned across the field of view to generate a 2- or 3-dimensional (2D or 3D) image [1]. A pair of confocal pinholes are used to preferentially reject light originating from outside of the focal plane, which significantly improves image contrast compared to wide-field microscopy. In general, confocal microscopy takes more time to acquire data, due to the need to scan in the spatial domain in order to form an image. Since only a single pixel photon detector and associated readout electronics are needed, this approach allows for the use of a very sensitive photo detection system with low noise at relatively low cost [2]. Single detection modules is particularly attractive in specific modalities such as time-resolved fluorescence [3] or spectroscopy [4, 5]. For instance, the cost and complexity of time-resolved imaging can be greatly reduced by using time-correlated single photon counting (TCSPC) methods [4, 5]. Point-scanning is also critical in other imaging modalities, such as multiphoton microscopy and stimulated emission depletion (STED) microscopy [6, 7].

There are many instances where the fast acquisition of a sample image is critical to the application. Live cell imaging, for instance, requires fast image acquisition such that the cells and intracellular components do not change significantly during acquisition [8]. Furthermore, in high content screening (HCS) a technique used in drug discovery, it requires a large number of images to be collected over a wide range of experimental conditions in order to establish a suitable set for statistical analysis [9].

One example is fluorescence lifetime imaging microscopy (FLIM), which measures the fluorescence decay dynamics in spectrally-resolved intensity images. It has found many applications in life sciences research [10] and drug discovery [11]. One of the major applications is to detect protein-protein interactions by using FLIM to measure Förster resonance energy transfer (FRET). In FLIM-FRET, the fluorescence lifetime is decreased when the excitation energy of a “donor” fluorophore is non-radiatively transferred to a nearby “acceptor” fluorophore [10–12]. Energy transfer is only efficient at very short distances, typically less than 10 nm. Therefore, by attaching donor and acceptor fluorophores onto a pair of targets (e.g. two proteins), it is possible to detect when these targets interact with each other [10, 12]. Compared to conventional steady state fluorescence imaging measurements, FLIM-FRET offers the advantages of not being affected by spectral crosstalk between the donor and acceptor, changes in excitation intensity, detector sensitivity and is relatively insensitive to differences in fluorophore population levels [10].

A new method of using both FLIM and intensity images to quantify protein-protein interactions in live cells [13, 14] has increased the need to collect data at high speed so that the intensity images and FLIM data are co-registered. Techniques able to achieve high speed FLIM-FRET measurements with good spatial resolution would be a significant advance for cell biology and drug discovery efforts, as it would enable measuring the modulation of protein-protein interactions by small molecules to identify leads for drugs. Current FLIM systems do not offer a good balance between resolution (spatial and temporal) and acquisition speed. For example, the majority of current FLIM systems use TCSPC techniques, which have good resolution but are too slow (minutes) for HCS applications [15]. Time-gated intensified CCD based time [16, 17] and frequency [18] domain imaging techniques offer faster (in seconds) images acquisition but suffer from limitations in spatial resolution, difficulty in resolving complex decay dynamics, and require high power excitation [15]. It is worth noting that out-of-focus light in a FLIM setup not only introduces a loss in spatial resolution, but distorts the multi-exponential decay collected at each pixel by adding contributions from out-of-focus fluorophores [19, 20].

One way to increase the rate of image acquisition is to increase the raster scan speed. This approach is of limited value when imaging biological specimens because scan speeds are frequently limited by the excitation time required to generate enough emitted photons to obtain a good image. Particularly for TCSPC, the requirement for low count rates renders it generally too slow for live cell imaging applications. In these situations, a good alternative to increase image acquisition speed is to parallelize the scanning process.

Point-scanning setups can be multiplexed using a pair of spinning disks, where specially patterned pinholes produce an array of foci, covering the entire sample when spun [21]. This multiplexing approach allows for a significant improvement in image quality, due to the rejection of out-of-focus light, while using cameras to achieve high frame rates. Spinning disk systems typically experience low excitation light throughput, which can increase the integration time required to obtain a quality image [22]. Furthermore, out of focus light rejection is limited particularly in thicker samples, since out-of-focus light can leak into adjacent pinholes [22]. The camera itself is also limited by its own noise sources, which are generally more significant than those of discrete single detectors. The cameras used in a spinning disk setup cannot be replaced with stationary single-point detectors, since the emission foci generated by the spinning disks are always moving. Furthermore, the high speed gated cameras required to record time domain information using a spinning disk system suffer from poor spatial resolution that many subcellular details are lost. This typically limits such systems to collecting intensity images.

Risi et. al. recently reported a modified line-scanning confocal microscope that was given true confocal capabilities by spinning a disk of radial slits perpendicular to the primary confocal slit [23]. This maintained the microscope’s ability to provide a line output for a slit detector (e.g. spectrometer), though is still not suitable for point detection, as the disk does not descan the foci for a stationary point detector array.

To take full advantage of a multiplexed confocal approach, the detection may be made by discrete single-point detectors, which offer better signal-to-noise performance and temporal resolution than a CCD array [24]. For these methods, a foci array is generated by a lenslet array or spatial light modulator, and raster scanning is performed by the entire array [25, 26]. The de-scanning process, by which the emitted light from the sample returns through the same path, allows for fixed positioning of the discrete detectors. A number of different methods using a foci array to multiplex image acquisition have been reported where an array of up to 16 excitation beams is scanned across the sample (Nikon/Prairie Technologies) or the sample itself can be scanned relative to a static foci array by using a motorized microscope stage, though the latter approach is limited in speed and results in vibrations that are not ideal for live cell imaging [25, 27, 28].

We recently demonstrated a square multiple foci array scheme to achieve FLIM using a streak camera as a detector [28]. In this method, a two-dimensional (2D) foci array was generated by a lenslet array and relayed onto the sample plane. The fluorescence signal of each focal point is returned back through the dichroic, and collected by a matching 2D optical fiber array, which rearranges the fiber channels from a 10x10 arrangement into a one-dimensional (1D) 1x100 arrangement suitable for the slit of the streak camera. In that system, the sample was scanned using a motorized high-precision stage. For live cell imaging and HCS applications, an efficient and fast beam scanning technique is required. In this communication, we present the design and development of a novel array scanning technique that allows stationary foci array readout. This technique is implemented with a streak camera based multiplexing FLIM system to demonstrate its performance. It should be noted that the applications of such array scanning techniques are not limited to FLIM, but are useful to other confocal applications where stationary foci array readout is desired.

2. Mirror galvanometer scanning

The majority of current single beam confocal applications use mirror galvanometers to achieve raster scanning of the focal spot [4, 5, 24, 29]. A pair of orthogonal mirror galvanometers are used to tilt the trajectory of a light beam along two axes. The beam is then focused, at which point the tilt in the beam in each axis corresponds to a lateral spatial shift in the location at which the focus is generated. There is a nearly linear relation between tilt angle (ϕ) and lateral shift (∆d), which is also affected by the focal length of the focusing lens (f), as shown in Eq. (1).

 

Fig. 1 (a) Beam propagation of a foci array member d_in from the normal axis of the lens. The solid line path shows how the beam propagates when the mirror is at its initial position, 45° to each lens. The dashed line shows the beam path when the mirror has been tilted by φ. The lateral displacement is highlighted as ∆d. (b) Plot of the deflection experienced by different foci (f₁, f₂, f₃) in a mirror galvanometer scanner. The curve shows an extreme case of non-linear response of deflection with tilt angle. Since the input foci are collimated at different angles relative to the mirror, they will experience different segments of the deflection response. Each red spot represents a different focal point when the galvo is in a neutral 45° position. The red line shows the range of motion of each output focal point as the galvanometer is tilted from −5° to + 5°. The deflection range for each focal point over this same angle range is highlighted in blue, showing a larger range for the outer focal points compared to the central focal point.

Download Full Size | PDF

While mirror galvanometers are capable of scanning an array of foci, divergence from the linear approximation begins to cause problems for high resolution scanning. Some of the collimated beams will be incident on the mirrors at large angles to the normal, causing non-linear scanning. This could also occur in a single beam mirror galvanometer scheme, but then the galvanometer angle steps could be adjusted to account for the non-linearity. When there are multiple beams, some of them will be in the linear scanning region, while some will be non-linear, causing different scan ranges for different foci array members. This foci-dependent non-linearity is demonstrated in Fig. 1(b). To mitigate the non-linearity, lenses with a longer focal length can be used. This will have the effect of compressing the angle range to the linear region, while the required angular resolution is reduced in order to maintain the same lateral resolution. This can quickly approach the angular resolution limits of the galvanometer, typically in the range of 15 μrad (e.g., GVS412 2D large beam diameter galvo system, Thorlabs, Newton, NJ).

One possible solution for this would be to replace the lenses of the mirror galvanometer scanner system with f-theta lenses, also known as scanning lenses. As the name suggests, f-theta lenses are corrected to have a linear relationship with the angle of a collimated beam: a beam propagating at angle θ will be focused to a point on the focal plane, at a distance of f·θ from the center. However, this requires the use of up to four f-theta lenses, one on either side of each mirror galvanometer increasing the cost of the scanning solution, which rises quickly with scanning field size and scanning aperture [31]. The addition of the scan lenses also adds several elements to the optical train, reducing the transmission efficiency and increasing the alignment complexity of the system. Both of which are of particular concern for high speed applications such as live cell imaging and HCS. While resolving the non-linearity issue, f-theta lenses do not manage to solve the tight angular resolution requirements inherent to mirror galvo systems. F-theta lenses have focal lengths in the range of 100 mm to 250 mm (e.g. FTH series f-theta lens, Thorlabs Inc., Newton, NJ). In order to achieve a pre-objective lateral step of 5 μm (or 250 nm after a 20x objective), a galvo scanner equipped with one of these f-theta lenses would need to take a 10-25 μrad step, which is at the limit of the galvanometer’s tilt repeatability.

A spatial light modulator-based approach for generating the foci array was recently demonstrated in the FLIM-FRET domain on which galvanometer mirrors were used to scan the foci [26]. Poland et. al. were able to perform FLIM with very high temporal (150 ps) and spatial (0.8 μm) resolution at high throughput speeds (10 sec/FLIM image), though it does highlight some of the limitations of the mirror galvanometer scanning scheme. First, each galvanometer is placed between a pair of relay lenses. Second, as was mentioned earlier, the mirror scan resolution required would very quickly approach the repeatability limits of the scanner, where the smallest step size of 15 μrad would correspond with a scanning resolution of 0.3 μm at the sample plane (83x83 scan).

The QuadScanner is another mirror galvanometer implementation designed specifically for stimulated emission depletion microscopy [32]. Contrary to previously discussed mirror galvanometer scanners, the QuadScanner is implemented in the image space, on either side of the conjugate sample plane. Two mirror galvanometers are required for each axis, in order to shift the beams in the image plane while maintaining the collimated beam position in the pupil plane. This removes the requirement of expensive f-theta relay lenses, allowing for a more compact system. It can also achieve beam multiplexing, as long as all beams fit on the scanners throughout the scan. Since the scanner is in the image space, the lateral displacement is not governed by Eq. (2), and so it is possible to achieve discrete lateral steps sizes on the scale of 0.2 μm before demagnification by the objective. The main drawback is the increase in the number of galvanometers, adding to the cost and complexity of the system.

3. Design of a window tilt scanning scheme

To relax the tilt resolution requirements, and reduce the number of optical components as well as the overall size of the multifocal mirror galvanometer, we developed an array scanning scheme using galvanometers mounted with flat windows to shift incident beams through refraction. Contrary to mirror galvanometers, window galvanometers do not require the foci array to be shifted into the Fourier domain. Through the use of basic refractive properties, a tilt (ϕ) in the window of refractive index n and thickness t will cause a lateral deflection (∆d) of the output beam according to Eq. (3).

 

Fig. 2 Beam propagation through a tilted glass. The solid line shows how the beam propagates when the window is normal to direction of propagation. The dashed line shows the beam path when the window has been tilted by φ.

Download Full Size | PDF

As the scan relies on the small angle approximation, it will be vulnerable to the same nonlinearities at larger scan angles that the mirror tilt scheme experiences. The difference is that, since all of the members of the foci array are experiencing the same deflection nonlinearity, it is possible to correct this by adapting the angle step size at large angles.

One important distinction is that, even in the original non-approximated relationship, the lateral displacement ∆d is not a function of the original location of the focal point. Therefore, this method will treat all foci equally, and the scan ranges will be inherently uniform. This makes the window galvanometer scheme especially well-suited for multiplexed beam scanning.

The refractive scan has a number of linear tuning parameters, most notably the window thickness. The window material and full scan angle can also be used to tune the lateral deflection of the output foci. In contrast, the mirror tilt scheme essentially requires that all of the possible angles that the collimated beams make to the normal of the mirror are within the small angle approximation range. This means that the scan angle will be small, and the scan step angle could approach the angle resolution of the scanner. The window tilt can be over much larger scan angles, allowing the system to relax the angle resolution requirements of the scanners.

There is also a reduction in the complexity of the optical system. The mirror tilt scheme requires lenses or concave mirrors to convert to and from the Fourier domain. Furthermore, typically two sets of lenses/mirrors are needed to ensure that each galvanometer mirror can be at a Fourier plane. The window tilt scheme does not require any additional lenses, as long as all of the beams can still fit on the two successive windows. As a result, the system can achieve higher throughput than a similar mirror-galvanometer system. A multifocal mirror galvanometer system, with two coated mirrors (e.g. GVS011 silver-coated, Thorlabs Inc., Newton, NJ) and two scan lenses (e.g. FTH100-1064 visible AR coating, Thorlabs Inc., Newton, NJ) is calculated to have a round-trip efficiency through the scanner of 61%. Alternatively, a window galvanometer system comprised of two AR-coated fused silica windows (e.g. UVFS windows AR-A coating, Thorlabs Inc., Newton, NJ) is calculated to have a round-trip efficiency of 95%. This is particularly important for multiplexing scenarios, where the laser’s power is already divided between the foci.

4. Results and discussion

To study the effectiveness of the refractive window tilt scheme in scanning a foci array, we integrated two scanners into a square foci array set up shown in Fig. 3. A 10x10 region of a 20x20 lenslet array (18-1453-100-000, 500 μm pitch, square lenslets, SUSS MicroOptics, Neuchâtel, Switzerland) was used to generate the foci array. Based on the pitch of the lenslets, in order for the foci array to reach every point of the sample, each scanner must allow for ± 250 μm lateral deflection. If the galvanometers are fitted with fused silica (n = 1.47) windows of thickness 5 mm, this lateral deflection can be accomplished with a tilt angle of ± 8.9°. The galvanometer heads used for the experiment are capable of ± 10°, or ± 280 μm.

 

Fig. 3 Instrument setup for multiplexed FLIM, using window galvanometer scanners.

Download Full Size | PDF

The setup uses a 440 nm diode laser (3 mW, 60ps LDH-P-C-440M, PicoQuant, Berlin, Germany) as the excitation source, which is reflected off a 445 nm dichroic filter (Di01-R442-25 × 36, Semrock, Rochester, New York) into the first lenslet array. The lenslet array generates the foci array, which is subsequently scanned by the two window galvanometers (6230HM40B, Cambridge Technology, Cambridge, Massachusetts) along the two principal axes. The image of the shifted foci array is relayed into an inverted fluorescence microscope (Axiovert 200, Carl Zeiss, Oberkochen, Germany) onto the sample by a 20x objective. Fluorescent light generated at each excitation focal point is reversed through the system, undergoing an inverted shift back into alignment with the lenslet array. The lenslet array re-collimates the light before a second lenslet array (18-1531-100-000, 500 μm pitch, square lenslets, SUSS MicroOptics) refocuses the channels into a foci array on a custom 2D fiber array of matching pitch (0.11 NA, fused silica, n = 1.44, 50 μm diameter core, 125 μm diameter cladding. This fiber array acts effectively as a stationary array of single detectors, and so permits multiplexing that cannot be accomplished with a spinning disk confocal system. The fiber cores also act as an array of pinholes, through which the multiplexed image can become confocal. The fibers of the 2D fiber array are rearranged from 10x10 to a 1x100 line of fibers, suitable to couple to the input slit of the streak camera (SC-10, Optronis GmbH, Kehl, Germany), where the fluorescence decay of each channel can be observed in the readout camera (SRU-ED 16-bit mono, Optronis GmbH, Kehl, Germany). The 2D-1D fiber array has been reported previously [28].

The streak camera runs at 40 MHz repetition rate, synchronized to the pulsed diode laser driver, and a relative delay is employed in order to properly center the fluorescence decay streaks on the readout. By coupling the input with 50 μm core fibers, the time resolution of the streak camera can be as low as 20 ps. However, the system response function is limited by the relatively long pulse width of the diode laser, which is on the order of 100 ps. The readout camera can acquire frames up to a rate of 100 fps. This is the current bottleneck for how quickly a scan can be performed.

One concern with a refractive scanning solution for fluorescence imaging is how the scanning will vary with wavelength. If there is a significant enough change in refractive index, the fluorescent light will not be descanned properly. Fused silica was selected as the window material due to its low chromatic dispersion in the visible range. For the window parameters selected, excitation light at 440 nm (n = 1.46635) will allow ± 250 μm at 8.92°. As a worst case scenario, if 700 nm (n = 1.45529) light is generated and returns through the system, it will be scanned back ± 246 μm, meaning there will be a descanning deviation of up to 4 μm. Even in this extreme case, the deviation is small compared to the size of the single point detectors. In our case, the single point detector array is the 2D fiber array, where the core diameter of 50 μm will make this deviation insignificant. Similarly, this means that the scanning scheme can be applied to multiple excitation sources with minimal chromatic deviation between scan positions.

The window scanning galvanometer was first tested in the setup shown in Fig. 4(a), to assess uniformity and linearity. In this case, uniformity is described as how similar different foci move as they travel through the scanner, and linearity describes how linearly the beam is deflected versus tilt angle. The beam of a diode laser (Picoquant LDH-510) was expanded to illuminate a lenslet array (1 mm pitch) to generate the foci array. A window galvanometer placed in the image plane of the foci array, caused the lateral shift. The image wass then relayed onto a CCD (Qimaging QICAM 10-bit mono, Surrey, BC, Canada). Successive images were collected at different window tilt angles to measure the lateral displacement of the foci relative to their mean scan positions. The relative displacements were used to create an ensemble to test the uniformity of the displacement.

 

Fig. 4 (a) Experimental setup for testing the linearity and uniformity of the window tilt scanning scheme. (b) Readout images of single beam spot as the differential voltage across the galvanometer is changed. (c) Plot of the relative vertical position of 24 foci as they are scanned, with 95% confidence intervals included for each data point.

Download Full Size | PDF

Each spot was located as the center of mass of the surrounding pixels (Fig. 4(b)). This creates a collection of spot locations for each fiber at each tilt angle step. By subtracting each fiber spot set by the mean value of the scan, the sets are normalized and can be compared to each other (Fig. 4(c)). By taking the ensemble average of all foci, there is an inter-spot variability of ± 1.6 μm (at 95% confidence). While already very small, this variability is very likely due to limitations in centroiding given the pixel size of the readout camera, which is 4.65 x 4.65 μm².

The linearity of the scan was >99.99% against a line of best fit, with a maximum deviation from the best fit of 5 μm. Therefore, due to the small angle approximation region, the window galvanometer can be driven with uniform step spacing to obtain suitably linear deflection. If the tilt range exceeded the small angle approximation region, or if better precision were required, a non-uniform stepping scheme could be used. Since all the spots would still scan identically to each other, there would never be the uniformity concerns that arise with the mirror galvanometer scheme.

The window galvanometer raster scan was used to image a fixed Convallaria specimen. The image shown in Fig. 5 is comprised of 100 raster-scanned squares stitched together. There are a few regions of the image where the boundaries of the squares can be visualized, due to artifacts arising from manufacturing errors in the fiber array (e.g. three dead fibers led to three black squares – highlighted by the white square outlines, and alignment with respect to the lenslet array). These problems are not unique to the window galvanometer solution, and can be solved through more precise optomechanics or through reconstruction corrections.

 

Fig. 5 Intensity (left) and lifetime (right) reconstructions of Convallaria sampled after a 30x30 raster scan with the window galvanometers. Dark regions outlined in white indicate the scan regions of channels that were not collected due to dead fibers in the fiber array.

Download Full Size | PDF

The Convallaria image shown in Fig. 5 is comprised of 100 raster-scanned squares stitched together. There are a few regions of the image where the boundaries of the squares can be visualized, due to artifacts arising from manufacturing errors in the fiber array (e.g. three dead fibers led to three black squares – highlighted by the white square outlines, and alignment with respect to the lenslet array). These problems are not unique to the window galvanometer solution, and can be solved through more precise optomechanics or through reconstruction corrections.

The sample image was collected at a 30x30 scan. The angle step size used was 11.6 mrad, though the galvanometer head itself has a repeatability of 8 μrad. Therefore, our window galvanometer system does not come close to reaching the possible resolution limits. Due to the noise level in the electronic signal, the galvanometer head is limited to 2000 resolvable steps, meaning it is possible to perform a 20,000 x 20,000 scan. In our implementation, this would correspond to an angular step size of 174 μrad, providing a 0.25 μm lateral step size – before the objective. After a 20x objective, the step size at the sample is effectively 13 nm. Therefore, the resolution of the reconstructed image will not be limited by the window galvanometer even for super-resolution imaging techniques with higher resolution than optical diffraction.

For test purposes the galvanometer head was operated in step mode, though it can be run with linear motion as well. In step mode, it takes a single step in 700 μs. This means that a 30 x 30 scan is possible in 630 ms. If the fast axis were scanned continuously instead of by step, without considering the rest of the system, the galvo is capable of performing a line scan in 2.5 ms, and so a 30 x 30 scan could be performed in 75 ms. Thus using window galvanometers data collection for virtually any application will not be limited by the scan speed. Instead, in our FLIM set up acquisition is limited by the camera readout time of 10 ms per frame. Due to the frame rate, a 30 x 30 scan of the 10 x 10 foci (corresponding to final reconstructed image area of 300 x 300 pixels and 700 pixels in time) requires 900 readout frames, taking at least 9 seconds, or effectively 0.1 ms per FLIM data point. By comparison, collection of a TCSPC image of 256 x 256 area and 256 pixels in time requires approximately 45 seconds using avalanche photodiodes, or 0.7 ms per FLIM data point. Our benchmark of 1 FLIM image per second could be achieved by further multiplexing the streak camera, such that fewer readout frames are required for a scan of equal size.

5. Conclusion

In summary, we presented a novel but simple window galvanometer scheme for multiplexed scanning schemes with stationary detectors. This technique exhibited perfect uniformity and excellent linearity over the scan range, and is easily scalable to further multiplexing. The choice of window material and thickness provide an accurate way to tune the scanner’s sensitivity to tilt angle. The scheme does not require any additional scan lenses, saving in size, complexity, throughput and cost, making it possible to add multiplexing confocal capability to any existing wide-field microscope. The lateral translation is also significantly less sensitive to tilt angle, which means the tolerance on tilt angle resolution can be relaxed. Conversely, operated at the angular resolution limits of the scanner, the window tilt scheme is capable of high resolution scans far below the optical diffraction limit.

While our implementation of this scanning scheme has been limited to FLIM applications and temporal detection, a window galvanometer offers the same advantages for any other imaging modality that could be improved by single-point detector arrays. For instance, multiplexed hyperspectral imaging could be performed using a similar 2D-1D fiber array fitted to a spectrometer slit. It is also an attractive option for the emerging capabilities of single-photon avalanche diode arrays [33], which could be used for low-light and time-of-flight imaging.