1. Introduction

Optical imaging techniques, such as multiphoton microscopy (MPM), confocal microscopy (CM), and optical coherence tomography (OCT), have been shown to be indispensable tools for structural and functional imaging of biological tissues noninvasively. Furthermore, the superior optical sectioning capability by MPM using localized nonlinear excitation [1], by CM using pinhole gating [2], and by OCT using coherence gating [3], also proves superior to conventional widefield microscopy in reducing out-of-focus background noise and achieving high contrast visualization. However, a critical limitation of these imaging techniques is the trade-off among spatial resolution, acquisition speed and field-of-view (FOV). Typically, CM and MPM are capable of subcellular and cellular imaging in biological samples, with unprecedented high contrast and detailed information. However, most of the CM and MPM are essentially based on point-scanning, which can be very time consuming in the acquisition process depending on the imaging volume and signal strength, thus hindered their adaptations in inspection of clinically relevant size of tissue in biological systems. On top of that, CM and MPM also have very limited penetration depth in scattering media such as biological tissues. In particular, CM allows subsurface imaging of tissues up to 200 μm deep [4], whereas MPM imaging up to a depth of 600 μm for vasculature and 700 μm for neurons of mouse brain has been demonstrated with near-infrared pulsed laser [5]. On the other hand, OCT is capable of considerable high resolution and cross-sectional tomographic imaging of microstructures in biological tissues. Typically, OCT can provide micron-level axial resolution for an imaging depth up to 2-3 mm in biological tissues depending on the optical scattering [6]. The development of spectral-domain OCT greatly improves its acquisition speed and detection sensitivity mainly because the Fourier domain detection method enables the retrieval of full depth information from the interference spectrum of the optical signal without the need of depth scanning [7]. Another significant feature of OCT is the use of broadband light source in order to achieve short coherent length, which determines its axial resolution. In other words, coherence gating preserves the micron-level axial resolution throughout the imaging depth and OCT usually uses low numerical aperture (NA) objectives to maximize the depth of field (DOF) and penetration depth. However, this greatly hampers its lateral resolution, which scales inversely to the NA of the objectives. Another variant of OCT, optical coherence microscopy (OCM) improves the lateral resolution of OCT by using high NA objectives to generate en face images with subcellular level resolution, comparable to that of CM and MPM [8, 9]. OCM has advantage in imaging depth than CM and MPM, as it uses coherence gating to enhance rejection of out-of-focus light while maintaining the high detection sensitivity [8]. Typically, OCM has the same sensitivity as that of OCT at large imaging depth, albeit with much smaller DOF due to the high NA focusing objectives.

Apart from distinctions in imaging speed, acquisition window, as well as principles in achieving high signal-to-background ratio imaging, the optical techniques aforementioned also provide unique intrinsic optical contrasts (i.e. contrast arising from endogenous properties) from the biological tissues. MPM is based on the excitation and detection of induced nonlinear signals such as two-photon excitation fluorescence (TPEF) and second harmonic generation (SHG). TPEF signals can be observed from endogenous fluorescent biochemical species such as reduced nicotinamide adenine dinucleotide (NAD(P)H) and flavin adenine dinucleotide (FAD), two major fluorophores in the cytoplasm [10]. On the other hand, SHG signals come from non-centrosymmetric structures such as type I collagen [10], an abundant connective tissue found in biological tissues. In CM and OCT, the imaging contrast is mainly due to scattering and reflection from the inhomogeneity of refractive indices in the cellular microstructures and boundaries of biological tissues, though CM can also operate in fluorescence mode.

The idea of combining multiple optical techniques into a single platform is very attractive as it brings the strength of different techniques together, be it large FOV or high spatial resolutions, for the rapid inspection of large volume of biological samples at subcellular spatial resolution. On top of that, each optical technique provides unique and complementary optical contrasts from the biological tissues. Hence, a multi-contrast imaging system delivers more useful information about the morphological and physiological state of the samples than its individual embodiments. The provision of such multi-scale multimodal optical imaging system could greatly improve the detection, diagnosis and decision making for personalized clinical applications and treatments [11].

Previous efforts in constructing multimodal microscopes were mainly focused on combining MPM and OCM in order to obtain multiple contrasts of TPEF, SHG and scattering from the samples, in which the MPM and OCM shared the same observation volume with FOV of up to a few hundred microns in lateral and axial directions, though the implementations could vary due to the optical configurations and laser sources being used [12–16]. Co-registration in the acquired multi-contrast images was achieved mainly by sharing the same laser source and scanning optics, while techniques using spectral broadening and pinhole detection to improve the OCM axial resolution so that it could be comparable with the MPM axial resolution were also reported. Jeong et al [17] reported a combined MPM and OCT system which utilized a Ti:sapphire laser and a wavelength swept source as excitation sources respectively. Such a multi-scale system provided great degree of freedom in adjusting the beam diameters and power input for the two modalities, though careful alignment and calibration for co-registration of MPM and OCT images were required. Recently, our group demonstrated a multi-scale bi-modal MPM/OCT system which enabled large FOV imaging using OCT and high resolution imaging using MPM with co-registration being achieved by sharing a sub-10 femtosecond (fs) pulsed laser source and scanning optics [18]. The adjustable FOV and multiple contrasts by the system were also validated by imaging various biological samples. In another recent publication, we highlighted the differences and advantages of bi-modal MPM/OCM and MPM/OCT in biomedical applications [19]. Nevertheless, in all the above references, only bi-modal imaging with either MPM/OCM or MPM/OCT has been reported.

In this paper, we report the first tri-modal imaging with MPM/OCM/OCT, where the three imaging modalities are combined in one single platform. With the tri-modal system, a rapid OCT imaging can be first acquired from a large FOV to provide an overview of the tissue layers and also to find a region-of-interest (ROI) where more detailed examination is desired. A high resolution zoom-in imaging can then be acquired on the ROI by the co-registered MPM/OCM which provides both nonlinear and scattering contrasts. With the tri-modal imaging, not only multiple contrasts but also scalable FOVs can be acquired to provide complementary information about the tissues, which is very important in label-free imaging to obtain a sufficient set of parameters for reliable sample analysis.

2. Methods

In this paper, we present a multi-scale and multimodal MPM/OCM/OCT system for both tissue and cellular level imaging. The OCM and OCT can image layered tissue structures based on scattering contrast while the MPM can image cellular and extracellular matrix structures by TPEF and SHG contrasts. The system setup is shown in Fig. 1.

 

Fig. 1 Schematic diagram of the MPM/OCM/OCT system. Ti: S, Ti:sapphire pulsed laser; BS, beam splitter; M, mirror; L, lens, PP, prism pair; NDF, neutral density filter; DG, diffraction grating; DM, dichroic mirror; F, filter; CCD, line-scan camera; PMT, photomultiplier tube. Dashed line represents the optical path for incident beam into the prism-based dispersion precompensation unit. Diagram not drawn in exact scale.

Download Full Size | PDF

A sub-10 fs Ti:sapphire laser (Fusion PRO 400, Femtolasers) is the laser source for all the MPM, OCM, and OCT imaging. The ultrashort pulses result in much higher peak power compared to typical femtosecond lasers with sub-picosecond pulses for a given pulse energy, and thus higher efficiency in generating nonlinear optical signals such as TPEF and SHG [5]. Another important feature of the sub-10 fs Ti:sapphire laser is the broad bandwidth output. The laser has a center wavelength of 800 nm with full-width-half-maximum (FWHM) bandwidth of ~120 nm, resulting in a theoretical coherence length of 2.3 μm in air (1.7 μm in tissue). This coherence length that determines the axial resolution of OCM and OCT, is comparable to the confocal parameter of the 40 × , 0.8 NA focusing objective used for MPM/OCM, which is about 2.26 μm based on Rayleigh approximation [20]. The close match of the confocal parameter and the coherence length of the laser source for MPM and OCM/OCT respectively, ensures co-registration of the MPM and OCM/OCT images in the axial direction.

The laser output from the oscillator first passes through a prism-based dispersion precompensation unit to revert the pulse broadening effect from the subsequent optics. Afterwards, the laser beam is split by a 50/50 beam splitter into two arms, the sample and the OCM/OCT reference arm. The two beam paths form the arms of a Michelson interferometer. In the sample arm, the laser beam is coupled into an Olympus BX51 microscope unit where it is raster scanned by two galvanometer mirrors (6215HSM40B, Cambridge Tech.) in an en face mode. The scanned laser beam is further expanded by a 1:2 telecentric lens pair before being directed by a short-pass dichroic mirror (FF670-SDi01, Semrock) to fill the back aperture of the objective. The objective focuses the laser beam onto the sample. In the reference arm, the light is reflected by a reference mirror. The reference arm also includes a variable density filter for power adjustment and a prism pair for phase balancing with the optics in the sample arm.

The TPEF and SHG signals are epi-collected by the same objective in a nondescanned mode. They are first separated from the back-reflected excitation light by a short-pass dichroic mirror (FF670-SDi01, Semrock). Then, the emission signals are further separated into a two-channel detection arrangement with a second dichroic mirror (450DCXRU, Chroma) and photomultiplier tubes (PMTs). Custom software based on C + + platform acquires the MPM images in 512 (x pixels) by 512 (y pixels) with a frame rate of 0.38 Hz based on 10 µs pixel dwell time.

For OCM/OCT imaging, the backscattered light from the sample is also collected by the focusing objective, reflected back to the beam splitter, where it combines with the return beam from the reference arm and is directed to a custom built spectrometer. In the spectrometer, the combined beam is dispersed by a 1200 lines/mm transmission grating (Wasatch Photonics), and is focused by a compound lens composed of two 75 mm focus achromatic doublets back-to-back with an effective focal length of 37.5 mm, onto a 1024 pixels line-scan CCD camera (AViiVA SM2 CL, E2V). The spectrometer has a spectral resolution of ~0.27 nm, resulting in a penetration depth of ~600 μm in air. The interference fringes detected by the spectrometer are acquired by a frame grabber (PCIe-1427, National Instruments). Processed by non-uniform Fast Fourier transform (NUFFT) [21], an OCT A-line profile in the axial (z) direction is obtained. A cross-sectional OCT image is obtained by beam scanning in the x direction and the image is displayed in real-time. Each cross-sectional image consists of 512 lateral (x) pixels with 512 axial (z) pixels forming a XZ cross-sectional OCT image. A three-dimensional (3D) volume data can be acquired by capturing multiple XZ frames across 512 lateral scans (y pixels). From the 3D volume data, OCM images showing the en face views (XY) at any depth locations can be reconstructed after post-processing by Matlab 7.9 (Natick, MA).

A water immersion 40 × objective (LUMPlanFL N, Olympus) of 0.8 NA is used for the high-resolution MPM/OCM imaging and a 4 × objective (Plan N, Olympus) of 0.1 NA is used for the large FOV OCT imaging. The two objectives are mounted on a slider, which can be switched without altering the sample. The same XY galvo-scanners control the transverse beam scanning for all MPM, OCM and OCT images. The axial scanning of MPM is achieved by a piezo-objective scanner (MIPOS 500, Piezosystem Jena), with a maximum total axial scanning range of 400 μm. The spectral domain OCM and OCT require no depth scanning. The lateral co-registration is achieved by sharing the same light source and scanning components.

3. Results

3.1. Co-registration of MPM and OCM images

For validation of the co-registration of the high resolution MPM and OCM imaging, yellow-green fluorescent microspheres are imaged, in which 6 micron diameter beads are immobilized in tissue phantom made from agarose gel. The 40 × objective used to image the sample is fixed static so that the focal volume of the MPM and OCM is identical.

The MPM and OCM images are shown in Fig. 2(a) and 2(b); with fluorescence and reflectance contrast mechanisms demonstrated. Figure 2(c) shows the overlay of the MPM and OCM images, which are color-coded as red and green colors respectively. Individual microspheres are co-registered well, though some microspheres appear as elongated shape in OCM image, probably due to imperfect wavefront of the backscattered light. The beads images confirmed the co-registration of our MPM and OCM images.

 

Fig. 2 (a) MPM and (b) OCM images of 6 micron fluorescent microspheres. The overlay of MPM and OCM images is shown in Fig. 2(c) with color-coding of red and green respectively. Good co-registration is achieved with the overlapping of green and red to result in yellow. Scale bar represents 50 μm.

Download Full Size | PDF

3.2. Ex-vivo tissue imaging

3.2.1 Leaf sample imaging

We first demonstrate our tri-modal MPM/OCM/OCT system for imaging of leaf samples (Taxus cuspidata). Fresh leaf with the bottom side facing up is imaged under the microscope. A large FOV cross-sectional OCT image is captured by using the low NA 4 × objective. MPM images representing a ROI are captured using the 40 × objective. MPM signals are further separated by dichroic mirrors and filters in order to distinguish the TPEF and SHG contrast from the samples. MPM imaging speed is about 2 seconds per frame. The OCM image is captured by using the same 40 × objective. A single volumetric data is captured with the focal plane of objective fixed at that of MPM imaging. As affected by the short focal depth of the objective, the useful OCM imaging volume is limited to a thickness of ~20 µm near the focal plane of the objective.

Figure 3 shows the OCT, MPM and OCM images from the leaf sample. Figure 3(a) shows the large FOV cross-sectional OCT imaging, where three tissue layers, namely the cuticle, lower epidermis, and mesophyll spongy can be identified. The full thickness of the leaf is not observed as the sample is highly scattering and absorbing. The OCT image size is ~800 μm in lateral and ~600 μm in axial direction. In the OCT image, a ROI in the cuticle layer is identified, as shown by the dashed line. A high resolution MPM/OCM imaging is carried out on this region to obtain zoom-in imaging of the structures. Figures 3(b)-3(c) show the en face high-resolution view of the leaf sample in the cuticle layer using MPM and OCM. The structures shown are likely the stomata (pores in a leaf) and the papillae of the cuticle. Figure 3(d) shows the overlay of the MPM and OCM images with color codes of red and green respectively. We can observe that complementary contrasts are shown in the images with MPM image showing the autofluorescence in the TPEF channel from the papillae while the OCM image showing the scattering contrast from the boundary of the papillae. No SHG contrast is observed in the leaf sample. Excellent co-registration of the structures with different imaging contrasts is demonstrated.

 

Fig. 3 OCT, MPM and OCM images from leaf sample. Figure 3(a) shows the OCT cross-sectional image of the leaf; with the cuticle (C), lower epidermis (LE) and mesophyll spongy (MS) clearly visible. Dashed line shows the region where the MPM/OCM is performed. Figures 3(b)-3(c) show the MPM and OCM images of the leaf sample respectively. Structures visible are likely stomata (S) and papillae (P) of the cuticle. Figure 3(d) is the overlay of the (b) MPM and (c) OCM images with color codes of red and green respectively. Scale bars in Fig. 3(a) represent 100 μm; Figs. 3(b)-3(d) have the same scale with the scale bar in Fig. 3(b) representing 50 μm.

Download Full Size | PDF

3.2.2. Fish cornea imaging

The tri-modal MPM/OCM/OCT system is also used to image fish cornea samples. Tilapia fish is obtained from local vendors and whole fish eye is extracted for imaging under the microscope. Figure 4 shows the OCT, MPM, and OCM images from fish cornea. A cross-sectional OCT image with a large FOV of the fish cornea captured using the 4 × objective is shown in Fig. 4(a). The image size is ~800 μm in lateral and ~600 μm in axial direction. We can observe two layers, namely epithelium and stroma, from the OCT image. The current OCT system allows fast image acquisition at ~100 frames/s over a larger observation volume than that of MPM/OCM. Thus, it can be applied as a rapid diagnostic method for identification of abnormalities in the tissues based on scattering contrast, and as guidance for further localized inspection using higher resolution and more specific MPM/OCM imaging. In the OCT image, a ROI near the boundary of the epithelium and stroma is identified, as shown by the dashed line. A high resolution MPM/OCM imaging is carried out on this region to obtain zoom-in imaging of the cells and collagen fibers.

 

Fig. 4 OCT, MPM, and OCM images from fish cornea sample. Figure 4(a) shows the large FOV cross-sectional OCT image where the epithelium (EP) and stroma (S) could be differentiated. Dashed line shows the region where the MPM/OCM is performed. Figures 4(b)-4(d) show the high-resolution TPEF, SHG and OCM images. Figure 4(e) shows the overlay of the TPEF, SHG and OCM images with color codes of red, blue, and green respectively. Scale bars in Figs. 4(a) and 4(b) represent 100 μm and 50 μm, respectively. Figures 4(b)-4(e) share the same FOV.

Download Full Size | PDF

Figure 4(b)-4(e) show the high resolution MPM (including TPEF and SHG) and OCM images. In Fig. 4(b), cells are clearly visible in the TPEF channel as there is strong autofluorescence from the cytoplasm of the cells. In Fig. 4(c), strong SHG signal appears due to strong second harmonic generation from the collagen fibers in stroma layer, and is absent in the TPEF channel. Whereas in OCM image (Fig. 4(d)), scattering contrast mainly from the extracellular matrix and cell boundaries in the cornea is revealed, which apparently covers the whole epithelial/stroma regions. The OCM image also clearly shows the boundary between the epithelium and the stroma. The overlay of the TPEF, SHG and OCM images with red, blue and green color-coding are shown in Fig. 4(e). It is interesting to observe that multiple contrasts can be obtained from the tissue sample based on different intrinsic contrast mechanisms. In the fish cornea samples, the cellular structures as well as surrounding extracellular matrices can be distinguished by our multimodal system without any exogenous labeling.

The OCM imaging with spectral domain detection can acquire signal over a large depth simultaneously, only limited by the illumination of the focused light which covers ~10 times of the focal depth. When one MPM image is acquired from the focal plane, a stack of ~20 OCM frames are acquired simultaneously from around the focal plane. Figure 5 shows a stack of OCM frames which have been acquired together with the MPM images shown in Figs. 4(b) and 4(c). The depth spacing between adjacent OCM frames is ~1 µm and the stack covers a total depth of ~20 µm. The OCM frame that co-registers with the MPM image in depth is frame number 256 (the same image shown in Fig. 4(d)), as determined by an axial calibration. The other OCM frames do not co-register with the MPM image because they are not acquired from the same depth. In the stack of the OCM images, the boundary between the epithelium and the stroma is observed to shift from the upper left corner toward the lower right corner when the image depth is increased. The signal intensity and lateral resolution are observed to decrease when the image depth is further away from the focal plane. However, the axial resolution of the OCM is not affected by the image depth because it is determined by the coherence length of the laser source. It shows another advantage of the tri-modal imaging where OCM can acquire a stack of images from multiple depths around the focal plane simultaneously when MPM acquires one frame on the focal plane.

 

Fig. 5 A stack of OCM images acquired together with the MPM images in Fig. 4 from the fish cornea sample. The frame numbers are labeled in the images. The OCM frame that co-registers with the MPM image in Fig. 4 is identified as frame number 256. The depth spacing between adjacent OCM frames is ~1 µm. The arrow shows the shifting direction of the boundary between the epithelium and stroma when the depth is increased. Scale bar is 50 µm.

Download Full Size | PDF

4. Discussions

In multimodal imaging, co-registration among the different modalities is an important issue in order to ensure that the images acquired are correlated in spatial locations and resolutions. In the tri-modal system, the high resolution MPM and OCM are co-registered precisely with matched resolutions. Between the large FOV OCT and the high resolution MPM/OCM, a coarse registration in the spatial locations is also achieved. The co-registration is achieved through careful engineering design and precise calibration.

The lateral co-registration between MPM and OCM is achieved by sharing the same laser source, the scanners, and the objective. The laser beam in the sample arm is the same for the MPM and OCM. Therefore, their lateral imaging locations and resolutions are, respectively, matched automatically. The axial co-registration is more challenging, which includes matching the axial resolutions and the depth locations. The axial resolution of MPM is determined by the focal depth while that of OCM is determined by the coherence length of the laser source. In order to match their axial resolutions, the bandwidth of the source has to be sufficiently broad which would provide a short coherence length. In the literature, when the laser bandwidth is not sufficient, other techniques such as using supercontinuum to broaden the laser bandwidth [12, 15] and adding a pinhole gating in front of the OCM detector [12,13,15] have been investigated, to improve the axial co-registration. In the tri-modal system, this is achieved by using an ultrafast Ti:sapphire laser of sub-10 fs pulsewidth and 120 nm bandwidth.

In the axial direction, in order to register the MPM plane with the OCM image frame at the same depth, a mirror is used as a sample to perform the axial calibration. The mirror is placed at the focal plane of the objective and OCM imaging is performed with the reference arm fixed. The relative depth location of the mirror plane (indicating the location of the focal plane) is identified in the OCM image stack. After calibration, the objective is fixed stationary while the sample can be moved by a stage. Since the focal plane stays stationary, its relative depth position in the OCM stack stays the same. This calibration is accurate when the imaging depth in tissue is close to the surface. When the imaging depth is increased, the calibration error will increase, due to the difference in the refractive index between the immersion water and the tissue, which affects the optical pathlength in the OCM imaging. A calibration error of ~5 µm depth shift is expected for an imaging depth of 100 µm. Since this depth shift is predictable, it could be compensated to maintain a good axial co-registration between MPM and OCM even at deep depths.

To perform large FOV OCT imaging on the same platform, a lower NA objective than that used for the MPM/OCM is needed. One method is to use a different beam path for the OCT imaging where a smaller beam diameter under fills the back aperture of the objective, which creates a smaller effective NA for the OCT imaging [17]. However, this approach has the limitation that the effective NA cannot be changed quickly because it requires changing the beam diameter by adjusting the beam expansion lenses. Therefore, it is difficult to acquire large FOV OCT and high resolution OCM with the same system using this method. In the tri-modal system, the large FOV OCT imaging is achieved by switching to a low NA objective mounted on a sliding nosepiece on the microscope. Switching objectives on a microscope is a common practice and the precision and repeatability is ensured by the mechanical design of the nosepiece. By switching between a low NA and a high NA objective, large FOV OCT imaging can be acquired with the low NA objective and high-resolution MPM/OCM imaging with the high NA objective. In the lateral direction, the smaller FOV of the MPM/OCM is located at the center of the FOV of the OCT. To co-register in the axial direction, the tissue surface can be used as a reference to correlate the depth locations in the OCT and the MPM/OCM. The optical pathlength from a ROI to the tissue surface can be determined from the OCT cross-sectional image. The optical pathlength can be converted to tissue depth based on the refractive index of the tissue. In the MPM/OCM imaging, the focal plane can be moved to specific tissue depth by moving the objective or the sample stage. Therefore, the depth location of the MPM/OCM image plane can be registered in the OCT cross-sectional image. Since the utilization of the OCT is to provide an overview of the tissue layers and guide where to perform the zoom-in imaging, a coarse registration between the OCT and MPM/OCM is sufficient. In the future, it is possible to improve the axial co-registration by using a mirror to calibrate the relative positions of the focal plane with both the low and high NA objectives.

5. Conclusion

Multi-scale and multimodal imaging method combining MPM, OCM and OCT is implemented, and its capability in providing multiple intrinsic contrasts from biological samples is demonstrated by imaging ex-vivo tissues. This system achieves tri-modal MPM/OCM/OCT imaging, in which OCT presents a significant larger FOV than that of MPM/OCM, while MPM/OCM extract both nonlinear and scattering contrasts with high resolution from the tissue of observation. These features are particular useful for various applications in biomedical researches as more tissue information can be obtained from a single imaging platform, and with much faster and larger observation region using OCT, followed by higher resolution and more specific analysis based on MPM/OCM.