Open Access
Add to BibSonomy
Abstract
Phase imaging microscopy with incoherent object illumination is convenient and affordable for biomedical research and clinics since it provides easy integration with a variety of bright-field optical microscopes. We report the design of a new hyperspectral imaging system based on a combination of a spatial light modulator (SLM) and an acousto-optic tunable filter (AOTF) for phase imaging microscopy. Contrast of phase-only objects originates from matched spectral and spatial filtering performed by the SLM and the AOTF located in Fourier-conjugate optical planes in the back-end of the optical system. The system is designed as an add-on to a standard optical microscope with incoherent diascopic sample illumination.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Hyperspectral imaging (HSI) is the main application of acousto-optic tunable filters (AOTFs) in biophotonics [1,2]. A number of specific applications of AOTFs in biomedical HSI has been reported to date [3–7]. AOTFs feature high spectral resolution (typically, 20 to 100 cm$^{-1}$ depending on AOTF design), random access wavelength tuning with $\sim 10$ $\mu$s transient time, and electronic control of transmitted bandwidth. Typical throughput of an AOTF is above 90% for linearly polarized light in visible and near-infrared spectral ranges.
Angular selectivity of Bragg diffraction in an AOTF enables phase imaging applications. First demonstrations of acousto-optic phase imaging were performed with coherent laser illumination [8,9]. Later on, it has been demonstrated that a special type of AOTF transfer function, namely bandpass spatial frequency filter [10–12], can be used for contrast enhancement of phase images with incoherent illumination [13,14]. For this purpose, nearly-wide-angle geometry of Bragg acousto-optic diffraction in birefringent crystals is used.
Several spectral phase imaging techniques with broadband or tunable illumination sources have been reported to date. Popescu et al. introduced the diffraction phase microscopy method and demonstrated its performance with spectrally filtered incoherent light [15,16]. Rinehart et al. demonstrated quantitative phase imaging with a wavelength-swept supercontimuum light source and a Mach–Zehnder interferometer [17]. Similar interferometric phase imaging techniques were later developed by Chen et al. using a tunable laser diode [18] and by Machikhin et al., who used an AOTF as an external monochromator for a low-coherent diode source [19].
In this work, we describe and characterize a phase imaging system based on an AOTF and a spatial light modulator (SLM) with programmable transmission for phase imaging. Image acquisition, calibration, and digital processing for fusion of phase and intensity images are described. Unlike previous works [15–19], we do not use interferometry for phase imaging and the setup has an architecture of a bright-field HSI system. The phase image is extracted by means of spatial filtering, and bright-field spectral images are used for intensity calibration.
2. Hyperspectral phase imaging system design
2.1 Wide-angle AOTF
A non-collinear wide-aperture paratellurite AOTF is a common device for HSI applications [20]. Image processing is associated with broad angular spectrum of the optical beam. Wide-angle diffraction type (noncritical phase matching geometry) is used in AOTFs for HSI with narrow transmission passband. The wavelength of diffracted light $\lambda$ is reciprocal to the ultrasound frequency $F$. This type of filters can be analyzed as a space-invariant optical system and characterized with the transfer function in $k$-space [21].
The efficiency of Bragg diffraction for plane waves at high coupling between incident and diffracted light is expressed as [22]
where $R$ is the phase mismatch parameter that depends on propagation direction of a plane wave component in the crystal. The wave vector diagram of noncollinear anisotropic Bragg phase matching in a uniaxial crystal is shown in Fig. 1. The optical wave vectors $\vec k_\textrm {o}$ and $\vec k_\textrm {e}$ correspond to orthogonal polarizations. They are coupled by means of the acoustic wave vector $\vec k_\textrm {ac}$, which makes an angle $\alpha$ with the crystal symmetry axis [110]. The axis $x''$ corresponds to the central angular spectrum component of the optical beam and makes an angle $(\alpha + \theta _0)$ with the crystal axis [001].
Fig. 1. Wave vector diagram of anisotropic Bragg diffraction: (a) definition of angular coordinates for off-axial diffraction in a uniaxial crystal; (b) conical phase matching near the wide-aperture diffraction point.
The normalized phase mismatch parameter $R$ is calculated as [12]:
The value of $\theta _0$ and the frequency $F$ are chosen to satisfy the conditions
2.2 Confocal HSI system
One of effective layouts of an AOTF-based HSI system is a confocal system with an AOTF placed near an intermediate image plane. This type of the optical system can provide the lowest geometrical and chromatic aberrations [23–25]. The lateral chromatic aberration is minimal in a strictly confocal AOTF position that enables controlling transmission bandwidth by means of spread-spectrum RF signals without deterioration of the image resolution [26].
The block diagram of the phase imaging setup is shown in Fig. 2. The optical system of the inverted microscope is unchanged. It consists of a halogen lamp, a bright-field condenser, a sample holder (object plane $\mathcal {I}_0$), an objective lens, a tube lens, and a port-selecting mirror or a beam splitter. Since phase objects broaden the angular spectrum of transmitted light, a high-NA objective is required. Light wave after a phase object can be represented as a composition of the transmitted (non-scattered) component and the high-frequency modulation component. The optical system is designed to block the non-scattered component of the phase-modulated light wave as described below.
Fig. 2. Schematic optical layout of a HSI phase microscopy system. The dashed line shows the interface between the microscope and the imaging system. The object and the image planes are shown as $\mathcal {I}$; Fourier-planes are shown as $\mathcal {F}$.
The confocal optical system relays the image twice: first from the microscope optical port ($\mathcal {I}_1$ plane) to the AOTF input, and second from the AOTF output to the CCD ($\mathcal {I}_3$ plane). The intermediate image plane $\mathcal {I}_2$ is in the acousto-optic interaction region inside the AOTF. A field lens at the interface between the microscope and the HSI system is used for correction of the ray paths since the optical train of the microscope is not telecentric. Two Fourier-domain spatial filters are used in this system. The first is the SLM in the front focal plane $\mathcal {F}_1$ of the first relay lens. This SLM operates as the amplitude mask blocking low spatial frequencies of the beam that correspond to the transmitted components of light wave. High-frequency components of the angular spectrum are transmitted by the SLM and diffracted by the AOTF. The second spatial filter is a static iris diaphragm in the back focal plane $\mathcal {F}_2$ of the second relay lens, i.e. the aperture stop, that is adjusted to transmit only the diffracted 1st-order beam from the AOTF output.
It has been demonstrated elsewhere [14] that for a flat field with pure phase modulation in $\mathcal {I}_1$ plane,
applying an amplitude mask $T_\textrm {am}(x,y)$ that blocks the non-scattered light component results in the transmitted field in $\mathcal {I}_2$ plane expressed as a convolution where $H_\textrm {am}(x,y)$ is the two-dimensional Fourier transform2.3 Point-spread function
Equation (7) describes the point spread function (PSF) of the phase imaging system. The mask $T_\textrm {am}(x,y)$ is a radially symmetric bandpass function
The simulations in Fig. 3 show the PSF absolute value $|H_\textrm {am}(x,y)|$ corresponding to different ratios $r_\textrm {min}/r_\textrm {max}$. The PSF with $r_\textrm {min}=0$, Fig. 3(a), is plotted for reference only since phase imaging is not possible without the dark-field mask. The grayscale map in each plot is normalized to the maximum of $|H_\textrm {am}(x,y)|$. The simulations demonstrate that the minimum possible $r_\textrm {min}$ satisfying (9) provides the best contrast since the sidelobes increase with $r_\textrm {min}/r_\textrm {max}$. On the other hand, the resolution, which is determined by the width of the main maximum of the PSF, remains almost constant.
Fig. 3. PSF of an imaging system with a ring-shaped pupil: (a) $r_\textrm {min}=0$, HSI operation mode; (b) $r_\textrm {min}/r_\textrm {max}=0.33$; (c) $r_\textrm {min}/r_\textrm {max}=0.5$; (d) $r_\textrm {min}/r_\textrm {max}=0.66$. White line — central profile. Inset — amplitude mask transmission.
2.4 System layout and components
The experimental phase imaging system is shown in Fig. 4. The optical beam emerges from the side port of the microscope. One steering mirror before the first relay lens and two mirrors after it (not shown in Fig. 2) were used for fine beam alignment.
The system was built on a custom-made wide-angle paratellurite AOTF designed and fabricated at NUST “MISIS”. The filter has the wavelength tuning range from 600 to 1100 nm, the resolution of 20 cm$^{-1}$ (0.9 nm FWHM passband at 650 nm), the clear aperture of 15 mm, and broadband angular dispersion compensation [27]. The input polarization is vertical (ordinary wave). The SLM was a commercially available liquid-crystal display L3C07U-85G13 (Epson Corp., Suwa, Japan) having HD resolution of 1920$\times$1080 pixels and 8.5 $\mu$m pixel pitch. The SLM was controlled with a HDMI interface board (BBS Bild- und Lichtsysteme GmbH., Bad Wiessee, Germany). The software was a custom script in MATLAB (The Mathworks Inc., Natick, MA). The images were captured with Alta U32 CCD (Apogee Imaging Systems Inc., Roseville, CA) 16 bit monochrome camera with 6.8 $\mu$m pixel pitch.
The HSI phase imaging system was commissioned with a standard inverted optical microscope Ti-E (Nikon Instruments, Tokyo, Japan). The microscope has an infinity-conjugated optical scheme with a high-NA apochromatic objective lens 40$\times$/0,95 Plan Apo $\lambda$. An additional DS-Fi2 color CCD (Nikon Instruments) installed in the microscope eyepiece port was used for frame tracking and capturing reference images. For illumination, a standard 100 W halogen lamp was used with a diascopic condenser, $\mathrm {NA}_\textrm {cond}=0.52$.
2.5 Image processing and false color fusion
At each wavelength, two images are captured in different modes. Without the band-pass mask applied to the SLM, the grayscale image is a monochromatic component of a HSI datacube. The digitized images $\widetilde {\mathbf {I}}$ are normalized using a standard procedure for background subtraction and flat field correction [28]
For visualization of both phase and amplitude of the images we use false color representation of the image as shown in Fig. 5. The grayscale phase image $\widetilde {\mathbf {I}}_\textrm {mask}[\lambda ,T_\textrm {am}]$ is converted to hue, and multiplied by the spectral image $\widetilde {\mathbf {I}}_\textrm {hsi}[\lambda ]$.
3. Results
The test samples were cytology smears of human red blood cells (RBCs) obtained by fine needle punction biopsy. The samples were not stained and contained different RBC density per unit area. The samples included both single RBCs and their aggregates.
The full field of view image is shown in Fig. 6 including the reference image captured at the eyepiece side port of the microscope (a), two image components $\mathbf {I}_\textrm {hsi}$ and $\mathbf {I}_\textrm {mask}$ at $\lambda =650$ nm, (b) and (c) respectively, and the false color fused image (d). The images have the resolution of $1320\times 1320$ pixels. The full field of view in the object plane is 180 $\mu$m. Digitally magnified images of single and aggregated RBCs are shown in Fig. 7. The experimental setup was configured with $r_\textrm {min}/r_\textrm {max}=0.33$. The phase image highlights thickest RBC points and improve the contrast with the background compared to RGB reference and spectral images.
Fig. 6. Full field-of-view cytology phase image: (a) reference image; (b) spectral image without amplitude mask; (c) phase image; (d) false color fused image.
Fig. 7. Magnified $10\times$ cytology phase image: (a) single RBC; (b) aggregated RBCs. The frame size equals $128\times 128$ pixels ($\sim 18$ $\mu$m).
4. Discussion
Optical wavelength is one of variable parameters of the setup. The fragment of a phase image at different wavelength in the range from 600 to 750 nm is shown in Fig. 8. Non-compensated lateral image drift along the AOTF diffraction plane, which is the vertical direction in the images, was approximately 1 pixel per 20 nm. Neither chromatic magnification nor defocusing were observed in this wavelength range. The image shift does not cause any resolution deterioration and, if necessary, it can be easily eliminated at image acquisition or processing stages via proper selecting the subframe of the CCD. Selection of the operating wavelength was governed by a trade-off of the AOTF and SLM efficiencies. The maximum brightness of the phase image was at $\lambda =650$ nm.
Fig. 8. Magnified $10\times$ phase images at different AOTF-transmitted wavelengths: (a) 600 nm; (b) 650 nm; (c) 700 nm; (d) 750 nm.
CCD exposure time in the experiments was set equal 0.5 s to obtain good signal to noise ratio. The maximum raw signal before background subtraction was approximately a half of the CCD electron well depth. After background subtraction, the average raw CCD signal was 21000 counts in bright-field HSI imaging mode and 4800 counts in phase imaging mode. The r.m.s. pixel noise was below 80 counts. While the CCD noise did no affect the experimental results at a relatively long exposure, we admit that negative effect of noise can be an important performance limiting factor in real-time image processing applications. There are several solutions that may improve the SNR at short exposures. The first is to use a CMOS or a emCCD camera instead of the CCD used in our experiments. The second option is to use AOTF transmission shaping to increase the diffraction bandwidth [26]. While both of these ways seem to be promising to improve the operation rate, they require in-depth analysis and experimental study and we leave them for future research.
5. Conclusions
The described instrument implements a new type of phase imaging involving matched spectral and spatial filtering of incoherent light. This is a bimodal imaging technique based on acousto-optic HSI. Precisely controlled spatial filtering is a distinguishing feature of this HSI system that enables phase imaging mode. Perfect spatial overlap of intensity and phase images is ensured by a common optical path for these two imaging modes without moving or replaceable elements. That enables imaging of partially absorbing objects when both phase and intensity images are relevant. Besides, bright field spectral imaging can be independently used with the same apparatus for imaging spectroscopy.
The system can be compared to various dark-field imaging techniques [29–33]. Tunable spectral filtering is the principal feature of this particular system. The narrow bandwidth of transmitted light minimizes the blur of the PSF. Programmable SLM as a band-pass angular filter is used to adjust scaling of $r_\textrm {min}$ and $r_\textrm {max}$ with $\lambda$ according to Eq. (7) and maintaining fixed $H_\textrm {am} (x,y)$ at different wavelengths. Accurate control of the PSF and the transmitted wavelength gives potential for quantitative phase retrieval.
Finally, all image processing is performed in the back-end optical system. That ensures full instrument compatibility with standard bright-field microscopes and white-light diascopic illumination.
Funding
Ministry of Science and Higher Education of the Russian Federation (075-02-2020-1588); National University of Science and Technology MISIS (K2-2020-007); Russian Foundation for Basic Research (18-07-00674).
Disclosures
The authors declare no conflicts of interest.
References
1. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19(1), 010901 (2014). [CrossRef]
2. N. S. Annamdevula, B. Sweat, P. Favreau, A. S. Lindsey, D. F. Alvarez, T. C. Rich, and S. J. Leavesley, “An approach for characterizing and comparing hyperspectral microscopy systems,” Sensors 13(7), 9267–9293 (2013). [CrossRef]
3. P. M. Kasili and T. Vo-Dinh, “Hyperspectral imaging system using acousto-optic tunable filter for flow cytometry applications,” Cytometry, Part A 69A(8), 835–841 (2006). [CrossRef]
4. B. Park, S. Lee, S.-C. Yoon, J. Sundaram, W. R. Windham, A. Hinton Jr., and K. C. Lawrence, “AOTF hyperspectral microscopic imaging for foodborne pathogenic bacteria detection,” in Sensing for Agriculture and Food Quality and Safety III, vol. 8027 of Proc. SPIEM. S. Kim, S.-I. Tu, and K. Chao, eds. (SPIE, 2011), pp. 802707 (1–11).
5. Z. Liu, H. Wang, and Q. Li, “Tongue tumor detection in medical hyperspectral images,” Sensors 12(1), 162–174 (2011). [CrossRef]
6. Y. Guan, Q. Li, Y. Wang, H. Liu, and Z. Zhu, “Pathological leucocyte segmentation algorithm based on hyperspectral imaging technique,” Opt. Eng. 51(5), 053202 (2012). [CrossRef]
7. S. J. Leavesley, N. Annamdevula, J. Boni, S. Stocker, K. Grant, B. Troyanovsky, T. C. Rich, and D. P. Alvarez, “Hyperspectral imaging microscopy for identification and quantitative analysis of fluorescently-labeled cells in highly autofluorescent tissue,” J. Biophotonics 5(1), 67–84 (2012). [CrossRef]
8. V. I. Balakshy and D. E. Kostyuk, “Acousto-optic image processing,” Appl. Opt. 48(7), C24–C32 (2009). [CrossRef]
9. K. B. Yushkov, V. Y. Molchanov, P. V. Belousov, and A. Yu. Abrosimov, “Contrast enhancement in microscopy of human thyroid tumors by means of acousto-optic adaptive spatial filtering,” J. Biomed. Opt. 21(1), 016003 (2016). [CrossRef]
10. V. B. Voloshinov, V. Y. Molchanov, and T. M. Babkina, “Two-dimensional selection of optical spatial frequencies by acousto-optic methods,” Opt. Eng. 41(6), 1273–1280 (2002). [CrossRef]
11. V. I. Balakshy, “Acousto-optic visualization of optical wavefronts,” Appl. Opt. 57(10), C56–C63 (2018). [CrossRef]
12. K. B. Yushkov, A. I. Chizhikov, O. Yu. Makarov, and V. Ya. Molchanov, “Optimization of noncollinear AOTF design for laser beam shaping,” Appl. Opt. 59(28), 8575–8581 (2020). [CrossRef]
13. K. B. Yushkov and V. Y. Molchanov, “Hyperspectral imaging acousto-optic system with spatial filtering for optical phase visualization,” J. Biomed. Opt. 22(6), 066017 (2017). [CrossRef]
14. K. B. Yushkov, J. Champagne, J.-C. Kastelik, and V. Y. Molchanov, “Hyperspectral phase imaging with a spatially matched acousto-optical tunable filter,” in Label-free Biomedical Imaging and Sensing (LBIS) 2019, vol. 10890 of Proc. SPIEN. T. Shaked and O. Hayden, eds. (SPIE, 2019), p. 108900V.
15. G. Popescu, T. Ikeda, R. R. Dasari, and M. S. Feld, “Diffraction phase microscopy for quantifying cell structure and dynamics,” Opt. Lett. 31(6), 775–777 (2006). [CrossRef]
16. H. Pham, B. Bhaduri, H. Ding, and G. Popescu, “Spectroscopic diffraction phase microscopy,” Opt. Lett. 37(16), 3438–3440 (2012). [CrossRef]
17. M. Rinehart, Y. Zhu, and A. Wax, “Quantitative phase spectroscopy,” Biomed. Opt. Express 3(5), 958–965 (2012). [CrossRef]
18. S. Chen, J. Ryu, K. Lee, and Y. Zhu, “Swept source digital holographic phase microscopy,” Opt. Lett. 41(4), 665–668 (2016). [CrossRef]
19. A. S. Machikhin, L. Burmak, O. V. Polschikova, A. G. Ramazanova, V. E. Pozhar, and S. V. Boritko, “Multispectral phase imaging based on acousto-optic filtration of interfering light beams,” Appl. Opt. 57(10), C64–C69 (2018). [CrossRef]
20. A. Goutzoulis and D. Pape, eds., Design and Fabrication of Acousto-Optic Devices (Marcel Dekker, 1994).
21. J. W. Goodman, Introduction to Fourier Optics (Roberts, 2005), 3rd ed.
22. A. Yariv and P. Yeh, Optical Waves in Crystals (Wiley, 1984).
23. D. R. Suhre, L. J. Denes, and N. Gupta, “Telecentric confocal optics for aberration correction of acousto-optic tunable filters,” Appl. Opt. 43(6), 1255–1260 (2004). [CrossRef]
24. V. B. Voloshinov, K. B. Yushkov, and B. Linde, “Improvement in performance of a TeO2 acousto-optic imaging spectrometer,” J. Opt. A: Pure Appl. Opt. 9(4), 341–347 (2007). [CrossRef]
25. A. Machikhin, V. Batshev, and V. E. Pozhar, “Aberration analysis of AOTF-based spectral imaging systems,” J. Opt. Soc. Am. A 34(7), 1109–1113 (2017). [CrossRef]
26. K. B. Yushkov, O. Yu. Makarov, and V. Ya. Molchanov, “Novel protocol of hyperspectral data acquisition by means of an acousto-optical tunable filter with synthesized transmission function,” Opt. Lett. 44(6), 1500–1503 (2019). [CrossRef]
27. S. P. Anikin, V. F. Esipov, V. Ya. Molchanov, A. M. Tatarnikov, and K. B. Yushkov, “An acousto-optical imaging spectrometer for astrophysical measurements,” Opt. Spectrosc. 121(1), 115–122 (2016). [CrossRef]
28. D. A. Glenar, J. J. Hillman, B. Saif, and J. Bergstralh, “Acousto-optic imaging spectropolarimetry for remote sensing,” Appl. Opt. 33(31), 7412–7424 (1994). [CrossRef]
29. T. Löffler, T. Bauer, K. J. Siebert, H. G. Roskos, A. Fitzgerald, and S. Czasch, “Terahertz dark-field imaging of biomedical tissue,” Opt. Express 9(12), 616–621 (2001). [CrossRef]
30. R. Yi, K. K. Chu, and J. Mertz, “Graded-field microscopy with white light,” Opt. Express 14(12), 5191–5200 (2006). [CrossRef]
31. N. Fairbairn, A. Christofidou, A. G. Kanaras, T. A. Newman, and O. L. Muskens, “Hyperspectral darkfield microscopy of single hollow gold nanoparticles for biomedical applications,” Phys. Chem. Chem. Phys. 15(12), 4163–4168 (2013). [CrossRef]
32. A. Ray, R. Kopelman, B. Chon, K. Briggman, and J. Hwang, “Scattering based hyperspectral imaging of plasmonic nanoplate clusters towards biomedical applications,” J. Biophotonics 9(7), 721–729 (2016). [CrossRef]
33. M. Naser, R. S. Schloss, P. Berjaud, and N. N. Boustany, “Label-free dynamic segmentation and morphological analysis of subcellular optical scatterers,” J. Biomed. Opt. 23(9), 1 (2018). [CrossRef]
