Open Access
Add to BibSonomy
Abstract
Optofluidic time-stretch quantitative phase imaging (OTS-QPI) is a powerful tool as it enables high-throughput (>10,000 cell/s) QPI of single live cells. OTS-QPI is based on decoding temporally stretched spectral interferograms that carry the spatial profiles of cells flowing on a microfluidic chip. However, the utility of OTS-QPI is troubled by difficulties in phase retrieval from the high-frequency region of the temporal interferograms, such as phase-unwrapping errors, high instrumentation cost, and large data volume. To overcome these difficulties, we propose and experimentally demonstrate frequency-shifted OTS-QPI by bringing the phase information to the baseband region. Furthermore, to show its boosted utility, we use it to demonstrate image-based classification of leukemia cells with high accuracy over 96% and evaluation of drug-treated leukemia cells via deep learning.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Optofluidic time-stretch quantitative phase imaging (OTS-QPI) has emerged as a powerful tool in diverse fields, in particular biomedical fields, as it enables high-throughput QPI of unlabeled objects such as cells and bacteria [1–6]. The principle of OTS-QPI is to decode temporally stretched spectral interferograms that carry the spatial profiles of cells flowing on a microfluidic chip into bright-field and quantitative phase (e.g., refractive index) images of the cells. It combines the ability of OTS imaging to perform high-throughput imaging of single live cells in large heterogeneous populations on a microfluidic platform without the need for a detector array [1,6–18] and the ability of QPI to spatially map the optical path length delay and thickness of each cell in a label-free manner and provide complementary information to well-established fluorescence microscopy [19–30]. By virtue of these unique traits, OTS-QPI has been exploited for a wide range of biomedical applications such as microalgal culture evaluation [2,3], blood screening [4], spleen tissue investigation [5], and cellular protein concentration characterization [6].
Unfortunately, the utility of conventional OTS-QPI to diverse biomedical applications is hindered by a few critical limitations. First, the recovery and analysis of cellular phase images are difficult to perform because they are contaminated by phase-unwrapping errors caused by high-frequency noise, resulting in the construction of noisy phase images [3]. Second, a highly costly wide-bandwidth photodetector is required for acquiring high-frequency time-stretched optical signals on which the phase maps of cells are encoded. Third, the acquisition of the high-frequency time-stretched optical signals leads to a requirement for a large data storage medium to store the big data. These limitations translate into the inability of conventional OTS-QPI imaging to obtain a comprehensive database of cellular phase images of various cell types required for the use of advanced computational analysis tools such as deep learning [11,12,31–33]. Therefore, there is an immediate need for a robust, cost-effective OTS-QPI system that provides a reliable library of noise-free cellular phase images that fuels the intelligent analysis.
In this paper, we proposed and experimentally demonstrated frequency-shifted OTS-QPI to overcome the above limitations and hence achieve highly reliable, high-throughput label-free QPI flow cytometry with intelligent image analysis. Specifically, we introduced a frequency shifter composed of a pair of acousto-optic modulators (AOMs) to the reference arm of the OTS-QPI setup to bring the phase-map-contained optical signals to the low-frequency region and hence to achieve low-bandwidth detection of the signals without compromising them against high-frequency noise and phase-unwrapping errors. To show the powerful utility of frequency-shifted OTS-QPI, we acquired more than 100,000 bright-field and phase images of white blood cells (WBCs) and cancer cells at a high throughput of 15,000 cells/s. In addition, we demonstrated intelligent image-based classification of various types of leukemia cells and WBCs with a convolutional neural network (CNN) trained with images of 32,000 cells, reaching a high average accuracy of 96.20%. We further validated the performance of the trained CNN by predicting the ratios of different cell types in spiked samples. We also used frequency-shifted OTS-QPI to evaluate the efficacy of an anticancer drug on leukemia cells, in which the drug-induced differentiation could be identified from the cell images with the CNN-based analysis. Therefore, the frequency-shifted OTS-QPI holds promise for providing reliable datasets of cellular phase images for various types of cells and applications in cancer biology, microbiology, immunology, and pathology.
2. Materials and methods
2.1 Experimental apparatus
Our frequency-shifted OTS-QPI setup is schematically shown in Fig. 1(a). The key of the frequency-shifted OTS-QPI setup that differentiates itself from OTS-QPI is an interferometric configuration with a typical signal arm and a frequency-shifted reference arm. The light source is a home-built mode-locked ytterbium-doped fiber laser with a center wavelength of 1030 nm, a bandwidth of 23.7 nm, a repetition rate fr of 33.97 MHz, and an average output power of 20 mW. The ultrashort optical pulse from the laser is stretched in the time domain by the temporal disperser, i.e., a 20-km-long single-mode fiber (Nufern, 1060XP), with a total group-velocity dispersion (GVD) of -380 ps/nm. Subsequently, the time-stretched pulse is amplified by an ytterbium-doped fiber amplifier (YDFA) with an optical gain of 20 dB and split by a 50:50 fiber coupler into the sample arm and the reference arm, respectively. In the signal arm, the pulse is incident on a diffraction grating (Thorlabs, GR25-1210, 1200 grooves/mm) to spatially map the pulse into a one-dimensional (1D) rainbow pattern, such that different frequency components of the pulse are focused by an objective lens (Olympus, LCPlan N, 50×, NA0.65) to probe different spatial portions of the flowing cell. As shown in Fig. 1(b), the phases and intensities of different frequency components of the pulse are encoded with the cellular refractive index and thickness as well as the absorption and scattering at each spatial portion, respectively. The information-encoded pulse is collected by a second objective lens and recombined by a second diffraction grating into a collimator. In the reference arm, the optical frequency of the pulse is first up-shifted by 3.75fr (i.e., 127.39 MHz) with the first AOM and then down-shifted by 3.5fr (i.e., 118.90 MHz) with the second AOM. The reference pulse is also coupled into the fiber via the collimator. The signal and reference beams are combined in another fiber coupler to form a beat note, which is detected by a photodetector with a bandwidth of 5 GHz and digitized by a high-speed oscilloscope (Tektronix, DPO71604B) at a sampling rate of 50 GS/s. Eventually, the obtained 1D interferometric signal is recovered into two-dimensional (2D) bright-field and phase images in the signal processor with algorithms discussed in the following section for further analysis (Fig. 1(c)).
Fig. 1. Frequency-shifted OTS-QPI. a. Schematic of the experimental setup of frequency-shifted OTS-QPI, including an OTS-QPI microscope and a microfluidic chip. b. Enlarged view of the microfluidic chip, where the cellular spatial profile is encoded into the intensity and phase of each pulse. c. Image recovery process. The temporal interferograms of four successive pulses are converted into one line of pixels in the bright-field and phase images. d. Architecture of the CNN. The CNN is constructed based on the autoencoder structure to extract key cellular features from the bottleneck layer during the reconstruction of the bright-field and phase images.
For the high-speed and continuous acquisition of cell images with the setup above, the microfluidic chip with both physical stability and optical transparency was designed and fabricated with polydimethylsiloxane (PDMS) and glass [34]. Specifically, the PDMS layer was fabricated with typical soft lithography and bonded with the glass slide after plasma cleaning. The channel was formed in between the PDMS layer and the glass slide, while three polyetheretherketone (PEEK) tubes were punched into the channel as the sheath inlet, sample inlet, and outlet, respectively. A coverslip was placed on the PDMS surface above the narrow channel to prevent possible leakage due to the high liquid pressure. During measurements, the microfluidic chip was placed in between the objective lenses with the sheath flow and the sample flow injected into the channel using syringe pumps. The infusion rates of the sheath flow and the sample flow were set to 0.6 mL/min and 0.06 mL/min, respectively, such that the flow speed of the cell in the channel was approximately 6 m/s. Specifically, the slow sample flow was surrounded by fast sheath flows to form a laminar flow condition in the channel, where the cells were hydrodynamically aligned and focused at the center position of the channel without overlapping with each other. The throughput was calculated to be 15,000 cells/s from the concentration of the cell suspension (1.5 ×107 cells/mL) and the sample infusion rate (0.06 mL/min).
For intelligent image analysis, as shown in Fig. 1(d), the CNN was built with an autoencoder architecture in order to extract key features from the cell images and perform high-accuracy classification. The autoencoder is a type of artificial neural network consisting of an encoder, a decoder, and a classifier [35,36]. Through the compression-reconstruction process, the autoencoder learns the crucial information of cells while discarding the background and noise in images. In our case, the encoder, with four hidden layers, compressed the input images into 1024-dimensional vectors at the bottleneck layer that contained key information of the cell images. Conversely, the decoder part was mirrored with the encoder, which performed the reconstruction of images from the bottleneck layer. The classifier was a 1024-dimensional fully connected layer that transformed the vectors from the bottleneck layer into the probability values of various classes. The CNN was built in Python on Keras [37] functional application program interface with TensorFlow backend [38]. The bright-field and phase images of the same cell were trimmed into 200×200 pixels with only the cell-contained regions and stored in two separate channels. The images were imported and normalized to 0-mean for training the CNN efficiently. Then, the images were randomly divided into three datasets for training, validation, and testing at a ratio of 3:1:1. The CNN was trained on the training dataset using the Adam optimization algorithm, the learning rate of which was 0.001 initially. The parameters of the CNN were optimized to minimize the total loss, which was the summation of the categorical cross-entropy between the true and predicted labels and the mean-squared error between the original and reconstructed images. After the training process in each epoch, the performance of the CNN was evaluated by calculating the total loss of the validation dataset. The learning rate was reduced by a factor of 0.1 when the validation loss stopped decreasing for three epochs until it achieved 1×10−8. The training process was completed when the validation loss stopped descending for 6 epochs, or the model had been trained for 200 epochs. The parameters of the trained CNN were fixed, stored, and tested on the images in the test dataset to generate a confusion matrix of the classification result. Also, the 1024-dimensional feature space was projected to a 2D plane as the t-distributed stochastic neighbor embedding (t-SNE) plot [39].
2.2 Theoretical framework
In OTS-QPI, the space-to-spectrum and spectrum-to-time conversions are the bases of reconstructing the bright-field and phase images of flowing cells from the 1D temporal interferograms (Fig. 1(c)). By applying the diffraction grating for spatial dispersion, the 1D spatial profile of a flowing cell (x) is probed by the pulse spectrum (ω) with a conversion factor (C), such that [3]
Similarly, due to the temporal dispersion in the dispersive fiber, the spectral information of the pulse (ω) is converted to the time domain (t) (i.e., the temporal waveform of the detected signal) with a conversion factor D, such that Although the conversion factor C is theoretically defined by the groove density of the diffraction grating, the magnification of the relay lenses, and the focal length of the objective lens, it is practically calculated and calibrated using a known resolution chart. Meanwhile, the conversion factor D is given by the total GVD of the setup, which mainly comes from the single-mode fiber in our case. The combination of Eqs. (1) and (2) gives indicating that the 1D spatial information of the cell can be obtained from the detected temporal signal.In frequency-shifted OTS-QPI, the bright-field and phase images of a cell are recovered by decoding the temporal waveform that consists of numerous temporal interferograms. Specifically, we shifted the optical frequency of the reference arm to bring the phase information of the cell to the low-frequency region of the signal. In our design, the sample beam with the original optical frequency ωS and the reference beam with the shifted frequency ωR are combined to generate a beat note, which is detected by the photodetector and digitized by the oscilloscope. The intensity of the temporal interferogram can be expressed as
2.3 Cell preparation
WBCs were separated from a whole blood sample for the intelligent OTS-QPI analysis. First, 5 mL of blood was drawn from a healthy donor into a vacuum tube with ethylenediaminetetraacetic acid (EDTA, Terumo Japan) as the anticoagulant. Then, the blood was carefully layered above 5 mL of Polymorphprep (AXIS-SHIELD), a density-gradient medium, in a 15 mL centrifuge tube for the separation of WBCs. The tube was centrifuged at 500 g for 30 minutes in a swing-out rotor at room temperature, after which two cell layers (upper: mononuclear cells; lower: polymorphonuclear cells, i.e., granulocytes) appeared in the tube. Next, the two bands were carefully collected by pipettes and harvested into two centrifuge tubes, in which 0.45% NaCl solution was added at a ratio of 1:1 to restore the normal osmolality of the cells. Finally, the cells were centrifuged at 400 g for 10 minutes, treated with lysis buffer (BioLegend) to remove the red blood cells, resuspended with 5 mL of 0.9% NaCl solution to reach a final concentration of 1.5 ×107 cells/mL, and stored at 4°C for the measurement. This study was approved by the Institutional Ethics Committee in the School of Medicine at the University of Tokyo [No. 11049-(6)]. Written informed consent was obtained from the healthy donors.
The leukemia cell lines, including the acute promyelocytic leukemia (APL) cell line HL-60, the acute T-cell leukemia cell line Jurkat, and the myelogenous leukemia cell line K562, were purchased from RIKEN CELL BANK and recovered from frozen for incubation. The cells were incubated with 5% carbon dioxide (CO2) at 37°C and cultured in 75 cm2 culture flasks (Corning, 430641U) using RPMI-1640 medium (Sigma-Aldrich, R8758) with 10% fetal bovine serum (FBS, Sigma-Aldrich) and 1% Penicillin-Streptomycin solution. The density of the cells was kept between 1×105 cells/mL and 1×106 cells/mL. Before measurements, the cells were centrifuged at 400 g, harvested and resuspended with phosphate-buffered saline (PBS) to reach a final concentration of 1.5 ×107 cells/mL. In the drug-treatment experiment, the HL-60 cells were treated with all-trans retinoic acid (ATRA, Sigma-Aldrich) to induce the differentiation toward granulocyte-like cells. First, the ATRA was dissolved in dimethyl sulfoxide (DMSO, Sigma-Aldrich) to reach a concentration of 2 mmol/L. Second, the HL-60 cells were centrifuged at 400 g, harvested and resuspended with 2 mL of fresh culture medium to reach a density of 1×105 cells/mL, in which 1 µL of the ATRA solution was added. The cells were incubated with 5% CO2 at 37°C for 4 days, harvested with centrifugation and washed with PBS for three times, and then resuspended into 5 mL of PBS for measurements.
3. Results and discussion
3.1 Basic performance
To validate the basic performance of frequency-shifted OTS-QPI, we compared the temporal interferograms and the bright-field and phase images obtained with both OTS-QPI and frequency-shifted OTS-QPI. The temporal interferograms of both methods are shown in Figs. 2(a) and 2(b). Compared with the interferograms of OTS-QPI, the four successive pulses of frequency-shifted OTS-QPI are smoother and less noisy. The conceptual spectra and detection bandwidths of the two methods in the frequency domain are shown in Figs. 2(c) and 2(d), respectively. In OTS-QPI, the phase information must be retrieved from the high-frequency region and is, hence, prone to the high-frequency noise that causes phase-unwrapping errors. Also, a costly wide-bandwidth photodetector and a high-speed analog-to-digital converter (ADC) are required to obtain and digitize the high-frequency signal, resulting in a large data size. In contrast, frequency-shifted OTS-QPI brings the phase information from the high-frequency region to the baseband that can easily be covered by a low-bandwidth photodetector. Moreover, as shown in Figs. 2(e) and 2(f), the bright-field and phase images of two K562 cells were obtained via both methods. Obviously, the phase image obtained with OTS-QPI is blurry and disturbed by high-frequency noise, while the phase image obtained with frequency-shifted OTS-QPI is clear and smooth, showing subtle cellular morphological features. The enhanced signal-to-noise ratio in frequency-shifted OTS-QPI primarily results from shifting the phase information to the low-frequency region as well as the four-line averaging in the image recovery algorithm. In addition, the shadows that appear in the bright-field images, which come from off-axis coupling [16], do not affect the phase images.
Fig. 2. Comparison of OTS-QPI and frequency-shifted OTS-QPI. a. Temporal interferograms of OTS-QPI that contain four successive pulses. b. Temporal interferograms of frequency-shifted OTS-QPI that contain four successive pulses. c. Detection bandwidth for intensity and phase measurements in OTS-QPI. A high-bandwidth photodetector is needed to acquire the phase information from the high-frequency region. d. Detection bandwidth for intensity and phase measurements in frequency-shifted OTS-QPI. A low-bandwidth photodetector is sufficient for acquiring both the intensity and phase information. e. Bright-field and phase images of a K562 cell obtained with OTS-QPI. The phase image is disturbed by high-frequency noise. f. Bright-field and phase images of a K562 cell obtained with frequency-shifted OTS-QPI. The phase image is clear and smooth due to the elimination of high-frequency noise. g. Phase images of a K562 cell recovered with different detection bandwidths. Frequency-shifted OTS-QPI can properly recover the phase images with a much lower detection bandwidth compared with conventional OTS-QPI. h. Phase images of a K562 cell recovered with different sampling rates. Frequency-shifted OTS-QPI can properly recover the phase images at a much lower sampling rate compared with conventional OTS-QPI. Scale bar, 10 µm.
We further investigated the quality of the phase images recovered with different detection bandwidths and sampling rates to demonstrate the advantages of frequency-shifted OTS-QPI. As shown in Fig. 2(g), we applied low-pass Fourier filters with cutoff frequencies of 1.5 GHz, 2.0 GHz, 3.0 GHz, and 4.5 GHz, respectively, to the raw waveforms acquired with both frequency-shifted and conventional OTS-QPI. It is clear that frequency-shifted OTS-QPI properly recovered the phase images with all the above detection bandwidths, while conventional OTS-QPI failed to recover the phase images when the detection bandwidth was lower than 2.0 GHz. Therefore, the detection bandwidth required in frequency-shifted OTS-QPI is lower than 1.5 GHz while that of conventional OTS-QPI is higher than 2.0 GHz. The phase images obtained by the two methods at different sampling rates are also shown in Fig. 2(h). Clearly, in frequency-shifted OTS-QPI, the phase images recovered with sampling rates of 50, 25, 16.7, and 12.5 GS/s were almost identical. In contrast, the phase image obtained with conventional OTS-QPI became distorted when the sampling rate dropped to 16.7 GS/s and could not be recovered at a sampling rate of 12.5 GS/s. Therefore, compared with conventional OTS-QPI, frequency-shifted OTS-QPI has much lower requirements on detection bandwidth and sampling rate, and hence allows for the employment of a lower-cost low-bandwidth photodetector, ADC, data transmission link, signal processor, and storage medium, significantly reducing the cost of the entire system.
To evaluate the spatial resolution of the frequency-shifted OTS-QPI setup, we employed the modulation transfer function (MTF) with a standard USAF1951 resolution chart. The frequency-shifted OTS-QPI setup resolved element 1 in Group 9 (512 line pairs/mm) of the resolution chart, which has a line width of 976.6 nm with an MTF function value of 0.1. By virtue of these advantages, the high-speed acquisition of high-resolution bright-field and phase images was achieved.
3.2 Intelligent detection and classification of leukemia cells
To demonstrate the utility of frequency-shifted OTS-QPI in biomedical applications, we performed intelligent detection of leukemia cells using the frequency-shifted OTS-QPI setup. The bright-field and phase images of WBCs and three types of leukemia cells (HL-60 cells, Jurkat cells, and K562 cells) acquired by the frequency-shifted OTS-QPI setup are shown in Fig. 3(a). The shape and size of the cells are clearly shown in the bright-field images, but few cellular morphological or structural features were evident especially when the cell was slightly out of focus. In the phase images, the shape of the high-phase-shift area of WBCs is irregular and clearly different from those of the leukemia cells, which is presumably because some WBCs had characteristic irregular nuclei while the nuclei of leukemia cells were usually more spherical. Therefore, it was demonstrated that the setup was able to visualize cellular structural information that could not be obtained from the bright-field images. Based on the obtained images, three CNN models were trained using the images of WBCs and each leukemia cell line, respectively, to achieve the intelligent detection of leukemia cells. The bright-field and phase images of 8,000 cells of each type were used as the training data. The confusion matrices of the classification results are shown in Fig. 3(b), showing high classification accuracies of over 99% of each model on the test sets. The t-SNE plots (Fig. 3(c)) indicate clear points of discrepancy between WBCs and leukemia cells in the feature space. To validate the performance of the models in distinguishing leukemia cells spiked in WBCs, each model was further tested on the image dataset of the corresponding spiked sample, i.e., a 9:1 mixture of WBCs and leukemia cells. As shown in Fig. 3(d), the predicted ratios of leukemia cells given by the models (7.78%, 7.67%, and 5.56%, respectively) were close to, but lower than the true value (10%). This may be explained by the bias in data acquisition, that is, more images of WBCs were obtained than leukemia cells at the same trigger level of the oscilloscope due to their different light-scattering abilities.
Fig. 3. Intelligent detection of leukemia cells. a. Image library of the WBCs, HL-60 cells, Jurkat cells, and K562 cells. The arrow indicates the flowing direction. Scale bar, 10 µm. b. Confusion matrices of the classification results of the three models. c. t-SNE plots generated from the bottleneck layer, showing clear discrepancies between WBCs and leukemia cells. d. Predictions of the models on the corresponding spiked samples.
Furthermore, we performed intelligent classification of leukemia cells with the frequency-shifted OTS-QPI setup by training a more comprehensive CNN model with the bright-field and phase images of four types of cells, including WBCs, HL-60 cells, Jurkat cells, and K562 cells. To evaluate the importance of the phase information in classification, another CNN model was trained using only the bright-field images of the four types of cells. The two models were eventually tested with the spiked sample, which contained 70% WBCs, 10% HL-60 cells, 10% Jurkat cells, and 10% K562 cells. The classification results of the model trained with both bright-field and phase images are manifested as confusion matrices (Fig. 4(a)), showing a high average accuracy of 96.20%. The relatively low classification accuracy between Jurkat cells and K562 cells is probably due to their morphological similarities. The t-SNE plot (Fig. 4(b)) also shows clear discrepancies between WBCs and three types of leukemia cells in the feature space. For the model trained with only bright-field images, the confusion matrix (Fig. 4(c)) and the t-SNE plot (Fig. 4(d)) also demonstrated accurate classification with an average accuracy of 97.36%. Note that the two t-SNE plots were generated independently from the high-dimensional feature spaces of two models and hence do not share the same coordinates. On the spiked sample, the predicted ratios of leukemia cells (6.69%, 6.32%, and 11.15%, respectively) of the model trained with both bright-field and phase images were close to the true values (Fig. 4(e)). However, the model trained with only the bright-field images failed to accurately distinguish the leukemia cells in the same spiked sample, with the predicted ratios being 8.48%, 1.79%, and 0.11%, respectively (Fig. 4(f)). Though both models exhibited high classification accuracies on the test datasets, it is clear that the model trained with both images was more robust and reliable than the other, demonstrating the importance of the phase information in classification. This is presumably because the phase images contained more information related to cellular structure, which was more significant in identifying the cell. Another possible reason is that the bright-field images were easily disturbed by random variations such as defocusing, to which the phase images are usually more robust.
Fig. 4. Intelligent classification of leukemia cells. a. Confusion matrix of the model trained with both bright-field and phase images of 8,000 cells of each type. b. t-SNE plot of the model in a. c. Confusion matrix of the model trained with only bright-field images of 8,000 cells of each type of the same dataset. d. t-SNE plot of the model in c. e. Predictions of the model trained with bright-field and phase images on the spiked sample, which are close to the true values. f. Predictions of the model trained with only the bright-field images on the spiked sample, showing poor classification accuracies.
3.3 Intelligent evaluation of drug-treated leukemia cells
In addition to the intelligent detection and classification of leukemia cells, we explored the feasibility of frequency-shifted OTS-QPI in evaluating the drug treatment efficacy of leukemia. Specifically, the ATRA-induced granulocytic differentiation of HL-60 cells, which is also used for differentiation-induction therapy of APL, was investigated as a proof-of-concept demonstration. First, the bright-field and phase images of mononuclear cells, granulocytes, and untreated HL-60 cells (Fig. 5(a)) were used for training a CNN model. As shown in Fig. 5(b), the confusion matrix and the t-SNE plot show that the model reached a high average accuracy of 98.91%. Then, the images of HL-60 cells of the solvent control group (treated with 0.5% DMSO for 4 days) and the drug-treated group (treated with 1 µM ATRA for 4 days) were classified by the trained model. The predictions of the two groups are shown in Figs. 5(c) and 5(d). For the solvent control group, 99.11% of the HL-60 cells were classified as untreated HL-60 cells, indicating that the DMSO-treatment had no effect on the cell morphology. For the ATRA-treated group, 67.63% of the HL-60 cells were classified as granulocytes, which was close to the previously reported ratio and demonstrated that the CNN model was able to correctly identify the differentiated cells [40]. This was also qualitatively observed from the phase images, as some ATRA-treated HL-60 cells exhibited lobed nuclei that were similar to granulocytes. Meanwhile, 17.34% of the cells were classified as HL-60 cells, which implies that some cells were still undifferentiated. Notably, 15.03% of the ATRA-treated cells were unexpectedly classified as mononuclear cells, which is presumably because they shared similar morphological features with the mononuclear cells. These results provided preliminary evidence that the intelligent OTS-QPI setup was able to monitor and evaluate the efficacy of drug-treated leukemia cells in a label-free manner, while the classification results need to be validated with further biological or molecular evidence, as well as more tests with different drugs and cell lines.
Fig. 5. Intelligent evaluation of leukemia treatment. a. Image library of the mononuclear cells, granulocytes, HL-60 cells in the control group, and HL-60 cells in the ATRA-treated group. The arrow indicates the flowing direction. MCs, mononuclear cells. Scale bar, 10 µm. b. Confusion matrix and t-SNE plot of the model trained with bright-field and phase images of mononuclear cells, granulocytes, and untreated HL-60 cells. c. Predictions of the model on the HL-60 cells in the control group. d. Predictions of the model on the HL-60 cells treated with ATRA.
4. Summary
We have proposed and experimentally demonstrated frequency-shifted OTS-QPI that overcomes the limitations of OTS-QPI by shifting the phase information to the baseband region and thus suppressing the phase unwrapping errors induced by high-frequency noise. With this setup, we acquired the bright-field and phase images of cells flowing at 6 m/s with a high throughput of 15,000 cells/s. We further demonstrated the image-based intelligent detection and classification of leukemia cells via deep learning, reaching a high classification accuracy of over 96%. The CNN models were validated on the samples in which leukemia cells were spiked in WBCs, proving that phase information significantly enhanced the classification accuracy. We also explored the feasibility of our method in evaluating the treatment efficacy of leukemia by predicting the ATRA-induced granulocytic differentiation of HL-60 cells. In conclusion, our method opens a new window onto the next-generation leukemia detection and diagnosis by label-free, high-throughput, and intelligent imaging flow cytometry.
Funding
Cabinet Office, Government of Japan; Japan Society for the Promotion of Science; Ministry of Education, Culture, Sports, Science and Technology; White Rock Foundation.
Acknowledgments
We thank Mr. Tadataka Ota and Ms. Yuko Kanda for their kind help with assembling electronic devices and culturing cells. We also thank Dr. Tinghui Xiao, Dr. Yi Wang, Dr. Aikihiro Isozaki, and Dr. Baoshan Guo for their valuable discussions related to this research.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. C. Lei, H. Kobayashi, Y. Wu, M. Li, A. Isozaki, A. Yasumoto, H. Mikami, T. Ito, N. Nitta, T. Sugimura, M. Yamada, Y. Yatomi, D. Di Carlo, Y. Ozeki, and K. Goda, “High-throughput imaging flow cytometry by optofluidic time-stretch microscopy,” Nat. Protoc. 13(7), 1603–1631 (2018). [CrossRef]
2. B. Guo, C. Lei, H. Kobayashi, T. Ito, Y. Yalikun, Y. Jiang, Y. Tanaka, Y. Ozeki, and K. Goda, “High-throughput, label-free, single-cell, microalgal lipid screening by machine-learning-equipped optofluidic time-stretch quantitative phase microscopy,” Cytom. Part A 91(5), 494–502 (2017). [CrossRef]
3. B. Guo, C. Lei, Y. Wu, H. Kobayashi, T. Ito, Y. Yalikun, S. Lee, A. Isozaki, M. Li, Y. Jiang, A. Yasumoto, D. Di Carlo, Y. Tanaka, Y. Yatomi, Y. Ozeki, and K. Goda, “Optofluidic time-stretch quantitative phase microscopy,” Methods 136, 116–125 (2018). [CrossRef]
4. K. C. Lee, M. Wang, K. S. Cheah, G. C. Chan, H. K. So, K. K. Wong, and K. K. Tsia, “Quantitative phase imaging flow cytometry for ultra-large-scale single-cell biophysical phenotyping,” Cytom. Part A 95(5), 510–520 (2019). [CrossRef]
5. A. K. Lau, T. T. Wong, K. K. Ho, M. Y. Tang, A. C. Chan, X. Wei, E. Y. Lam, H. C. Shum, K. K. Wong, and K. K. Tsia, “Interferometric time-stretch microscopy for ultrafast quantitative cellular and tissue imaging at 1 µm,” J. Biomed. Opt 19(7), 076001 (2014). [CrossRef]
6. A. Mahjoubfar, C. Chen, K. R. Niazi, S. Rabizadeh, and B. Jalali, “Label-free high-throughput cell screening in flow,” Biomed. Opt. Express 4(9), 1618–1625 (2013). [CrossRef]
7. K. Goda, K. Tsia, and B. Jalali, “Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena,” Nature 458(7242), 1145–1149 (2009). [CrossRef]
8. K. Goda and B. Jalali, “Dispersive Fourier transformation for fast continuous single-shot measurements,” Nat. Photonics 7(2), 102–112 (2013). [CrossRef]
9. K. Goda, A. Ayazi, D. R. Gossett, J. Sadasivam, C. K. Lonappan, E. Sollier, A. M. Fard, S. C. Hur, J. Adam, C. Murray, C. Wang, N. Brackbilla, D. Di Carlo, and B. Jalali, “High-throughput single-microparticle imaging flow analyzer,” Proc. Natl. Acad. Sci. U. S. A. 109(29), 11630–11635 (2012). [CrossRef]
10. Y. Jiang, C. Lei, A. Yasumoto, H. Kobayashi, Y. Aisaka, T. Ito, B. Guo, N. Nitta, N. Kutsuna, Y. Ozeki, A. Nakagawa, Y. Yatomi, and K. Goda, “Label-free detection of aggregated platelets in blood by machine-learning-aided optofluidic time-stretch microscopy,” Lab Chip 17(14), 2426–2434 (2017). [CrossRef]
11. H. Kobayashi, C. Lei, Y. Wu, C.-J. Huang, A. Yasumoto, M. Jona, W. Li, Y. Wu, Y. Yalikun, Y. Jiang, B. Guo, C.-W. Sun, Y. Tanaka, M. Yamada, Y. Yatomi, and K. Goda, “Intelligent whole-blood imaging flow cytometry for simple, rapid, and cost-effective drug-susceptibility testing of leukemia,” Lab Chip 19(16), 2688–2698 (2019). [CrossRef]
12. H. Kobayashi, C. Lei, Y. Wu, A. Mao, Y. Jiang, B. Guo, Y. Ozeki, and K. Goda, “Label-free detection of cellular drug responses by high-throughput bright-field imaging and machine learning,” Sci. Rep. 7(1), 12454 (2017). [CrossRef]
13. H. Mikami, C. Lei, N. Nitta, T. Sugimura, T. Ito, Y. Ozeki, and K. Goda, “High-speed imaging meets single-cell analysis,” Chem 4(10), 2278–2300 (2018). [CrossRef]
14. C. Lei, B. Guo, Z. Cheng, and K. Goda, “Optical time-stretch imaging: Principles and applications,” Appl. Phys. Rev. 3(1), 011102 (2016). [CrossRef]
15. A. K. Lau, H. C. Shum, K. K. Wong, and K. K. Tsia, “Optofluidic time-stretch imaging–an emerging tool for high-throughput imaging flow cytometry,” Lab Chip 16(10), 1743–1756 (2016). [CrossRef]
16. T. T. Wong, A. K. Lau, K. K. Ho, M. Y. Tang, J. D. Robles, X. Wei, A. C. Chan, A. H. Tang, E. Y. Lam, K. K. Wong, G. C. Chan, H. C. Shum, and K. K. Tsia, “Asymmetric-detection time-stretch optical microscopy (ATOM) for ultrafast high-contrast cellular imaging in flow,” Sci. Rep. 4(1), 3656 (2015). [CrossRef]
17. T. T. Wong, A. K. Lau, K. K. Wong, and K. K. Tsia, “Optical time-stretch confocal microscopy at 1 µm,” Opt. Lett. 37(16), 3330–3332 (2012). [CrossRef]
18. A. Mahjoubfar, D. V. Churkin, S. Barland, N. Broderick, S. K. Turitsyn, and B. Jalali, “Time stretch and its applications,” Nat. Photonics 11(6), 341–351 (2017). [CrossRef]
19. Y. Park, C. Depeursinge, and G. Popescu, “Quantitative phase imaging in biomedicine,” Nat. Photonics 12(10), 578–589 (2018). [CrossRef]
20. T. H. Nguyen, M. E. Kandel, M. Rubessa, M. B. Wheeler, and G. Popescu, “Gradient light interference microscopy for 3D imaging of unlabeled specimens,” Nat. Commun. 8(1), 210 (2017). [CrossRef]
21. M. Shan, M. E. Kandel, and G. Popescu, “Refractive index variance of cells and tissues measured by quantitative phase imaging,” Opt. Express 25(2), 1573–1581 (2017). [CrossRef]
22. G. Popescu, T. Ikeda, R. R. Dasari, and M. S. Feld, “Diffraction phase microscopy for quantifying cell structure and dynamics,” Opt. Lett. 31(6), 775–777 (2006). [CrossRef]
23. H. Majeed, T. H. Nguyen, M. E. Kandel, A. Kajdacsy-Balla, and G. Popescu, “Label-free quantitative evaluation of breast tissue using Spatial Light Interference Microscopy (SLIM),” Sci. Rep. 8(1), 6875 (2018). [CrossRef]
24. B. Bhaduri, H. Pham, M. Mir, and G. Popescu, “Diffraction phase microscopy with white light,” Opt. Lett. 37(6), 1094–1096 (2012). [CrossRef]
25. K. Lee, K. Kim, J. Jung, J. Heo, S. Cho, S. Lee, G. Chang, Y. Jo, H. Park, and Y. Park, “Quantitative phase imaging techniques for the study of cell pathophysiology: From principles to applications,” Sensors 13(4), 4170–4191 (2013). [CrossRef]
26. S. Shin, K. Lee, Z. Yaqoob, P. T. C. So, and Y. Park, “Reference-free polarization-sensitive quantitative phase imaging using single-point optical phase conjugation,” Opt. Express 26(21), 26858–26865 (2018). [CrossRef]
27. S. Shin, D. Kim, K. Kim, and Y. Park, “Super-resolution three-dimensional fluorescence and optical diffraction tomography of live cells using structured illumination generated by a digital micromirror device,” Sci. Rep. 8(1), 9183 (2018). [CrossRef]
28. J. Jung, S.-J. Hong, H.-B. Kim, G. Kim, M. Lee, S. Shin, S. Lee, D.-J. Kim, C.-G. Lee, and Y. Park, “Label-free non-invasive quantitative measurement of lipid contents in individual microalgal cells using refractive index tomography,” Sci. Rep. 8(1), 6524 (2018). [CrossRef]
29. M. Paturzo, A. Finizio, P. Memmolo, R. Puglisi, D. Balduzzi, A. Galli, and P. Ferraro, “Microscopy imaging and quantitative phase contrast mapping in turbid microfluidic channels by digital holography,” Lab Chip 12(17), 3073–3076 (2012). [CrossRef]
30. P. Wang, R. Bista, R. Bhargava, R. E. Brand, and Y. Liu, “Spatial-domain low-coherence quantitative phase microscopy for cancer diagnosis,” Opt. Lett. 35(17), 2840–2842 (2010). [CrossRef]
31. N. Nitta, T. Sugimura, A. Isozaki, H. Mikami, K. Hiraki, S. Sakuma, T. Iino, F. Arai, T. Endo, Y. Fujiwaki, H. Fukuzawa, M. Hase, T. Hayakawa, K. Hiramatsu, Y. Hoshino, M. Inaba, T. Ito, H. Karakawa, Y. Kasai, K. Koizumi, S. Lee, C. Lei, M. Li, T. Maeno, S. Matsusaka, D. Murakami, A. Nakagawa, Y. Oguchi, M. Oikawa, T. Ota, K. Shiba, H. Shintaku, Y. Shirasaki, K. Suga, Y. Suzuki, N. Suzuki, Y. Tanaka, H. Tezuka, C. Toyokawa, Y. Yalikun, M. Yamada, M. Yamagishi, T. Yamano, A. Yasumoto, Y. Yatomi, M. Yazawa, D. Di Carlo, Y. Hosokawa, S. Uemura, Y. Ozeki, and K. Goda, “Intelligent image-activated cell sorting,” Cell 175(1), 266–276.e13 (2018). [CrossRef]
32. A. Isozaki, H. Mikami, K. Hiramatsu, S. Sakuma, Y. Kasai, T. Iino, T. Yamano, A. Yasumoto, Y. Oguchi, N. Suzuki, Y. Shirasaki, T. Endo, T. Ito, K. Hiraki, M. Yamada, S. Matsusaka, T. Hayakawa, H. Fukuzawa, Y. Yatomi, F. Arai, D. Di Carlo, A. Nakagawa, Y. Hoshino, Y. Hosokawa, S. Uemura, T. Sugimura, Y. Ozeki, N. Nitta, and K. Goda, “A practical guide to intelligent image-activated cell sorting,” Nat. Protoc. 14(8), 2370–2415 (2019). [CrossRef]
33. C. L. Chen, A. Mahjoubfar, L.-C. Tai, I. K. Blaby, A. Huang, K. R. Niazi, and B. Jalali, “Deep learning in label-free cell classification,” Sci. Rep. 6(1), 21471 (2016). [CrossRef]
34. G. M. Whitesides, E. Ostuni, S. Takayama, X. Jiang, and D. E. Ingber, “Soft Lithography in Biology and Biochemistry,” Annu. Rev. Biomed. Eng. 3(1), 335–373 (2001). [CrossRef]
35. J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Networks 61, 85–117 (2015). [CrossRef]
36. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Neural Information Processing Systems 2012, 2012, 1097–1105.
37. F. Chollet, “Keras: Deep learning for humans” (Github repository, 2015), retrieved https://github.com/keras-team/keras.
38. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, X. Zheng, and G. Brain, “Tensorflow: A system for large-scale machine learning,” in 12th USENIX Symposium on Operating Systems Design and Implementation, 2016), 265–283.
39. L. v. d. Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res. 9, 2579–2605 (2008).
40. P. Jian, Z. W. Li, T. Y. Fang, W. Jian, Z. Zhuan, L. X. Mei, W. S. Yan, and N. Jian, “Retinoic acid induces HL-60 cell differentiation via the upregulation of miR-663,” J. Hematol. Oncol. 4(1), 20 (2011). [CrossRef]
