Open Access
Add to BibSonomy
Abstract
We present a fast label-free computational flow cytometer based on a strategy of compressive imaging. Scattered light from flowing objects is sub-divided into user-defined basis patterns by a deformable mirror and routed to different detectors associated with each pattern. The patterns can be optimized to be matched to the object features of interest, thus facilitating object identification and separation. Compared to conventional scanning flow cytometers, our technique provides increased information capacity without sacrificing flow velocity. Unique features of our matched-filter strategy are that it can simultaneously probe multiple objects throughout large fields of view with long depths of field. In our proof-of-concept demonstrations, we achieve throughputs of over 10,000 particles/s, working at flow velocities of over 1m/s.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Conventional flow cytometry uses fluorescent or scattered light to measure multiple parameters of cells or organisms, which are used for identification and quantification [1]. Examples of its broad biological and medical applications include disease diagnosis (such as Leukemia) [2], quantifying waterborne bacteria in drinking water [3], and monitoring the immune status of patients by measuring populations of immune cells [4].
Throughput and classification performance are two important properties that characterize flow cytometry performance. However, tradeoffs exist. For example, the most common scanning flow cytometers (SFC) based on light scattering measure only two parameters, the amount of forward scatter (FS) and the amount of side scatter (SS) [5]. Analysis of these two parameters can provide basic size and granularity information, which can be used for cell classification. Since only two detectors are required, this type of flow cytometer can be very fast, allowing flow velocities on the order of meters per second and very high throughput. However, since the information contained in only two parameters is inherently limited, classification is generally rudimentary and simultaneous processing of multiple objects is not achievable. A straightforward way to improve classification performance is to increase the number of measured parameters. This is the premise behind imaging flow cytometers (IFCs), which rely on a variety of imaging techniques to acquire actual images of cells or organisms [6–10]. These images, which provide thousands of signals (pixels), can significantly improve classification performance because of their increased information content. However, IFCs are almost always based on the use of cameras, which impedes system throughput due to lower frame rates. Even with the fastest cameras available (frame rates in the kHz range), flow speeds of meters per second are completely unattainable. To increase throughput, interesting variants of IFC have emerged making use of spectral information in addition to spatial information [11,12]. For example, ultrafast IFC can be obtained using time-stretch spectrometers, providing amplitude [13] or phase [14] images, though at the cost of technical complexity. Moreover, the efficient reconstruction and analysis of high dimensional image data can be challenging, requiring sophisticated algorithms [15–17] or dedicated computational hardware [13].
An alternative strategy to increase information content involves not necessarily increasing the number of measured parameters, but increasing the information contained within each parameter, by making the parameters as orthogonal as possible. As an example, polarimetry-based flow cytometers measure the four orthogonal components of the Mueller matrix and achieve 200 cell/s throughput rates [18]. Another strategy combines pupil engineering with IFC [19], making use of filters in the pupil plane to selectively enhance sample features of interest. These are generally used in conjunction with cameras, since their applications generally do not require high throughput. Recently, the idea of matched-filtering has been applied with the use of a single-pixel camera [20], which is an example of compressive imaging [21,22].
In this work, we describe a label-free computational flow cytometer based on a similar concept of compressive imaging. We do not obtain the original images of our sample, but rather we detect a few (here four) parameters that are selected to enable optimal task-based classification of particular types of particles. This idea is a generalization of the single-pixel strategy presented in [20, 23], except that instead of compressing the image to a single pixel, we compress it to a small number of pixels, so that we significantly increase the information content of each measurement without sacrificing acquisition speed.
A unique feature of our design is that we use partially coherent illumination. Since coherence is needed for the generation of scattering patterns, FS/SS flow cytometers and scattering-pattern-based IFCs almost always make use of lasers, which provide fully coherent illumination [24,25]. But full coherence can lead to spurious interference between multiple particles within the field of view (FOV), confounding signal interpretation. In other words, full coherence limits the number of particles that can be monitored at any given time to a maximum of one. On the other hand, if the illumination is partially coherent, with coherence area only slightly larger than the particle size, then particles from different regions of the FOV produce scatter patterns that do not interfere but rather add linearly in intensity (See Fig. 1). This property of linearity enables us to detect and identify multiple particles simultaneously within a much wider FOV without danger of crosstalk, thus allowing higher throughput. For example, we demonstrate count rates of over 10,000 particles/sec for two types of particles with high classification accuracy. Theoretically we can achieve still higher throughput by further increasing the FOV and flow velocity, both of which are feasible.
Fig. 1 Schematics of the scattering patterns with fully and partially coherent illumination. Fully coherent laser illumination causes scattered light from different particles to interfere. With partially coherent illumination, patterns add linearly, facilitating the identification of multiple particles within a large FOV.
2. Methods
Our matched-filter compressive imaging (MFCI) flow cytometry setup is illustrated in Fig. 2, and is based on a standard widefield microscope design. A Thorlabs 625nm LED sends light through a 5× objective to provide transillumination. The spatial coherence area of the illumination is adjusted with a simple adjustable iris in the illumination Fourier plane, such that the illumination is fully coherent over the scale of the particles of interest, but incoherent across different particles. The scattered light produced by one or more particles is then collected by a 20× 0.46 NA Olympus objective. The scattering pattern is projected onto a pupil plane, where a reconfigurable deformable mirror (DM, Boston Micromachines Corp. Hex-1011) subdivides the pupil plane into four non-overlapping user-defined basis patterns. The DM mirror is comprised of 337 hexagonal segments, each supported by three surface-normal actuators that allow independent control of tip-tilt and piston. In other words, the light incident on each segment can be steered to an arbitrary direction [26]. The light incident on each of the four basis patterns is then routed to four different directions, specifically four different quadrants of a high-speed 2×2 array detector (quadrant detector, QD, SensL). Each quadrant integrates the incident intensity over a 6×6mm2 area. To help equalize the signals, a square neutral density filter is placed in front of one of the quadrants (the one with the strongest signal, corresponding to ch1) to ensure that the detectors operate within their dynamic range. The signals are then independently amplified and acquired by a 4-channel high-speed data acquisition board (NI, PCI 6110). Downstream classification/counting is performed with only the signals from these four channels. In addition, to verify our result, a Thorlabs CCD camera is configured to directly image the flow channel. Flow is provided by a syringe pump (Harvard Apparatus) applied to a custom polydimethylsiloxane (PDMS) microfluidic flow channel, of transverse width 500μm and axial depth 25μm. The signals are first sampled at 5MHz, lowpass-filtered, and then downsampled to 200kHz to improve signal-to-noise ratio (SNR). Such temporal resolution can easily accommodate flow velocities on the order of m/s.
Fig. 2 (a) Schematic of our MFCI flow cytometry setup. Dotted lines indicate conjugate object/image planes. For simplicity, the camera that images the DM plane is not shown. (b) Schematic of four basis patterns on the DM. Actuator tilts are applied such that each group of similarly tilted segments redirects light onto an associated detector/channel. (c) Image of the DM plane without the application of actuator voltages (i.e. flat). The size of the bright spot is an indication of spatial coherence range, controlled by the adjustable iris, which is adjusted here to focus most of the light power onto the central DM segment (ch1)
A key component of our MFCI strategy is the design of the non-overlapping matched filters in the detection pupil plane (i.e. the DM tilt patterns). In principle, an arbitrary pattern selection could provide signal diversity that could enable the separation of different particle types. However, to obtain optimal particle separability, a calibration procedure is prescribed. Such optimal separability is achieved when the matched-filter pattern minimizes the correlation between signals from different particle types, In other words, the optimal pattern depends on the specific features of the particle types of interest. In this work, proof-of-principle demonstrations are performed with polystyrene beads of diameters 6μm and 10μm (Polysciences, Inc.). Since the beads are circularly symmetric, the filters are also chosen to be circularly symmetric, in the shape of annuli of user-defined inner and outer radii. In order to calibrate our system, we performed a grid search with different combinations of annuli geometries and illumination iris sizes such that the signal changes caused by the two types of beads produced minimum linear correlation. This filter calibration process was relatively time-consuming. However, once optimized, the filters remained static. It should be noted that, once optimized, the DM could in principle be replaced by a low-cost interchangeable diffractive optical element (DOE). The advantage of using a reconfigurable DM here is that it facilitates the task-based filter search for fast prototyping.
Because of the linear property of our system, when multiple particles appear in the same FOV, the detected (intensity) signals are linear combinations of the signals generated by individual beads, which can be written as S(t) = T · Np(t), where Np(t) is the number of particles at time t (here a 2 × 1 vector), S(t) is the four signals measured at t (a 4 × 1 vector), and T is the scattering matrix comprising the characteristic basis signals produced by each type of particle (a 4 × 2 matrix). This linear relationship allows us to formulate the identification process as a straightforward linear inverse problem: Np(t) = T−1S(t). Though a simple pseudo-inverse can provide reasonable results, we also make use of a priori information. Specifically, the number of particles in the FOV is known to be a non-negative integer. We can impose this constraint on the least-squares computation to improve robustness. This problem is known as an overdetermined box-constrained integer least squares (OBILS) problem, and we applied the MILES algorithm [27] to perform the reconstruction. We note that MILES is fast when there are only two types of particles, though the efficiency is expected to decrease exponentially with increasing number of particle types (that is, OBILS is a NP-hard problem).
It should be emphasized that the accurate classification of multiple particles in the same FOV is not always straightforward. Conventional flow cytometers can only monitor single particles at a time and rely on a variety of techniques to prevent particle clustering [28]. With IFCs, the counting process can be done using computer vision or deep learning algorithms [29] at increased computation cost. Even so, frames with multiple/clustered particles are generally intentionally discarded to avoid erroneous readings [13].
3. Results
We begin by evaluating the performance of our MFCI flow cytometer with low flow velocity, in order to validate our data processing algorithms and classification accuracy. The raw signals are first corrected for background levels (measured with no particles in the flow channel), different amplifications and the ND filter, thus arriving at the corrected signals (S(t)). The characteristic vectors for each type of particle, embodied by the scattering matrix T, are predetermined from the calibration procedure. As explained above, S(t) is a linear combination of the characteristic vectors weighted by the numbers of beads (Np(t)). A simple least-square (LS) fit can be applied to S(t) to infer the number and types of particles within the FOV (see Fig. 3(b,c)). By plotting a histogram of each trace, we observe that the inferred numbers are clustered about non-negative integers, as expected. The LS results with non-negative integer constraint are also shown (Fig. 3(d,e)), as are a set of ground truth images acquired by the CCD camera, provided for comparison (these ground-truth images are acquired at a very low velocity to obtain well-resolved images). We manually counted the numbers and types of particles in each ground-truth image and compared with the results obtained from reconstruction. The reconstruction results were found to be correct for 91% of the images (134 images were processed), where a discrepancy in either the number or type of particle constituted an error. In other words, our MFCI strategy provides a classification accuracy on par with image-based FC. To confirm that four detection channels provided greater classification accuracy than two channels, we attempted classification using only ch1 and ch2. This led to a poorer classification accuracy systematically worse than 40%.
Fig. 3 (a) Corrected signals from the four channels. Conventional least square (LS) reconstruction is applied to the signals. Two traces representing the number of 6μm beads and 10μm beads are shown in (b) and (c), respectively. To the right of each trace is the corresponding histogram, showing that most numbers cluster about non-negative integers. (d) and (e) are the LS reconstruction results with non-negative integer constraint. (f) compares our reconstruction results with results obtained from ground truth images. Five example images (acquired by the CCD camera) are shown, associated with the red dotted lines in (d) and (e). A reconstruction is considered accurate when both number and types of beads are correct. We achieved 91% accuracy in this experiment.
But the key advantage of our MFCI strategy over image-based FC is its throughput. While IFC throughput is limited by the camera frame rate, in our case we use detectors that can operate well over MHz rates. We demonstrate this throughput advantage by increasing flow velocity (see Fig. 4). Although our pump provides a constant debit (volume/sec), we found that the local flow velocity varied somewhat at different locations along the channel (laminar flow is assumed), presumably due to structural imperfections in our channel. A “findpeaks” function (MATLAB) was applied to the number-of-beads curve (LS with constraint) to obtain heights and widths of the peaks (Fig. 4(a) – note that the 10μm beads were processed similarly though the data is not shown). A plot of the histogram of the peak widths allowed us to estimate the local average bead transit time to be 64μs. Since the transit length of our rectangular FOV is about 80μm, we estimate the local flow velocity to be about 1.25m/s, much faster than can be achieved with an IFC, and on par with flow velocities attained with conventional scanning FCs. Then we estimate the throughput of the system. This is done by simply summing the heights of all the peaks found earlier (see Fig. 4(a)). The resulting throughput is over 10,000 particles/s. We confirmed that the concentration ratio of the two bead types (6μm:10μm) remained about 4.67, which is close to the ratio of 4.79 measured at slow flow velocity.
Fig. 4 Estimation of flow velocity and throughput. (a) Number of 6μm beads, with peaks indicated. (b) Distribution of the width of the peaks. The median peak width is used to estimate local flow velocity. (c) Estimated throughput is obtained by summing the peak values in (a), and inferred to be more than 10,000 particles/s.
Yet another advantage of our MFCI strategy compared to an IFC relates to depth of field (DOF). Specifically, when particles are out of focus in an IFC, they become blurred. Such blurring hinders accurate classification and imposes severe constraints on the maximum channel depth. In our case, defocus leads to (quadratic) phase changes of the scattered field in the DM plane, not amplitude changes. While such phase changes, in turn, lead to a spreading of the signal powers at the detection plane, we are insensitive to such spreading because our detectors are large in area (6 × 6mm2). In other words, the measured signals are quite insensitive to defocus. To illustrate this advantage, we compared signals obtained with the quadrant detector and the camera when defocusing two static 10μm beads mounted on a slide. We reconstructed the number of beads using our MFCI procedure and compared this to the contrast (standard-deviation/mean) calculated in the red-circled area in the image stack. We note that these two curves do not strictly represent DOF but are close proxies that allow us to compare the robustness of each setup against defocus. We observe that image contrast almost completely disappears at 50μm defocus, meaning any image based classification/count algorithm would fail. In contrast, the particle number reconstructed by our technique continues to remain accurate, suggesting a much improved robustness against defocus and a capacity to work with channels even greater than 100μm in depth (in our demonstration, the channel depth was 25 μm). In other words, MFCI allows us to probe not only large sample areas, but also large sample volumes, enabling higher throughput still. To illustrate this advantage, we compared signals obtained with the quadrant detector and the camera when defocusing two static 10µm beads mounted on a slide (Fig. 5)).
4. Conclusion
In summary, we have demonstrated a MFCI flow cytometer where basis pupil patterns are “matched” to the samples of interest. The technique combines the increased information capacity of image-based FCs with the increased throughput capacity of conventional non-image-based FCs. Specifically, a high throughput capacity is enabled not only by our use of fast detectors (as opposed to a camera), but also by the ability of our device to classify multiple particles within a same FOV. This last advantage comes from our use of partially coherent illumination (as opposed to laser illumination) designed to ensure signal linearity. Moreover, our system provides much larger DOF than image-based FCs, allowing the monitoring of particles throughout larger flow volumes, also facilitating higher throughput and enabling classifications rates of 10,000 particles/s with flow velocities of over 1 m/s.
One limitation of our technique is that the optimization of the basis patterns requires a labor-intensive search. In our proof-of-concept demonstration the problem was reduced to finding optimal sizes of the annulus-shaped DM patterns. However, designing optimized patterns becomes more challenging when the particles become more varied or structured. This problem is directly analogous to the kernel design problem in machine learning, where the goal is to maximally separate data in a high-dimensional space with as few coefficients as possible. In future work, we plan to derive basis pupil patterns using a machine learning model for automated optimization.
Our MFCI FC technique is conceptually simple and potentially low-cost. Here, we used a specialized tip-tilt DM to search for and apply our matched-filter patterns. In practice, this could be done more cost-effectively with diffractive optics elements, pre-calibrated experimentally or designed based on a physical model of the scattering of interest. The simplicity of our device, along with its information capacity and throughput advantages, make it attractive for biomedical applications.
Fig. 5 Comparison of the depth of field (DOF) allowed for MCFI flow cytometry versus camera-based widefield imaging (IFC). Top trace shows reconstructed particle number as a function of defocus (correct answer is 2). Bottom trace shows image contrast within red-circle delimited FOV (also shown are representative images associated with dashed lines).
Funding
National Institute of Health (R21GM128020); National Science Foundation (IIP-1068070).
Acknowledgments
We thank Jenny Wang and Yixin Wang for initial help on the development of microfluidic flow channels.
Disclosures
Thomas Bifano acknowledges a financial interest in Boston Micromachines Corporation, which makes the Hex-1011 deformable mirror system that was loaned to Boston University for this work.
References
1. V. V. Tuchin, Advanced optical flow cytometry: methods and disease diagnoses (John Wiley & Sons, 2011). [CrossRef]
2. C. Alix-Panabières and K. Pantel, “Circulating tumor cells: liquid biopsy of cancer,” Clin. Chem. 59, 110 (2012).
3. F. Hammes, M. Berney, Y. Wang, M. Vital, O. Köster, and T. Egli, “Flow-cytometric total bacterial cell counts as a descriptive microbiological parameter for drinking water treatment processes,” Water Res. 42, 269–277 (2008). [CrossRef]
4. S. P. Perfetto, P. K. Chattopadhyay, and M. Roederer, “Seventeen-colour flow cytometry: unravelling the immune system,” Nat. Rev. Immunol. 4, 648 (2004). [CrossRef] [PubMed]
5. A. N. Shvalov, I. V. Surovtsev, A. V. Chernyshev, J. T. Soini, and V. P. Maltsev, “Particle classification from light scattering with the scanning flow cytometer,” Cytom. The J. Int. Soc. for Anal. Cytol. 37, 215–220 (1999).
6. D. Dannhauser, D. Rossi, F. Causa, P. Memmolo, A. Finizio, T. Wriedt, J. Hellmers, Y. Eremin, P. Ferraro, and P. Netti, “Optical signature of erythrocytes by light scattering in microfluidic flows,” Lab on a Chip 15, 3278–3285 (2015). [CrossRef] [PubMed]
7. Y. Han, Y. Gu, A. C. Zhang, and Y.-H. Lo, “imaging technologies for flow cytometry,” Lab on a Chip 16, 4639–4647 (2016). [CrossRef]
8. T. Blasi, H. Hennig, H. D. Summers, F. J. Theis, J. Cerveira, J. O. Patterson, D. Davies, A. Filby, A. E. Carpenter, and P. Rees, “Label-free cell cycle analysis for high-throughput imaging flow cytometry,” Nat. communications 7, 10256 (2016). [CrossRef]
9. F. Merola, P. Memmolo, L. Miccio, R. Savoia, M. Mugnano, A. Fontana, G. D’ippolito, A. Sardo, A. Iolascon, A. Gambale, et al., “Tomographic flow cytometry by digital holography,” Light. Sci. & Appl. 6, e16241 (2017). [CrossRef]
10. M. M. Villone, P. Memmolo, F. Merola, M. Mugnano, L. Miccio, P. L. Maffettone, and P. Ferraro, “Full-angle tomographic phase microscopy of flowing quasi-spherical cells,” Lab on a Chip 18, 126–131 (2018). [CrossRef]
11. L. Golan, D. Yeheskely-Hayon, L. Minai, and D. Yelin, “High-speed interferometric spectrally encoded flow cytometry,” Opt. letters 37, 5154–5156 (2012). [CrossRef]
12. L. Golan, D. Yeheskely-Hayon, L. Minai, E. J. Dann, and D. Yelin, “Noninvasive imaging of flowing blood cells using label-free spectrally encoded flow cytometry,” Biomed. optics express 3, 1455–1464 (2012). [CrossRef]
13. K. Goda, A. Ayazi, D. R. Gossett, J. Sadasivam, C. K. Lonappan, E. Sollier, A. M. Fard, S. C. Hur, J. Adam, C. Murray, et al., “High-throughput single-microparticle imaging flow analyzer,” Proc. Natl. Acad. Sci. 109, 11630–11635 (2012). [CrossRef] [PubMed]
14. T. T. Wong, A. K. Lau, K. K. Ho, M. Y. Tang, J. D. Robles, X. Wei, A. C. Chan, A. H. Tang, E. Y. Lam, K. K. Wong, et al., “Asymmetric-detection time-stretch optical microscopy (atom) for ultrafast high-contrast cellular imaging in flow,” Sci. reports 4, 3656 (2014). [CrossRef]
15. Y. Saeys, S. Van Gassen, and B. N. Lambrecht, “Computational flow cytometry: helping to make sense of high-dimensional immunology data,” Nat. Rev. Immunol. 16, 449 (2016). [CrossRef] [PubMed]
16. Y. Jo, J. Jung, M.-h. Kim, H. Park, S.-J. Kang, and Y. Park, “Label-free identification of individual bacteria using fourier transform light scattering,” Opt. express 23, 15792–15805 (2015). [CrossRef] [PubMed]
17. S. Ota, R. Horisaki, Y. Kawamura, M. Ugawa, I. Sato, K. Hashimoto, R. Kamesawa, K. Setoyama, S. Yamaguchi, K. Fujiu, K. Waki, and H. Noji, “Ghost cytometry,” Science 360, 1246–1251 (2018). [CrossRef] [PubMed]
18. D. I. Strokotov, A. E. Moskalensky, V. M. Nekrasov, and V. P. Maltsev, “Polarized light-scattering profile—advanced characterization of nonspherical particles with scanning flow cytometry,” Cytom. Part A 79, 570–579 (2011). [CrossRef]
19. J.-Y. Zheng, R. M. Pasternack, and N. N. Boustany, “Optical scatter imaging with a digital micromirror device,” Opt. express 17, 20401–20414 (2009). [CrossRef] [PubMed]
20. J. Zhou, Z. Cao, H. Xie, and L. Xu, “Digital micro-mirror device-based detector for particle-sizing instruments via fraunhofer diffraction,” Appl. optics 54, 5842–5849 (2015). [CrossRef]
21. D. Takhar, J. N. Laska, M. B. Wakin, M. F. Duarte, D. Baron, S. Sarvotham, K. F. Kelly, and R. G. Baraniuk, “A new compressive imaging camera architecture using optical-domain compression,” in Computational Imaging IV, vol. 6065 (International Society for Optics and Photonics, 2006), p. 606509. [CrossRef]
22. M. A. Davenport, M. F. Duarte, M. B. Wakin, J. N. Laska, D. Takhar, K. F. Kelly, and R. G. Baraniuk, “The smashed filter for compressive classification and target recognition,” in Computational Imaging V, vol. 6498 (International Society for Optics and Photonics, 2007), p. 64980H. [CrossRef]
23. M. Copeland, I. Price, F. Rigaut, G. Bloxham, R. Boz, D. Bundy, B. Espeland, and R. Sharp, “Gmtifs: deformable mirror environmental testing for the on-instrument wavefront sensor,” in Adaptive Optics Systems V, vol. 9909 (International Society for Optics and Photonics, 2016), p. 990980. [CrossRef]
24. D. Tycko, M. Metz, E. Epstein, and A. Grinbaum, “Flow-cytometric light scattering measurement of red blood cell volume and hemoglobin concentration,” Appl. optics 24, 1355–1365 (1985). [CrossRef]
25. W. Huang, L. Yang, G. Yang, and F. Li, “Microfluidic multi-angle laser scattering system for rapid and label-free detection of waterborne parasites,” Biomed. optics express 9, 1520–1530 (2018). [CrossRef]
26. J. B. Stewart, T. G. Bifano, S. Cornelissen, P. Bierden, B. M. Levine, and T. Cook, “Design and development of a 331-segment tip–tilt–piston mirror array for space-based adaptive optics,” Sensors Actuators A: Phys. 138, 230–238 (2007). [CrossRef]
27. X.-W. Chang and T. Zhou, “Miles: Matlab package for solving mixed integer least squares problems,” GPS Solutions 11, 289–294 (2007). [CrossRef]
28. D. Di Carlo, D. Irimia, R. G. Tompkins, and M. Toner, “Continuous inertial focusing, ordering, and separation of particles in microchannels,” Proc. Natl. Acad. Sci. 104, 18892–18897 (2007). [CrossRef] [PubMed]
29. W. Xie, J. A. Noble, and A. Zisserman, “Microscopy cell counting and detection with fully convolutional regression networks,” Comput. methods biomechanics biomedical engineering: Imaging & Vis. 6, 283–292 (2018).
