Open Access
Add to BibSonomy
Abstract
The precise, quantitative evaluation of intracellular organelles in three-dimensional (3D) imaging data poses a significant challenge due to the inherent constraints of traditional microscopy techniques, the requirements of the use of exogenous labeling agents, and existing computational methods. To counter these challenges, we present a hybrid machine-learning framework exploiting correlative imaging of 3D quantitative phase imaging with 3D fluorescence imaging of labeled cells. The algorithm, which synergistically integrates a random-forest classifier with a deep neural network, is trained using the correlative imaging data set, and the trained network is then applied to 3D quantitative phase imaging of cell data. We applied this method to live budding yeast cells. The results revealed precise segmentation of vacuoles inside individual yeast cells, and also provided quantitative evaluations of biophysical parameters, including volumes, concentration, and dry masses of automatically segmented vacuoles.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Biological cells, with their complex structure and specialized organelles separate from the cytoplasm, are the subject of increasingly intricate study. As the physical properties of intracellular compartments such as density and crowding come under focus [1,2], their tightly regulated, functionally relevant characteristics have been revealed [3]. Deviations from normal density have been linked with altered cell function during starvation [4], cellular senescence [5], and differentiation [3,6–8]. Furthermore, shifts in cytoplasm density can influence essential processes like phase separation [9–11], signaling [12,13], and growth [14,15]. However, the limited availability and scalability of methods for studying cell density at subcellular resolution pose significant challenges [16].
For instance, in budding yeast (Saccharomyces Cerevisiae), whole-cell density measurements are heavily skewed by the vacuole, an organelle with a function analogous to the mammalian lysosome and less dense than the rest of the cell. Therefore, an accessible, high-throughput method for automatic segmentation of vacuoles and cytoplasm is required.
Fluorescence (FL) microscopy, employing the use of labeled organelles, has served as a pivotal tool in imaging intracellular compartments within budding yeast. The vacuole, a prominent subcellular structure, is commonly visualized through the employment of Vph1 tagged with green fluorescent protein (GFP). Vph1 is an integral membrane protein localized primarily to the vacuolar membrane [17]. Traditional 3D FL imaging methods entail stacking several axial images and implementing deconvolution algorithms. Alternatively, confocal or light sheet microscopy techniques can be employed for three-dimensional imaging acquisition. While these methods are effective in identifying intracellular structures, they may induce phototoxicity and provide limited information on the properties of the cytoplasm.
Fig. 1. Volumetric segmentation via FL and RI correlative imaging. (a) Traditional FL microscopy utilizes axial scanning to generate a 3D image of Vph1-GFP. Although the deconvolved 3D data facilitate the identification of vacuole morphology, this method is constrained by photobleaching and limitations in quantifying cell density. (b) HT applies off-axis holography and angular scanning of incident plane waves to reconstruct an RI tomogram. While RI offers label-free contrast convertible to cell density, it lacks the required specificity of the vacuolar protein Vph1 for accurate analysis. (c) In this study, we employed correlative imaging to quantify cell density of vacuoles and cytoplasm with enhanced specificity. We also utilized machine learning for the automatic 3D segmentation of vacuoles.
Quantitative phase imaging (QPI) techniques offer a solution by providing a label-free method for precise quantification of intracellular organization [18] (Fig. 1(b)). These techniques employ interferometric microscopy to scan a plane wave angularly and reconstruct the volumetric distribution of the refractive index (RI). As RI increases proportionally with dry-mass concentration [19], this method offers dry-mass density as image contrast without the use of fluorescent labels. However, the lack of chemical specificity in RI imaging presents challenges for accurate segmentation of intracellular components.
To surmount these limitations, we have developed a hybrid machine-learning framework that utilizes correlative imaging of 3D QPI, also known as holotomography (HT), and 3D FL in labeled cells (Fig. 1(c)). Our method is implemented from our previously developed methodology, where a neural network is trained using the correlative imaging dataset and subsequently applied to HT of cell data [20]. This method successfully merges the chemical specificity of fluorescence imaging with the quantitative capabilities of HT. Additionally, the incorporation of a machine learning framework enables automatic volumetric segmentation from raw images, thereby streamlining high-content analysis. Application of this method enabled us to quantitatively analyze the cytoplasm and vacuoles in live budding yeast cells in their native states. This study highlights its potential applicability in investigating intracellular density and its relationship with cellular function.
2. Methods
2.1 Sample preparation
Budding yeast cultures (Saccharomyces Cerevisiae, S288C BY4741) were cultured in synthetic complete (SC) medium at 30°C. Overnight grown cultures were diluted to an optical density (OD600) of 0.1 with fresh SC and allowed a 2-hour recovery period before imaging during the log phase or treating with rapamycin (0.1 or 1 µM) dissolved in ethanol. We imaged rapamycin-treated cells post 2-hour treatment. Stationary phase cells were imaged 24 hours after initial inoculation. For imaging purposes, we coated the glass bottom of the Tomodish (Tomocube Inc.) with 2 mg/mL Concanavalin A (Sigma) for 15 minutes, rinsed twice with SC, allowed the cells to attach for 5 minutes, followed by a single SC wash, and finally covered with a coverslip.
2.2 Optical imaging and reconstruction
For imaging budding yeasts, we used a HT system equipped with 3D FL imaging capability (HT-2 H; Tomocube Inc.) [21]. HT is an optical counterpart to X-ray computed tomography. It operates by obtaining multiple 2-D optical field images at various illumination angles. From these measurements, a 3D RI tomogram of a sample is reconstructed by inversely solving a wave equation [22,23]. Due to its label-free and quantitative imaging capability, HT has been utilized for the study of cell biology [24], biophysics [25], neuroscience [26], regenerative medicine [27,28], and organoids [29].
The imaging system was equipped with water-immersion objective lenses (UPLSAPO60XW, numerical aperture = 1.2; Olympus Inc.). In FL mode, we scanned the sample stage to image raw stacks of Vph1-GFP-expressed yeasts with an axial spacing of 300 nm. These raw data were then deconvolved using the built-in software (Tomostudio, Tomocube Inc.) to generate processed volumetric images with enhanced resolution [30].
To acquire RI data, we captured 49 raw off-axis holograms in an off-axis Mach-Zehnder interferometer by angularly scanning a green plane wave using a digital micromirror device [31–33]. An RI tomogram was reconstructed from these holograms in the built-in software using the Rytov approximation and non-negativity constraints [34]. The theoretical lateral and axial resolution of HT were 110 nm and 360 nm, respectively [35]. The RI and FL data were registered and resized to achieve an isotropic voxel pitch of 200 nm. A regularization algorithm based on non-zero criteria was employed to mitigate the ‘missing cone’ problem, a phenomenon arising from the inability to access side scattering signals due to the limited numerical aperture of the objective lens [34].
2.3 Machine-learning-based segmentation
For high-throughput intracellular organization analysis, automated segmentation is key. We achieved this by implementing a machine-learning framework (Fig. 2). We explored the application of various classifier algorithms, including the support vector machine. However, we found these algorithms to be relatively less accurate in comparison to the alternatives we ultimately chose to use in our study. Instead, we utilized ilastik, an open-source machine-learning annotation software, to prepare 3D labels of vacuoles [36] (Fig. 2(a)).
Fig. 2. Data flowchart for automated 3D vacuole segmentation. (a) Sparse masks are annotated across XY, XZ, and YZ cross sections of 3D RI and FL Vph1-GFP images to prepare vacuole labels. These annotations are used to train a random forest segmentation algorithm, producing random-forest labels (RF). The dataset is then curated by excluding misregistered and poorly segmented data. (b) The neural network (NN) is trained using RI and the RF labels from (a) as inputs and training labels, respectively. The trained NN subsequently predicts 3D vacuole masks using only RI tomograms. (c) Depicts the architecture of the NN used for the segmentation task.
In each dataset, we manually annotated cytoplasm and vacuoles in at least 3 slices of XY, XZ, and YZ cross sections. A total of 163 individual cells from 21 randomly selected FL and RI datasets were annotated manually, training a random-forest (RF) classifier [28] built into the software. The following selected features were used; five Gaussian smoothing parameters (σ = 0.3, 0.7, 1.6, 3.5, and 10), three edge parameters (σ = 0.7, 1, and 5), and three texture parameters (σ = 1.6, 5, and 10) for each RI and FL image. This algorithm was then applied to 156 datasets, from which 69 datasets were curated as well-segmented, excluding those with artifacts due to movement during imaging or inaccurate segmentation.
To improve our framework and eliminate reliance on FL data, we introduced a deep neural network (NN) designed to segment vacuoles using solely RI data (Fig. 2(b)). To train the network, we used the curated set of cells that were segmented based on the Vph1 vacuolar signal. Despite the exclusive localization of Vph1 to the vacuole membrane [17], its presence proved adequate for differentiating the vacuole from other cellular structures. The input images for this network were RI tomograms, normalized from [1.35, 1.45] float to [0, 255] integer formats. The labels used in the network were the same as those obtained from ilastik.
Our network was adapted from a classical 3D U-Net used for brain tumor segmentation [37] (Fig. 2(c)). The structure comprises a down-sampling encoder pathway and up-sampling decoder pathway [38], each with its unique components. Each encoder path includes a context module that consists of two 3 × 3 × 3 convolution layers and a dropout layer (p = 0.75) in between. The context module is connected by a 3 × 3 × 3 convolution layer with an input stride 2. Each decoder path includes a localization module that has a 3 × 3 × 3 convolution layer followed by a layer that reduces the number of feature maps by half. The localization module is up-sampled and concatenated with the features from the corresponding level of the context module. A segmentation layer is set up at different levels, and the network outputs are combined to form the final network output. Nonlinear activations are achieved through leaky ReLU activation functions with a 10−2 negative slope and instance normalization [39].
We trained the architecture on a cloud computing platform (Google Colaboratory Pro plus, Google Inc.) using Python Pytorch code. We segmented 354 individual cells from 69 datasets and divided them into 266 training and 88 validation sets. With a batch size of 8, the network was trained for 200 epochs. The loss function comprised 90% binary cross entropy with a logistic function and 10% focal Tversky function (α = 0.25, β = 0.75, and γ = 2) [40], minimized by the Adam optimizer [41] with a learning rate of 10−4.
2.4 Quantitative analysis of vacuole and cytoplasm
To obtain the segmented vacuole masks, we used either the RF algorithm or the NN, depending on which algorithm provided larger Pearson coefficients with the RI image. To segregate the cytoplasm from the surrounding medium, we refrained from employing Random Forest (RF) or Neural Network (NN) algorithms. This decision was based on our observation that conventional thresholding methods were quite sufficient for this particular task. In detail, we defined the cytoplasm mask by thresholding the RI with the average value between the mean cytoplasm RI defined in ilastik and the mean vacuole RI, followed by binary erosion with one voxel. Note that we did not use the binary erosion and the RI average thresholding on the vacuole segmentation. The level of precision required for this segmentation task surpasses what can be achieved by employing simple thresholding algorithms. In the discussion section, we will elaborate on the advantages of utilizing RF or NN methods over the RI average thresholding technique specifically for vacuole segmentation. Finally, we converted the obtained RI to dry-mass concentration using the RI increment of 0.1907 mL/g [42].
3. Results
3.1 Validation of segmentation performance
We evaluated the segmentation performance of both the RF classifier and the deep NN for vacuoles (Fig. 3). We first examined the cross-sections of individual cell data for both the training and untrained datasets (Figs. 3(a) and (b)). In the training dataset, we curated data of 211 cells for which either RF labels or deep-learning outputs were well-segmented (Figs. 3(a) and (c)). As expected, the comparison of RI, FL, RF labels, and NN outputs indicated that both algorithms provided well-segmented vacuole masks. Additionally, we curated 268 cell data from the untrained dataset (Figs. 3(b) and (d)). We found that the RF labels were inaccurate when FL images were photobleached during the long fluorescence imaging or misregistered when the cell moved (see Fig. 3(b), bottom). This issue is overcome by the NN, which robustly provided vacuole masks where the RI values were smaller than the other regions.
Fig. 3. Segmentation performance evaluation. (a, b) Representative cross-sections of RI (gray), Vph1-GFP (green), label from random forest (RF, cyan), and output from NN (magenta) for (a) the training dataset and (b) the untrained dataset, respectively. (c, d). Heat scatter maps and projected histograms of Pearson correlation coefficients (ρ) between 3D RI distributions and 3D cell masks for (c) the training dataset and (d) the untrained dataset, respectively. Axis label: (Horizontal, Vertical) = (ρoutput, ρlabel). (e, f). Similarity parameters between RF labels and NN outputs. Intersections over union (IOUs), Pearson correlation coefficients (PCCs), sensitivity, and specificity for (e) the training dataset and (f) the untrained dataset. The data are presented as mean ± standard deviation (SD). The magenta region indicates where the output mask from the NN has a higher ρ than the label from the RF, and the cyan vice versa. Scale bar = 2 µm.
To objectively evaluate the segmentation performance of both the RF and NN algorithms, we computed the Pearson correlation coefficients between the obtained binary mask X and the refractive index (RI) image (ρX). Given that we were correlating continuous RI values (ranging from 1.35 to 1.45) with binary values (either 0 or 1), we anticipated relatively low correlation coefficients. However, the emergence of a positive correlation serves as an indication that lower RI values are more likely to be identified as part of the vacuole. Heat-scatter plots from both the training and untrained datasets depicted a positive correlation between ρRF and ρNN (as illustrated in Figs. 3(c), d). This suggests that the neural network was effectively trained.
In the training set, the mean value of ρRF (mean ± SD = 38.7 ± 8.5%) was slightly larger than ρNN (mean ± SD = 37.5 ± 8.8%) (Fig. 3(c)). The number of datapoints with larger ρRF (magenta region in Fig. 3(c)) and larger ρNN (cyan region in Fig. 3(c)) were 117 and 94, respectively. This indicates that the RF segmentation outperforms the NN network in the trained dataset, which was expected because the NN was trained using the RF labels. In contrast, in the untrained dataset, the output masks from the NN output exhibited slightly larger correlation coefficients (ρNN = 38.7 ± 9.6%) than the RF labels with smaller standard deviations (ρRF = 35.2 ± 11.9%) (Fig. 3(d)). Correspondingly, the number of datapoints with larger ρRF (magenta region in Fig. 3(d)) and larger ρNN (cyan region in Fig. 3(d)) were 95 and 173, respectively.
In order to compare the similarity between the labels generated by the RF and the outputs of the NN, we calculated the intersection over union (IOUs), PCCs, sensitivity, and specificity (Figs. 3(e) and (f)). Notably, consistent with our previous observations, the similarity between the RF labels and the NN outputs appears to be lower in the untrained dataset. However, considering the NN output demonstrates higher correlations with the raw RI tomogram (as displayed in Fig. 3(d)), our results suggest an enhanced segmentation performance of the NN within the untrained dataset. Additionally, the steady trends of these parameters imply that the PCC serves as a sufficient measure for representing the segmentation performance within our study.
Considering the visualized images and the quantitative results, deep learning provided prediction performances for 3D vacuoles comparable to the RF algorithm without the need for FL images. Since deep learning is not affected by image registration artifacts caused by vacuole movement between the acquisition of the RI and FL images, it demonstrated slightly improved performance for untrained datasets. Collectively, the results suggest the viability of using deep learning for a more generalized analysis utilizing label-free RI images.
3.2 Quantitative analysis of cytoplasm and vacuoles
We utilized the established framework to perform quantitative biological analysis of cytoplasm and vacuoles in budding yeast cells under different conditions (Fig. 4). Budding yeast is a commonly used model organism for studying stress response effects, cell morphology and growth [43]. We investigated four conditions commonly used in yeast cell biology for which pre-existing knowledge on cytoplasmic properties is available. These conditions consist of exponentially growing cells (log phase, N = 96), nutrient depleted cells, (stationary phase, N = 141) and cells treated with the TORC1 inhibitor rapamycin in two conventional concentrations (0.1 µM (N = 82) and 1 µM (N = 92)).
Fig. 4. Biological application and quantitative analysis of budding yeast cells. (a) Representative cross-sections of RI (gray), FL (green; Vph1-GFP), and defined label (white) for four experimental groups: log phase, stationary phase, rapamycin 0.1 µM, and rapamycin 1 µM. (b) Schematic illustrating the overall results. Budding yeasts in log phase are used as controls. In the stationary phase, yeasts exhibit larger and denser cytoplasm. When the cells are treated with rapamycin, their volumes and dry masses increase while their concentration decreases. Scale bar = 2 µm.
Cells in stationary phase have been shown to be more refractile than log-phase cells [44]. Treatment with rapamycin was observed to increase cell volume and diffusion in the cytoplasm [9], which is an indicator for lower crowding and therefore lower dry-mass concentration. These known behaviors are reflected in our statistical results (Fig. 5). Consistent with a previous study [43], the mean cell volumes in the log and stationary phases were 21.7 and 26.1 fL, respectively. The volume of the cytoplasm showed a significant increase in the stationary phase (Fig. 5(a)). Treatment with 1 µM rapamycin further increased the cytoplasmic volume, which is consistent with previous studies by Chan and Marshall [45].
Fig. 5. Biological application and quantitative analysis of budding yeast cells. (a) Quantitative results of volumetric parameters, including total cell volume, cytoplasm volume, and vacuole volume. (b) Quantitative results of dry mass parameters, including total cell dry mass, cytoplasm dry mass, and vacuole dry mass. (c) Quantitative results of median dry-mass concentration in cytoplasm and vacuoles. Tukey's range test was performed, and the data are presented as mean ± standard deviation (SD). *: P < 0.05, **: P < 0.01, ***: P < 0.001, ****: P < 0.0001.
The vacuole volume exhibited a similar trend except for the stationary phase. During the stationary phase, the size and mass of the cytoplasm increase, while the vacuoles retain their original size. It is important to note here that the vacuoles can be fragmented in logarithmic phase while they are merged into one large vacuole in the other conditions used in this study (Fig. 4(b)). This is a well-known behavior [17,46] that we addressed by adding up the separate vacuoles per cell for our analysis. However, small errors in segmentation can accumulate when comparing multiple vacuoles to one. Therefore, it is preferable to make comparisons between conditions that have only one vacuole, such as the stationary phase and rapamycin-treated conditions. In these conditions, the vacuole volume exhibited a significant, dose-dependent increase when treated with rapamycin, consistent with previous observations [42].
While the volume statistics (Fig. 5(a)) exhibited a similar trend to the dry mass (Fig. 5(b)), the median dry-mass concentrations showed stark differences (Fig. 5(c)). In the cytoplasm, the dry-mass concentration was largest in the stationary phase and decreased significantly when treated with increasing rapamycin concentration. This is in line with previous observations of increased crowding in starved cells [47] and decreased crowding upon rapamycin treatment [9]. In the vacuole, the dry-mass concentration remained comparable across conditions, except for a decrease in the 1 µM rapamycin treatment. All the above results follow previously observed patterns and therefore indicate that our method is robust to and able to quantify changes in both the cytoplasm and vacuole in different conditions.
4. Discussion & conclusion
Our study presents a quantitative evaluation of intracellular components, achieved by combining RI tomography with a machine-learning-based algorithm. Unlike the task of segmenting an entire cell, the accurate segmentation of subcellular compartments poses a significant challenge within RI images, largely due to the lack of chemical specificity. This limitation was counterbalanced by correlative fluorescence imaging, enabling precise annotations of yeast vacuole. Even a minimal set of annotations was enough to kick-start the supervised volumetric segmentation of intracellular organization, facilitating high-throughput statistical analysis of budding yeasts.
Our approach bears similarities to previous studies employing machine learning for segmenting immunological synapses [48,49] or predicting fluorescence images from RI [20]. However, our study stands apart in its data preparation stage, where we utilized ilastik to significantly streamline the labor-intensive annotation procedure. This streamlining of dataset preparation is essential for generalizing machine-learning-based analysis tasks and enhancing accessibility.
Our hybrid segmentation strategy succeeded fundamentally because there is a significant RI difference between the cytoplasm and the vacuole (Fig. 6(a)). One may then question whether a simple segmentation using a RI-thresholding algorithm should also work. As in Fig. 5(c), however, this is not simple because the intracellular RI difference depends significantly upon the treatment and culturing conditions. We quantitatively validated this by comparing the volume obtained from a conventional RI-thresholding algorithm and our machine-learning algorithms (Figs. 6(b) and (c)). For paired Student t tests, we used the dataset where the vacuole masks obtained from both the RF classifier and the deep NN were successfully segmented. The results indicated that the simple thresholding algorithm segmented larger vacuole volumes than the other algorithms (Fig. 6(b)). The similarity with the labels obtained from the RF classifier was also compared by computing the Pearson correlation coefficients, which indicated significantly lower correlation when an RI-thresholding algorithm was used (Fig. 6(c)). Taken together, both RI and complex morphological features should be considered for accurate segmentation of objects in cell.
Fig. 6. Comparison with conventional thresholding algorithm. (a) Statistical analysis of the relative refractive index to the background medium for defined cytoplasm (gray) and vacuole (green). (b) Vacuole volumes obtained from thresholding (gray; threshold) by the average value of the cytoplasm and the vacuole (ΔRIthresh = 4.31 × 10−2) and the RF (cyan) and the NN (magenta). (c) Pearson correlation coefficients of either the thresholding volume (gray) and the output from the deep NN (magenta) with the label from the RF classifier. n = 192. Two-tailed paired Student t tests were performed. and the data are presented as mean ± standard deviation (SD). *****: P < 1−10. n.s.: nonsignificant p value.
The segmentation performance can be further improved simply by updating the algorithms used in both ilastik and our NN. It is important to note that the architecture used in our study is a basic model. We anticipate that future studies will enhance model performance by adopting recently developed algorithms such as diffusion models [50] or attention models [51,52]. Moreover, we anticipate an enhancement in our 3D FL data through a transition from our current widefield FL module to a confocal imaging modality. This shift is expected to refine the accuracy, resolution, and precision of segmentation, thereby enabling a better distinction between the vacuolar membrane and individual compartments within the vacuoles in future studies. However, it is essential to acknowledge the need for careful evaluation of potential photobleaching effects associated with this modification.
The biological results presented here corroborate interesting correlations between cellular metabolic activities and the density of intracellular compartments. Unlike previous studies that focused on size and density of the entire cell, our method allows for simultaneous and detailed quantification of cytoplasmic and vacuolar densities, offering new possibilities for applications in yeast cell biology. For example, cellular classifications of yeast can be achieved by combining our framework with machine-learning classifiers [53–57]. This will facilitate research on regulatory mechanisms underlying cytoplasmic density as well as its effects on other processes, such as phase separation and pathologies. Additionally, combining our method with a microfluidic chip would allow for quantitative time-lapse measurements of the cell’s response to fast changes, such as glucose starvation, osmotic shock or temperature changes, which have all been implicated in affecting cytoplasmic density [14,47,58]. Moreover, it would allow us to follow cells over longer time-scales and track the effect of aging on cytoplasmic density [5]. Our method is available online (see Data availability) and we hope that it will prove a useful tool for the analysis of RI tomograms of various living cells.
Funding
Young researchers’ exchange programme between Korea and Switzerland (KR_EG_042022_05); Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (310030_212660); National Research Foundation of Korea (2015R1A3A2066550, 2022M3H4A1A02074314).
Disclosures
ML and YKP have financial interests in Tomocube Inc., a company that commercializes HT and is one of the sponsors of the work. The other authors declare no conflicts of interest.
Data availability
The used dataset and codes are available in github.com/moosunglee/Vacuole_segmenter.
References
1. M. Schürmann, J. Scholze, P. Müller, J. Guck, and C. J. Chan, “Cell nuclei have lower refractive index and mass density than cytoplasm,” J. Biophotonics 9(10), 1068–1076 (2016). [CrossRef]
2. S. Abuhattum, K. Kim, T. M. Franzmann, A. Eßlinger, D. Midtvedt, R. Schlüßler, S. Möllmert, H.-S. Kuan, S. Alberti, and V. Zaburdaev, “Intracellular mass density increase is accompanying but not sufficient for stiffening and growth arrest of yeast cells,” Front. Phys. 6, 131 (2018). [CrossRef]
3. W. H. Grover, A. K. Bryan, M. Diez-Silva, S. Suresh, J. M. Higgins, and S. R. Manalis, “Measuring single-cell density,” Proc. Natl. Acad. Sci. 108(27), 10992–10996 (2011). [CrossRef]
4. B. R. Parry, I. V. Surovtsev, M. T. Cabeen, C. S. O’Hern, E. R. Dufresne, and C. Jacobs-Wagner, “The bacterial cytoplasm has glass-like properties and is fluidized by metabolic activity,” Cell 156(1-2), 183–194 (2014). [CrossRef]
5. G. E. Neurohr, R. L. Terry, J. Lengefeld, M. Bonney, G. P. Brittingham, F. Moretto, T. P. Miettinen, L. P. Vaites, L. M. Soares, and J. A. Paulo, “Excessive cell growth causes cytoplasm dilution and contributes to senescence,” Cell 176(5), 1083–1097.e18 (2019). [CrossRef]
6. V. C. Hecht, L. B. Sullivan, R. J. Kimmerling, D.-H. Kim, A. M. Hosios, M. A. Stockslager, M. M. Stevens, J. H. Kang, D. Wirtz, and M. G. Vander Heiden, “Biophysical changes reduce energetic demand in growth factor–deprived lymphocytes,” J. Cell Biol. 212(4), 439–447 (2016). [CrossRef]
7. K. L. Cooper, S. Oh, Y. Sung, R. R. Dasari, M. W. Kirschner, and C. J. Tabin, “Multiple phases of chondrocyte enlargement underlie differences in skeletal proportions,” Nature 495(7441), 375–378 (2013). [CrossRef]
8. S. Oh, C. Lee, W. Yang, A. Li, A. Mukherjee, M. Basan, C. Ran, W. Yin, C. J. Tabin, and D. Fu, “Protein and lipid mass concentration measurement in tissues by stimulated Raman scattering microscopy,” Proc. Natl. Acad. Sci. 119(17), e2117938119 (2022). [CrossRef]
9. M. Delarue, G. P. Brittingham, S. Pfeffer, I. Surovtsev, S. Pinglay, K. Kennedy, M. Schaffer, J. Gutierrez, D. Sang, and G. Poterewicz, “mTORC1 controls phase separation and the biophysical properties of the cytoplasm by tuning crowding,” Cell 174(2), 338–349.e20 (2018). [CrossRef]
10. Y. Shin and C. P. Brangwynne, “Liquid phase condensation in cell physiology and disease,” Science 357(6357), eaaf4382 (2017). [CrossRef]
11. T. Kim, J. Yoo, S. Do, D. S. Hwang, Y. Park, and Y. Shin, “RNA-mediated demixing transition of low-density condensates,” Nat. Commun. 14(1), 2425 (2023). [CrossRef]
12. R. Babazadeh, C. B. Adiels, M. Smedh, E. Petelenz-Kurdziel, M. Goksör, and S. Hohmann, “Osmostress-induced cell volume loss delays yeast Hog1 signaling by limiting diffusion processes and by Hog1-specific effects,” PLoS One 8(11), e80901 (2013). [CrossRef]
13. S. Krishnaswamy, M. H. Spitzer, M. Mingueneau, S. C. Bendall, O. Litvin, E. Stone, D. Pe’er, and G. P. Nolan, “Conditional density-based analysis of T cell signaling in single-cell data,” Science 346(6213), 1250689 (2014). [CrossRef]
14. B. D. Knapp, P. Odermatt, E. R. Rojas, W. Cheng, X. He, K. C. Huang, and F. Chang, “Decoupling of rates of protein synthesis from cell expansion leads to supergrowth,” Cell Systems 9(5), 434–445.e6 (2019). [CrossRef]
15. M. Mir, Z. Wang, Z. Shen, M. Bednarz, R. Bashir, I. Golding, S. G. Prasanth, and G. Popescu, “Optical measurement of cycle-dependent cell growth,” Proc. Natl. Acad. Sci. 108(32), 13124–13129 (2011). [CrossRef]
16. T. A. Zangle and M. A. Teitell, “Live-cell mass profiling: an emerging approach in quantitative biophysics,” Nat. Methods 11(12), 1221–1228 (2014). [CrossRef]
17. L. Wang, E. S. Seeley, W. Wickner, and A. J. Merz, “Vacuole fusion at a ring of vertex docking sites leaves membrane fragments within the organelle,” Cell 108(3), 357–369 (2002). [CrossRef]
18. Y. Park, C. Depeursinge, and G. Popescu, “Quantitative phase imaging in biomedicine,” Nat. Photonics 12(10), 578–589 (2018). [CrossRef]
19. R. Barer, K. Ross, and S. Tkaczyk, “Refractometry of living cells,” Nature 171(4356), 720–724 (1953). [CrossRef]
20. Y. Jo, H. Cho, W. S. Park, G. Kim, D. Ryu, Y. S. Kim, M. Lee, S. Park, M. J. Lee, and H. Joo, “Label-free multiplexed microtomography of endogenous subcellular dynamics using generalizable deep learning,” Nat. Cell Biol. 23(12), 1329–1337 (2021). [CrossRef]
21. K. Kim, W. S. Park, S. Na, S. Kim, T. Kim, W. Do Heo, and Y. Park, “Correlative three-dimensional fluorescence and refractive index tomography: bridging the gap between molecular specificity and quantitative bioimaging,” Biomed. Opt. Express 8(12), 5688–5697 (2017). [CrossRef]
22. E. Wolf, “Three-dimensional structure determination of semi-transparent objects from holographic data,” Opt. Commun. 1(4), 153–156 (1969). [CrossRef]
23. K. Kim, J. Yoon, S. Shin, S. Lee, S.-A. Yang, and Y. Park, “Optical diffraction tomography techniques for the study of cell pathophysiology,” J. Biomed. Photonics Eng. 2(2), 020201 (2016). [CrossRef]
24. M. Esposito, C. Fang, K. C. Cook, N. Park, Y. Wei, C. Spadazzi, D. Bracha, R. T. Gunaratna, G. Laevsky, and C. J. DeCoste, “TGF-β-induced DACT1 biomolecular condensates repress Wnt signalling to promote bone metastasis,” Nat. Cell Biol. 23(3), 257–267 (2021). [CrossRef]
25. S. H. Kang, Y. S. Shin, D.-H. Lee, I. S. Park, S. K. Kim, D. Ryu, Y. Park, S.-H. Byun, J.-H. Choi, and S. J. Hong, “Interactions of nanoparticles with macrophages and feasibility of drug delivery for asthma,” Int. J. Mol. Sci. 23(3), 1622 (2022). [CrossRef]
26. A. J. Lee, D. Yoon, S. Han, H. Hugonnet, W. Park, J.-K. Park, Y. Nam, and Y. Park, “Label-free monitoring of 3D cortical neuronal growth in vitro using optical diffraction tomography,” Biomed. Opt. Express 12(11), 6928–6939 (2021). [CrossRef]
27. J. Sivalingam, Y. SuE, Z. R. Lim, A. T. Lam, A. P. Lee, H. L. Lim, H. Y. Chen, H. K. Tan, T. Warrier, and J. W. Hang, “A scalable suspension platform for generating high-density cultures of universal red blood cells from human induced pluripotent stem cells,” Stem Cell Rep. 16(1), 182–197 (2021). [CrossRef]
28. Y. Kim, T.-K. Kim, Y. Shin, E. Tak, G.-W. Song, Y.-M. Oh, J. K. Kim, and C.-G. Pack, “Characterizing organelles in live stem cells using label-free optical diffraction tomography,” Molecules and cells 44(11), 851 (2021). [CrossRef]
29. D. Park, D. Lee, Y. Kim, Y. Park, Y.-J. Lee, J. E. Lee, M.-K. Yeo, M.-W. Kang, Y. Chong, and S. J. Han, “Cryobiopsy: a breakthrough strategy for clinical utilization of lung cancer organoids,” Cells 12(14), 1854 (2023). [CrossRef]
30. D. S. Biggs and M. Andrews, “Acceleration of iterative image restoration algorithms,” Appl. Opt. 36(8), 1766–1775 (1997). [CrossRef]
31. S. Shin, K. Kim, J. Yoon, and Y. Park, “Active illumination using a digital micromirror device for quantitative phase imaging,” Opt. Lett. 40(22), 5407–5410 (2015). [CrossRef]
32. K. Lee, K. Kim, G. Kim, S. Shin, and Y. Park, “Time-multiplexed structured illumination using a DMD for optical diffraction tomography,” Opt. Lett. 42(5), 999–1002 (2017). [CrossRef]
33. S. Shin, K. Kim, T. Kim, J. Yoon, K. Hong, J. Park, and Y. Park, “Optical diffraction tomography using a digital micromirror device for stable measurements of 4D refractive index tomography of cells,” in Quantitative Phase Imaging II (SPIE, 2016), pp. 156–163.
34. J. Lim, K. Lee, K. H. Jin, S. Shin, S. Lee, Y. Park, and J. C. Ye, “Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography,” Opt. Express 23(13), 16933–16948 (2015). [CrossRef]
35. C. Park, S. Shin, and Y. Park, “Generalized quantification of three-dimensional resolution in optical diffraction tomography using the projection of maximal spatial bandwidths,” J. Opt. Soc. Am. A 35(11), 1891–1898 (2018). [CrossRef]
36. S. Berg, D. Kutra, T. Kroeger, C. N. Straehle, B. X. Kausler, C. Haubold, M. Schiegg, J. Ales, T. Beier, and M. Rudy, “Ilastik: interactive machine learning for (bio) image analysis,” Nat. Methods 16(12), 1226–1232 (2019). [CrossRef]
37. F. Isensee, P. Kickingereder, W. Wick, M. Bendszus, and K. H. Maier-Hein, “Brain tumor segmentation and radiomics survival prediction: Contribution to the brats 2017 challenge,” in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: Third International Workshop, BrainLes 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Revised Selected Papers 3 (Springer, 2018), pp. 287–297.
38. Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3D U-Net: learning dense volumetric segmentation from sparse annotation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19 (Springer, 2016), pp. 424–432.
39. D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Instance normalization: The missing ingredient for fast stylization,” arXiv, arXiv:1607.08022 (2016). [CrossRef]
40. N. Abraham and N. M. Khan, “A novel focal tversky loss function with improved attention u-net for lesion segmentation,” in 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019) (IEEE, 2019), pp. 683–687.
41. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv, [PubMed] (2014).10.48550/arXiv.1412.6980
42. H. Zhao, P. H. Brown, and P. Schuck, “On the distribution of protein refractive index increments,” Biophys J 100(9), 2309–2317 (2011). [CrossRef]
43. D. Midtvedt, E. Olsén, F. Höök, and G. D. Jeffries, “Label-free spatio-temporal monitoring of cytosolic mass, osmolarity, and volume in living cells,” Nat. Commun. 10(1), 340 (2019). [CrossRef]
44. V. Bugeja, J. Piggott, and B. Carter, “Differentiation of Saccharomyces cerevisiae at slow growth rates in glucose-limited chemostat culture,” Microbiology 128(11), 2707–2714 (1982). [CrossRef]
45. Y.-H. M. Chan and W. F. Marshall, “Organelle size scaling of the budding yeast vacuole is tuned by membrane trafficking rates,” Biophys J 106(9), 1986–1996 (2014). [CrossRef]
46. S. P. Rayermann, G. E. Rayermann, C. E. Cornell, A. J. Merz, and S. L. Keller, “Hallmarks of reversible separation of living, unperturbed cell membranes into two liquid phases,” Biophys J 113(11), 2425–2432 (2017). [CrossRef]
47. R. P. Joyner, J. H. Tang, J. Helenius, E. Dultz, C. Brune, L. J. Holt, S. Huet, D. J. Müller, and K. Weis, “A glucose-starvation response regulates the diffusion of macromolecules,” eLife 5, e09376 (2016). [CrossRef]
48. M. Lee, Y.-H. Lee, J. Song, G. Kim, Y. Jo, H. Min, C. H. Kim, and Y. Park, “Deep-learning-based three-dimensional label-free tracking and analysis of immunological synapses of CAR-T cells,” eLife 9, e49023 (2020). [CrossRef]
49. J. Choi, H.-J. Kim, G. Sim, S. Lee, W. S. Park, J. H. Park, H.-Y. Kang, M. Lee, W. D. Heo, and J. Choo, “Label-free three-dimensional analyses of live cells with deep-learning-based segmentation exploiting refractive index distributions,” bioRxiv, 2021.2005. 2023.445351 (2021). [CrossRef]
50. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International Conference on Machine Learning (PMLR, 2015), pp. 2256–2265.
51. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems 30 (2017).
52. J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2019), pp. 3146–3154.
53. Y. Li, J. Di, K. Wang, S. Wang, and J. Zhao, “Classification of cell morphology with quantitative phase microscopy and machine learning,” Opt. Express 28(16), 23916–23927 (2020). [CrossRef]
54. M. D. Priscoli, P. Memmolo, G. Ciaparrone, V. Bianco, F. Merola, L. Miccio, F. Bardozzo, D. Pirone, M. Mugnano, and F. Cimmino, “Neuroblastoma cells classification through learning approaches by direct analysis of digital holograms,” IEEE J. Sel. Top. Quantum Electron. 27(5), 1–9 (2021). [CrossRef]
55. A. V. Belashov, A. A. Zhikhoreva, T. N. Belyaeva, A. V. Salova, E. S. Kornilova, I. V. Semenova, and O. S. Vasyutinskii, “Machine learning assisted classification of cell lines and cell states on quantitative phase images,” Cells 10(10), 2587 (2021). [CrossRef]
56. M. Jiang, M. Shao, X. Yang, L. He, T. Peng, T. Wang, Z. Ke, Z. Wang, S. Fang, and Y. Mao, “Automatic classification of red blood cell morphology based on quantitative phase imaging,” Int. J. Opt. 2022, 1–13 (2022). [CrossRef]
57. D. Ryu, J. Kim, D. Lim, H.-S. Min, I. Y. Yoo, D. Cho, and Y. Park, “Label-free white blood cell classification using refractive index tomography and deep learning,” BME Front. 2021, 9893804 (2021). [CrossRef]
58. L. B. Persson, V. S. Ambati, and O. Brandman, “Cellular control of viscosity counters changes in temperature and energy availability,” Cell 183(6), 1572–1585.e16 (2020). [CrossRef]
