Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Volumetric segmentation of biological cells and subcellular structures for optical diffraction tomography images

Open Access Open Access

Abstract

Three-dimensional, quantitative imaging of biological cells and their internal structures performed by optical diffraction tomography (ODT) is an important part of biomedical research. However, conducting quantitative analysis of ODT images requires performing 3D segmentation with high accuracy, often unattainable with available segmentation methods. Therefore, in this work, we present a new semi-automatic method, called ODT-SAS, which combines several non-machine-learning techniques to segment cells and 2 types of their organelles: nucleoli and lipid structures (LS). ODT-SAS has been compared with Cellpose and slice-by-slice manual segmentation, respectively, in cell segmentation and organelles segmentation. The comparison shows superiority of ODT-SAS over Cellpose and reveals the potential of our technique in detecting cells, nucleoli and LS.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Imaging of biological cells and their organelles is nowadays an integral part of scientific research. By conducting analysis on the cellular and subcellular levels, there is an opportunity to improve the understanding of the physiology of biological cells and their internal structures, as well as changes in their morphology under various conditions [13]. Expanding knowledge in this area provides advances in a variety of biology sub-disciplines, for instance in wound healing and antibiotic treatment [46] or studies of diseases [711]. Therefore, the number of biomedical research in need of more precise and non-invasive measurements is increasing. Quantitative phase imaging (QPI) techniques are the most promising methods in taking the analysis of biological samples to a higher level due to their non-invasive manner and the lack of chemical treatment. These techniques enable visualization of biological samples in two- and three-dimensional space by using the information about phase delay of light passing through them, which is closely related to the refractive index (RI) [12]. Currently available 2D QPI methods, like digital holographic microscopy, although often used in biomedical research, do not allow for precise measurements of highly heterogeneous biological cells. Without detailed visualization of the entire volume of cells and their structures, the biological analysis carried out may be inaccurate. Overcoming these limitations, 3D QPI techniques have become widespread in modern science, among which optical diffraction tomography (ODT) is the most popular one. Due to the capabilities of ODT to obtain high accuracy quantitative information about the 3D RI distribution of analyzed samples, it has been used in many studies both on the subcellular and cellular levels [3,13]. As indicated in numerous works, ODT may be successfully applied to the imaging of biological cells, including erythrocytes, leukocytes and cancer cells [9,14,15] and also intracellular structures, such as nucleus, nucleolus and lipid droplets [4,1618]. However, to conduct quantitative analysis of ODT reconstructions, access to volumetric RI values of biological cells and their organelles is required. This is made possible by performing volumetric segmentation, but unfortunately, available methods like simple global thresholding, region growing, level-set method or watershed [19,20] have difficulties in processing ODT images due to a number of reasons. One of the difficulties is related to missing-cone artifacts that are present when projections within a limited angular range are captured. The other difficulty arises from the nature of biological cells $-$ they are heterogeneous structures with various shapes, within which there are organelles that may have unusual appearance depending on the cell line, for example nucleus in neutrophils [21]. Finally, processing ODT images is even more problematic if we take into account the low RI contrast between some compartments of a biological cell. In the case of fluorescence imaging, the low contrast of biological structures can be enhanced through staining, however, ODT is a label-free method, and thus the problem still persists. Therefore, more advanced segmentation methods, such as those combining several processing techniques [18,22,23] and those based on deep-learning and machine-learning, are required. The application of such methods to ODT images has been indicated in several works: for segmentation of vacuoles [24], cell nucleus [25], immune-cancer cell pairs and immunological synapse [26]. Nevertheless, these techniques were implemented to solve a particular problem and therefore do not enable segmentation of various types of cells and their organelles. One of the techniques that has been proposed for this purpose uses volume rendering [27]. The limitation of this approach, however, is its dependence on RI values and gradient magnitude. Furthermore, some of the popular segmentation approaches which are made available as toolkits are CellSegm [28] and Cellpose [29]. Both of them allow segmentation of a whole biological cell, but only Cellpose provides an opportunity to extract one of the subcellular structures $-$ nucleus. The utility of CellSegm in performing 3D segmentation of ODT images is limited due to the fact that it is designed for processing fluorescence images, and the solution is not generalized to other microscopic modalities. Cellpose is the answer to this problem, as this deep-learning-based tool has a larger number of microscope image types that can be segmented. Despite the fact that the possibility of 3D segmentation has been introduced in Cellpose, it is still challenging to obtain accurate results, especially when the analyzed biological cell has a complex shape, which changes substantially in ’z’ direction. Other methods have also been reported, but the algorithms have never been made available making their applicability in biomedical studies limited [30].

Therefore, this work presents a new semi-automatic method that uses conventional approaches not based on machine learning to process ODT images, called ODT-SAS (Semi-Automatic Segmentation). Unlike machine learning techniques, it does not require training dataset with high-quality ground truth to properly segment ODT images. Acquiring such training dataset is challenging, especially for a wide variety of biological cell types. ODT-SAS is made available for general use and enables 3D segmentation of the whole biological cells and 2 types of their organelles: nucleolus and lipid structures (stress granules and lipid droplets), that are being increasingly analyzed through ODT technique. The choice of organelles to be segmented is associated with two aspects: (1) their importance in biomedical research [3133] and (2) high enough RI contrast which makes these structures visible under ODT.

This paper is organized as follows: in Section 2 we present a description of our method for 3D segmentation of whole biological cell and its organelles. In Section 3 we show the performance of ODT-SAS with comparison to Cellpose (for cell segmentation) and manual segmentation (for organelle segmentation). Finally, in Section 4 we present the conclusions.

2. Proposed 3D segmentation algorithm

ODT-SAS is a semi-automatic technique for processing ODT images, which is made available in [34]. It enables extraction of biological cells and their organelles: nucleoli and lipid structures (LS) from the whole volume obtained. Our approach combines different processing techniques, so as to successfully match the segmentation method to 3D label-free quantitative imaging of biological cells. The scheme of ODT-SAS is depicted in Fig. 1. The method is divided into 2 parts, each having its own processing steps.

 figure: Fig. 1.

Fig. 1. Scheme of ODT-SAS algorithm. (A) Steps for 3D biological cell segmentation. (B) Steps for 3D organelles segmentation: nucleoli (left) and lipid structures (right).

Download Full Size | PDF

2.1 3D segmentation of whole cells

The description of the first part of ODT-SAS, namely 3D segmentation of whole cells, is presented in this Section. The proposed method consists of 4 stages: preprocessing, background and foreground markers detection, 3D watershed segmentation, and postprocessing (Fig. 1(A)). Importantly, a priori knowledge about the data analyzed is needed for manual setting of 3 volume thresholds and 2 RI thresholds: V1 $-$ maximum volume of artifacts smaller than analyzed biological cells; V2 $-$ minimum volume of artifacts larger than analyzed biological cells; V3 $-$ minimum volume of investigated biological cells; R1 $-$ maximum RI of low intensity noise; R2 $-$ RI of immersion medium.

2.1.1 Preprocessing

The images acquired by ODT often suffer from artifacts (e.g. noise), the presence of which is an obstacle for many available 3D segmentation methods in achieving accurate results. To address this issue, a 3-part preprocessing stage has been introduced (as shown in Fig. 2) and includes: thresholding, morphological filtration and artifacts removal. At first, manual global thresholding [19] with threshold R1 (see Section 2.1) is applied to ODT image in order to remove low intensity noise (Fig. 2(B)). Next, a morphological filtration is performed by using two consecutive operations $-$ opening and closing [35] (Fig. 2(C)). We used cubic-shaped structuring element (SE) whose size $[x\, y\, z]$ is chosen by the user: $[2\, 2\, 2]$ or $[3\, 3\, 3]$. The size and shape of the SE were chosen experimentally so that only the noise is addressed. The morphological filtration removes small elements and smooths out the shape of a biological cell, however still large-volume artifacts are not taken into account. Thus, in the final step, based on two thresholds, V1 and V2 (see Section 2.1), structures whose volume is smaller than V1 and those larger than V2 are removed (Fig. 2(D)). The resulting 3D image is further used to determine the background marker.

 figure: Fig. 2.

Fig. 2. Schematic 2D cross-section representation of the preprocessing stage in 3D segmentation of a whole biological cell. (A) Original image. (B-D) Images generated as a result of three preprocessing steps: (B) thresholding, (C) morphological filtration and (D) artifacts removal. The regions in the red boxes (right column) include noise and artifacts (caused by the limitation of the field of view by the galvanometric mirror) that the ODT suffers from. The contrast of these regions has been artificially enhanced to increase the visibility of the noise reduction.

Download Full Size | PDF

2.1.2 Background and foreground marker detection

Determination of markers for background and foreground objects (biological cells) is a crucial part of the ODT-SAS algorithm. To define the background marker, the preprocessed ODT image is binarized by using threshold R2 (see Section 2.1). This 3D binarized image is then morphologically dilated to create a margin around the biological cells. In the final step, the hole-filling operation is performed and the complement of the resulting 3D image is calculated to create the background marker.

The original 3D image masked by the background marker is used to compute the foreground markers as the regional maxima. The regional maxima in 3D space are defined as the group of connected, homogeneous voxels, the intensity values of which are higher than the intensity values in neighbourhood. Unfortunately, the calculation of the regional maxima in original ODT image results in excessive foreground marker detection due to the presence of small local maxima [36,37]. To eliminate spurious foreground markers, additional processing elements, namely 3D Gaussian filtering ($\sigma = 4$) and morphological opening by reconstruction (with automatically generated disk-shaped SE) [38,39], have been performed sequentially before the regional maxima identification, as shown in Fig. 3. The former operation enables the reduction of the spurious local maxima, and the latter operation creates the plateaus where the foreground markers are.

 figure: Fig. 3.

Fig. 3. Sequential steps of the spurious foreground markers reduction presented in 2D cross-sections of (A1) masked ODT image, (B1) masked ODT image after 3D Gaussian filtering and (C1) masked ODT image after 3D Gaussian filtering and morphological opening by reconstruction. (A2, B2, C2) Images with superimposed foreground markers (yellow area) obtained for A1, B1, C1 images, respectively.

Download Full Size | PDF

2.1.3 3D watershed segmentation and postprocessing

In ODT we used marker-controlled version of 3D watershed algorithm [40,41] to extract biological cells from original ODT images (see [3D Watershed segmentation] block in Fig. 1(A)). This algorithm requires markers in the form of regional minima, and therefore, ODT image must be modified. The modification consists of 3 steps: (1) the original ODT image is smoothed by 3D Gaussian filtering ($\sigma = 4$), (2) the complement of this image is calculated, (3) the regional maxima (calculated in Section 2.1.2) are converted to regional minima. The resulting image is the input to the 3D watershed method, which results in a 3D matrix containing biological cell labels. It should be noted that when there are more foreground markers than cells (intentionally or not), an excess number of 3D volumes will be labelled as cells, which results in cell fragmentation. To overcome this problem, the postprocessing in the form of a merging algorithm has been implemented [23]. The merging process is preceded by two steps: (1) the detected objects are divided into two groups: biological cells and fragments of cells based on threshold V3 (see Section 2.1), (2) all fragments are sorted in descending order based on their volumes. After that, the adjacent fragments are merged together: fragment is merged with another fragment with which it shares the largest surface. If volume of the merged fragment is below threshold V3, it is further analyzed as a fragment of a cell, otherwise it is treated as a biological cell. This procedure is repeated as long as there are neighbouring fragments to be merged. Any fragments remaining after this stage are then merged with that biological cell with which they share the largest volume, finally completing the merging algorithm. Its result is then used to create binary mask of a biological cell.

2.2 3D segmentation of internal structures

2.2.1 Nucleoli segmentation

The accurate detection of nucleoli boundaries can be hampered by a large amount of details or noise in the ODT reconstructions. In order to remove unwanted elements while preserving the shape and size of nucleoli, the segmentation begins with the conversion of the reconstruction (input 3D image) into its piecewise-constant version. To achieve this result, in the first step object projections are recalculated based on Fourier Diffraction Theorem [42]. Then, from these projections a tomographic reconstruction is recalculated, however this time with a strong total variation (TV) regularization [20]. This operation is performed with Astra Tomography Toolbox [43,44]. The TV-regularized reconstruction is then multiplied by the mask of the biological cell obtained in the previous stage (Section 2.1.3) and the result is used in the next 3 stages of the nucleoli segmentation: semi-automatic nucleoli markers detection, nucleoli ROI determination and 3D watershed segmentation (Fig. 4).

 figure: Fig. 4.

Fig. 4. Three stages of the 3D nucleoli segmentation: (A) semi-automatic nucleoli markers detection, (B) nucleoli ROI determination and (C) 3D Watershed segmentation.

Download Full Size | PDF

The first operation after the TV-regularization is determining the locations of nucleoli (nucleoli markers). In order to obtain the most precise results, automatic and manual detection are combined. The automatic detection is carried out in three steps: (1) global thresholding, (2) morphological reconstruction and (3) determination of the nucleoli markers (Fig. 4(A)). In the first step, average RI from all 2D lateral cross-sections are calculated. The highest of these values is selected as a threshold. All voxels whose value is below this threshold are set to the immersion medium RI value. After that, the morphological opening by reconstruction, the same operation as in the 3D cell segmentation method (Section 2.1.2), but with different SE radius: calculated on the basis of the ratio of nucleolus radius to cytoplasm radius in the model of eukaryotic cell [45]. The result of the morphological reconstruction is used to compute the regional maxima that are the nucleoli markers. After this step, some spurious markers can still be detected, and therefore, the user is given a possibility to manually select the correct ones.

Next, a region of interest (ROI) is selected from which the nucleoli are to be segmented (Fig. 4(B)). The initial ROI is formed by increasing the size of the nucleoli markers determined in the previous stage, based on the manually specified nucleolus diameter. Then, the initial ROI is presegmented using Rosin’s thresholding method [46,47], after which the dilation operation is performed to create a margin around the nucleoli. In the end, the resulting ROI is corrected by performing 3D watershed segmentation and the nucleoli mask is created (Fig. 4(C)).

2.2.2 Lipid structures segmentation

In our approach to LS segmentation (Fig. 1), first, the substructures identified and segmented in previous steps are removed, and the reconstruction volume is limited with the biological cell mask. In the volume thus obtained, the contrast enhancement is performed to accentuate the LS area by combining two morphological operations, top-hat and bottom-hat transformations [48]. Then, the enhanced 3D image is presegmented by automatic thresholding with the threshold determined based on Rosin’s algorithm. The presegmentation result specifies an initial ROI, which is then dilated to ensure that its boundaries exceed the actual boundaries of the LS and then segmented through the same operation as in the presegmentation. At the end, the LS mask is created.

2.3 Evaluation method

The effectiveness of ODT-SAS was evaluated on four 3D RI reconstructions of SH-SY5Y neuroblastoma cell. The data set was obtained with custom-built ODT system in off-axis Mach-Zehnder configuration operating with limited angular range of projections. 90 projections of the sample were generated for $632nm$ wavelength light and for the imaging system $\text {NA}=1.3$. Detailed information about the ODT system is available in [49]. The Gerchberg-Papoulis iterative algorithm, which minimizes missing-cone artifacts [50], was used to calculate the 3D RI reconstructions of the investigated samples. The SH-SY5Y cell line was chosen as they have a relatively complex shape containing fragments (e.g. pseudopodia), that are difficult to segment. Cell culture description is given in [18].

The assessment of ODT-SAS was made by comparing the results with Cellpose (in case of whole cell segmentation) and manually determined masks (in case of internal structures segmentation). Cellpose segmentation was carried out with cytoplasm $2.0$ model and two manually set parameters: cell diameter ($diam$) and minimum number of pixels per cell mask ($size$). The Volume Segmenter app available in MATLAB R2020b was used to perform manual segmentation [51]. It should be noted that the manual segmentation is a slice-by-slice segmentation, not a volumetric one.

3. Results and discussion

In this Section, the results for ODT-SAS are presented. In Section 3.1 the results of the 3D biological cell segmentation are described and in Section 3.2 the results of the 3D nucleoli and lipid structures segmentation are shown.

3.1 Whole cell segmentation

ODT-SAS has been compared with Cellpose in whole cell segmentation. The 3D segmentation of SH-SY5Y cells using Cellpose was tedious as it required many attempts to finetune inner parameters. Despite the efforts made, a satisfactory level of segmentation accuracy has not been achieved. This is confirmed by a comparison of biological cell masks obtained with Cellpose and with ODT-SAS depicted in Fig. 5.

 figure: Fig. 5.

Fig. 5. Comparison of Cellpose and ODT-SAS in the efficiency of 3D biological cell segmentation. The resulting cell masks obtained through both methods are presented in three-dimensional space (first row) and in two-dimensional space (second and third row) by superimposing the boundaries of masks (red contour) on selected XY and ZY cross-sections of ODT images: (a-b,e-f,i-j) boundaries of biological cell obtained through Cellpose and (c-d,g-h,k-l) through ODT-SAS.

Download Full Size | PDF

To show the precision of cell edges detection, the boundaries of resulting masks are superimposed on XY and ZY cross-sections of ODT images (Fig. 5). The parameters that was selected for Cellpose and ODT-SAS are presented in Tab 1. It should be noted, that in one case analyzed, Cellpose returned the biological cell which was fragmented into 17 parts (Fig. 5(e-f)). The over-segmentation problem did not occur in ODT-SAS due to the postprocessing stage (see Section 2.1.3) $-$ all fragments generated prior to the postprocessing step (9 fragments in this case) were successfully merged into one object (Fig. 5(g-h)). As presented in Fig. 5(c-d, g-h, k-l) ODT-SAS identifies actual edges of cell with high accuracy $-$ it provides the opportunity to properly segment thin fragments, which are considered to be difficult to detect. Cellpose, on the contrary, segments cells with relatively low precision, mainly preserving their middle part (Fig. 5(a-b,e-f,i-j)). Moreover, the results reveal different levels of noise robustness. In two cases, Cellpose did not handle the presence of noise, especially close to the cell area, leading to the detection of part of the background (Fig. 5(f,j)). ODT-SAS successfully segmented 3D images containing noise without including areas beyond the biological cells (Fig. 5(c-d, g-h, k-l)). To assess the segmentation results not only in 2D cross-sections but also in 3D space, we presented 3D biological cell masks (first row in Fig. 5). The 3D representation shows better performance of ODT-SAS in terms of preserving volume of thin biological cell fragments.

Tables Icon

Table 1. The Cellpose and ODT-SAS parameters selected to segment three reconstruction from Fig. 5: reconstruction A (a-d), B (e-h) and C (i-l).

In order to demonstrate the capability of ODT-SAS to separate high-confluence biological cells, the segmentation was also performed on a reconstruction containing three adjacent cells in the field of view (FOV). The generated masks ($V1=500 \mu m^{3}$, $V3=3000 \mu m^{3}$, $R1=1.34$, $SE = [3\, 3\, 3]$) were compared with those obtained through Cellpose ($diam=120px$, $size=200000px$) in two- and three-dimensional space. The comparison presented in Fig. 6 shows superiority of ODT-SAS over Cellpose. The proposed method correctly determined the number of biological cells (third row in Fig. 6), while Cellpose identified 2 distinct cells as one object, giving fewer masks than expected (second row in Fig. 6 $-$ mask marked in blue). In addition, similar to the 3D segmentation of a single biological cell (Fig. 5), ODT-SAS identifies cells with relatively high precision, whereas Cellpose leads to underestimation of the cell areas (it is visible in the 2D cross-sections and in the 3D volume $-$ second row in Fig. 6).

 figure: Fig. 6.

Fig. 6. The effectiveness of ODT-SAS in biological cell separation compared to Cellpose. The first three columns show selected XY cross-sections of ODT image, and the last column presents the 3D volume. The results obtained through Cellpose (second row) and through ODT-SAS (third row) are presented by superimposing generated masks on the original ODT image (first row).

Download Full Size | PDF

In terms of computational performance, ODT-SAS outperforms Cellpose. The runtime of ODT-SAS is roughly 4 times lower than the runtime of Cellpose (the algorithms was tested on a $3.20$ GHz AMD Ryzen $5$). For instance, it took approximately $10$ minutes to segment an $802x802x802$ ODT image with one biological cell in the FOV using ODT-SAS algorithm, while it took approximately $39$ minutes with Cellpose. The lower computational cost of the proposed method is also noticeable when processing ODT image with three connected cells in the FOV $-$ for ODT-SAS the runtime is about 7 minutes while for Cellpose it is about 33 minutes.

3.2 Internal organelles segmentation

The results of the 3D nucleoli segmentation in comparison with the results obtained through manual segmentation are shown in Fig. 7. For the purpose of evaluation, one nucleolus was extracted from each analyzed 3D reconstruction. The boundaries of the nucleoli masks obtained were superimposed on the selected 2D (XY) cross-sections. ODT-SAS does relatively well in detecting borders of the nucleoli areas. For the second reconstruction (Fig. 7(B)) ODT-SAS gives a nucleoli mask that is highly similar to the one manually determined. However, for the first (Fig. 7(A)) and third (Fig. 7(C)) reconstructions, it defines larger nucleoli masks, with the differences primarily observed on slices 391 and 318, respectively. To confirm results presented in 2D cross-sections, the volume of each nucleoli mask is computed and presented in Tab. 2. The calculated volumes also indicate lower precision for the first and third (A, C) reconstructions than for the second (B) one. It should be noted, however, that it is not entirely clear whether the manually segmented fragments belong to the actual nucleoli areas, and thus the accuracy of ODT-SAS may, in fact, be higher.

 figure: Fig. 7.

Fig. 7. The results of the 3D segmentation of nucleoli through ODT-SAS in comparison to manual segmentation presented in selected 2D cross-sections of three tomographic reconstructions: (A), (B), (C). The boundaries of the obtained nucleoli masks are marked in yellow.

Download Full Size | PDF

Tables Icon

Table 2. The volumes of the nucleoli masks from Fig. 7 obtained through manual segmentation and ODT-SAS.

The effectiveness of the 3D lipid structures (LS) segmentation is depicted in Fig. 8. The LS from a small fragment of a biological cell were analyzed (manual segmentation of these subcellular structures within the whole cell volume would be very time-consuming). The 3D LS masks are shown on 1 chosen XY cross-section for each reconstruction. It should be noted that the proposed method properly locates the LS areas without misidentifying other compartments of biological cells. Nevertheless, on such small structures, it is hard to visually notice any deviation from the results obtained through manual segmentation. Therefore, the volumes of the LS masks are calculated. The computed volumes shown in Tab. 3 reveal that the proposed method gives the 3D LS areas with a certain margin, nevertheless this does not mean that the performance of our approach is low. Such a result can be associated with the inaccuracy of the manual segmentation performed slice by slice.

 figure: Fig. 8.

Fig. 8. The effectiveness of ODT-SAS in 3D lipid structures (LS) segmentation in the case of three reconstructions. The results are presented in 2D cross-sections of original reconstruction (first and second column) with superimposed LS masks obtained through manual segmentation (third column) and those obtained through ODT-SAS (fourth column).

Download Full Size | PDF

Tables Icon

Table 3. The volumes of the lipid structures masks from Fig. 8 obtained through manual segmentation and ODT-SAS.

The 3D segmentation of internal structures through ODT-SAS takes more time than the 3D segmentation of biological cell. For the ODT data analyzed in this work, the average time of the organelle segmentation is around $15$ minutes. Thus, the overall runtime of ODT-SAS needed to extract both biological cells and their organelles is approximately $25$ minutes (see Section 3.1).

4. Conclusions

A new semi-automatic ODT-SAS method for processing ODT images of biological cells was presented. ODT-SAS is a volumetric segmentation method combining different processing techniques that are not based on machine learning. It allows to extract not only the whole volume of a biological cell, but also some of its organelles: nucleoli and LS, the quantitative analysis of which is crucial in biomedical research. Moreover, it is made available for general use.

The effectiveness of ODT-SAS in biological cell segmentation was evaluated in comparison to one of the available toolkits $-$ Cellpose. The results obtained demonstrate an advantage of ODT-SAS over Cellpose. Our approach identifies the areas of biological cells with high accuracy. Furthermore, unlike Cellpose, it performs well in the presence of noise and in biological cell separation. It should be noted, that satisfactory level of 3D cell segmentation can be achieved if the preprocessing parameters of ODT-SAS method will be properly selected. In case of 3D organelle segmentation, our approach has been compared with slice-by-slice manual segmentation. The comparison reveals that in some cases, ODT-SAS has lower precision than manual segmentation, which may be due to the inaccuracy of the manual segmentation. This means that ODT-SAS can be still successfully used in detecting the actual borders of nucleoli and LS areas.

Apart from allowing precise quantitative analysis of ODT results, ODT-SAS is a promising tool for generating ground truth data in machine learning approaches to 3D segmentation. It should however be noted, that there is still room for improving computational performance of the proposed solution along with reducing the dependence of segmentation results on manually set parameters.

Funding

European Commission H2020 Industrial Leadership (101016726).

Acknowledgments

We would like to thank Natalia Nowak from Nencki Institute of Experimental Biology, Polish Academy of Sciences, Warsaw for preparing cell cultures.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are available in [52]. The code used to produce the results is available on a GitHub repository in [34].

References

1. E. C. Jensen, “Overview of Live-Cell Imaging: Requirements and Methods Used,” The Anat. Rec. 296, 1–8 (2013). [CrossRef]  

2. K. Alm, H. Cirenajwis, L. Gisselsson, A. G. Wingren, B. Janicke, A. Mölder, S. Oredsson, J. Persson, K. Alm, H. Cirenajwis, L. Gisselsson, A. G. Wingren, B. Janicke, A. Mölder, S. Oredsson, and J. Persson, Digital Holography and Cell Studies (IntechOpen, 2011) Publication Title: Holography, Research and Technologies.

3. V. Balasubramani, A. Kus, H.-Y. Tu, C.-J. Cheng, M. Baczewska, W. Krauze, and M. Kujawinska, “Holographic tomography: techniques and biomedical applications,” Appl. Opt. 60(10), B65–B80 (2021). [CrossRef]  

4. A. J. Lee, H. Hugonnet, W. Park, and Y. Park, “Three-dimensional label-free imaging and quantification of migrating cells during wound healing,” Biomed. Opt. Express 11(12), 6812–6824 (2020). [CrossRef]  

5. Q. Chen, J. Yang, H. Yin, Y. Li, H. Qiu, Y. Gu, H. Yang, D. Xiaoxi, S. Xiafei, B. Che, and H. Li, “Optimization of photo-biomodulation therapy for wound healing of diabetic foot ulcers in vitro and in vivo,” Biomed. Opt. Express 13(4), 2450–2466 (2022). [CrossRef]  

6. J. Oh, J. S. Ryu, M. Lee, J. Jung, S. Han, H. J. Chung, and Y. Park, “Three-dimensional label-free observation of individual bacteria upon antibiotic treatment using optical diffraction tomography,” Biomed. Opt. Express 11(3), 1257–1267 (2020). [CrossRef]  

7. G. Simionato, K. Hinkelmann, R. Chachanidze, P. Bianchi, E. Fermo, R. v. Wijk, M. Leonetti, C. Wagner, L. Kaestner, and S. Quint, “Red blood cell phenotyping from 3D confocal images using artificial neural networks,” PLoS Comput Biol 17(5), e1008934 (2021). [CrossRef]  

8. A. Esposito, J.-B. Choimet, J. N. Skepper, J. M. Mauritz, V. L. Lew, C. F. Kaminski, and T. Tiffert, “Quantitative Imaging of Human Red Blood Cells Infected with Plasmodium falciparum,” Biophys. J. 99(3), 953–960 (2010). [CrossRef]  

9. K. Kim, J. Yoon, S. Shin, S. Lee, S.-A. Yang, and Y. Park, “Optical diffraction tomography techniques for the study of cell pathophysiology,” J. Biomed. Photonics & Eng. 2, 020201-1 (2016). [CrossRef]  

10. Y. Kim, H. Shim, K. Kim, H. Park, S. Jang, and Y. Park, “Profiling individual human red blood cells using common-path diffraction optical tomography,” Sci. Reports 4(1), 6659 (2014). [CrossRef]  

11. S.-e. Koo M.D, S.-e. Koo M.D, S. Jang M.D, C. J. Park M.D, Y.-U. Cho M.D, and Y. Park M.D, “Unique Red Blood Cell Morphology Detected in a Patient with Myelodysplastic Syndrome by Three-dimensional Refractive Index Tomography,” Lab. Med. Online 9(3), 185–188 (2019). [CrossRef]  

12. K. Lee, K. Kim, J. Jung, J. Heo, S. Cho, S. Lee, G. Chang, Y. Jo, H. Park, and Y. Park, “Quantitative Phase Imaging Techniques for the Study of Cell Pathophysiology: From Principles to Applications,” Sensors 13(4), 4170–4191 (2013). [CrossRef]  

13. D. Jin, R. Zhou, Z. Yaqoob, and P. T. C. So, “Tomographic phase microscopy: principles and applications in bioimaging [Invited],” J. Opt. Soc. Am. B, Opt. physics 34(5), B64–B77 (2017). [CrossRef]  

14. J. J. Y. Ong, J. Oh, X. Yong Ang, et al., “Optical diffraction tomography and image reconstruction to measure host cell alterations caused by divergent Plasmodium species,” Spectrochimica Acta Part A: Mol. Biomol. Spectrosc. 286, 122026 (2023). [CrossRef]  

15. S.-A. Yang, J. Yoon, K. Kim, and Y. Park, “Measurements of morphological and biophysical alterations in individual neuron cells associated with early neurotoxic effects in Parkinson’s disease,” Cytom. 91(5), 510–518 (2017). [CrossRef]  

16. M. Baczewska, K. Eder, S. Ketelhut, B. Kemper, and M. Kujawinska, “Refractive index changes of cells and cellular compartments upon paraformaldehyde fixation acquired by tomographic phase microscopy,” Cytom. 99, 388–398 (2021). [CrossRef]  

17. T.-K. Kim, B.-W. Lee, F. Fujii, J. K. Kim, and C.-G. Pack, “Physicochemical Properties of Nucleoli in Live Cells Analyzed by Label-Free Optical Diffraction Tomography,” Cells 8(7), 699 (2019). [CrossRef]  

18. M. Baczewska, M. Królikowska, M. Mazur, N. Nowak, J. Szymanski, W. Krauze, C.-J. Cheng, and M. Kujawinska, “Influence of Yokukansan on the refractive index of neuroblastoma cells,” Biomed. Opt. Express 14(5), 1959–1973 (2023). [CrossRef]  

19. O. Wirjadi, “Survey of 3D image segmentation methods,” ITWM Rep.123 (2007).

20. W. Krauze, “Optical diffraction tomography with finite object support for the minimization of missing cone artifacts,” Biomed. Opt. Express 11(4), 1919–1926 (2020). [CrossRef]  

21. M. Webster, K. L. Witkin, and O. Cohen-Fix, “Sizing up the nucleus: nuclear shape, size and nuclear-envelope assembly,” J. Cell Sci. 122(10), 1477–1486 (2009). [CrossRef]  

22. M. Baczewska, M. Mazur, and W. Krauze, “Towards true volumetric refractive index investigation in tomographic phase microscopy at the cellular level,” Opt. Continuum 2(2), 484–489 (2023). [CrossRef]  

23. P. S. Umesh Adiga and B. B. Chaudhuri, “An efficient method based on watershed and rule-based merging for segmentation of 3-D histo-pathological images,” Pattern Recognit. 34(7), 1449–1458 (2001). [CrossRef]  

24. M. Lee, M. Kunzi, G. Neurohr, S. S. Lee, and Y. Park, “Hybrid machine-learning framework for volumetric segmentation and quantification of vacuoles in individual unlabeled yeast cells using holotomography,” bioRxiv, bioRxiv.2023:2023-06 (2023). [CrossRef]  

25. J. Lee, H. Kim, H. Cho, Y. Jo, Y. Song, D. Ahn, K. Lee, Y. Park, and S.-J. Ye, “Deep-learning-based label-free segmentation of cell nuclei in time-lapse refractive index tomograms,” IEEE Access 7, 83449-83460 (2019). [CrossRef]  

26. M. Lee, Y.-H. Lee, J. Song, G. Kim, Y. Jo, H. Min, C. H. Kim, and Y. Park, “Deep-learning-based three-dimensional label-free tracking and analysis of immunological synapses of CAR-T cells,” eLife 9, e49023 (2020). [CrossRef]  

27. T. Kim and J. Park, “Analyzing 3D cell data of optical diffraction tomography through volume rendering,” in 2018 International Workshop on Advanced Image Technology (IWAIT), (2018), pp. 1–4.

28. E. Hodneland, T. Kögel, D. M. Frei, H.-H. Gerdes, and A. Lundervold, “CellSegm - a MATLAB toolbox for high-throughput 3D cell segmentation,” Source Code for Biol. Med. 8(1), 16 (2013). [CrossRef]  

29. C. Stringer, M. Michaelos, and M. Pachitariu, “Cellpose: a generalist algorithm for cellular segmentation,” Nat. Mathods 18, 100 (2020). [CrossRef]  

30. J. Choi, H.-J. Kim, G. Sim, S. Lee, W. S. Park, J. H. Park, H.-Y. Kang, M. Lee, W. D. Heo, J. Choo, H. Min, and Y. Park, “Label-free three-dimensional analyses of live cells with deep-learning-based segmentation exploiting refractive index distributions,” biorxiv, biorxiv:2021.05.23.445351 (2022). [CrossRef]  

31. M.-L. Dubois and F.-M. Boisvert, “The Nucleolus: Structure and Function,” The Funct. Nucl. 1, 29–49 (2016). [CrossRef]  

32. G. Onal, O. Kutlu, D. Gozuacik, and S. Dokmeci Emre, “Lipid Droplets in Health and Disease,” Lipids Health Dis. 16(1), 128 (2017). [CrossRef]  

33. F. Wang, J. Li, S. Fan, Z. Jin, and C. Huang, “Targeting stress granules: A novel therapeutic strategy for human diseases,” Pharmacol. Res. 161, 105143 (2020). [CrossRef]  

34. M. Mazur, “ODT-SAS,” Github, 2023, https://github.com/biopto/ODT-SAS.

35. M. A. B. Ramasamy, “Efficient 2-d structuring element for noise removal of grayscale images using morphological operations,” Int. J. Comput. Appl. 143(6), 24–28 (2016). [CrossRef]  

36. S. Eddins, “Local maxima, regional maxima, and the function imregionalmax,” MATLAB Central File Exchange, 2021, https://blogs.mathworks.com/steve/2021/08/19/local-maxima-regional-maxima-and-the-function-imregionalmax//.

37. S. Eddins, “Small-Peak Suppression with the H-Maxima Transform,” MATLAB Central File Exchange, 2021,https://blogs.mathworks.com/steve/2021/09/17/small-peak-suppression-with-the-h-maxima-transform/.

38. L. Vincent, “Morphological grayscale reconstruction in image analysis: applications and efficient algorithms,” IEEE Trans. on Image Process. 2(2), 176–201 (1993). [CrossRef]  

39. S. Eddins,“Morphological reconstruction: MATLAB & simulink,” MATLAB Central File Exchange, 2010, https://www.mathworks.com/help/images/understanding-morphological-reconstruction.html.

40. Z. Shi, L. He, K. Suzuki, T. Nakamura, and H. Itoh, “Survey on Neural Networks Used for Medical Image Processing,” International Journal of Computational Science 3, 86–100 (2009).

41. K. Parvati, B. S. Prakasa Rao, and M. Mariya Das, “Image Segmentation Using Gray-Scale Morphology and Marker-Controlled Watershed Transformation,” Discret. Dyn. Nat. Soc. 2008, 1–8 (2008). [CrossRef]  

42. O. Haeberlé, K. Belkebir, H. Giovaninni, and A. Sentenac, “Tomographic diffractive microscopy: basics, techniques and perspectives,” J. Mod. Opt. 57(9), 686–699 (2010). [CrossRef]  

43. W. Van Aarle, W. J. Palenstijn, J. De Beenhouwer, T. Altantzis, S. Bals, K. J. Batenburg, and J. Sijbers, “The ASTRA toolbox: A platform for advanced algorithm development in electron tomography,” Ultramicroscopy 157, 35–47 (2015). [CrossRef]  

44. W. Van Aarle, W. J. Palenstijn, J. Cant, E. Janssens, F. Bleichrodt, A. Dabravolski, J. De Beenhouwer, K. Joost Batenburg, and J. Sijbers, “Fast and flexible X-ray tomography using the ASTRA toolbox,” Opt. Express 24(22), 25129 (2016). [CrossRef]  

45. S. Heyden and M. Ortiz, “Investigation of the influence of viscoelasticity on oncotripsy,” Comput. Methods Appl. Mech. Eng. 314, 314–322 (2017). [CrossRef]  

46. P. Rosin, “Unimodal thresholding,” Pattern Recognit. 34(11), 2083–2096 (2001). [CrossRef]  

47. S. Khan, “Rosin thresholding,” MATLAB Central File Exchange, 2023, https://www.mathworks.com/matlabcentral/fileexchange/45443-rosin-thresholding.

48. I. M. O. Widyantara, I. M. D. P. Asana, N. Wirastuti, and I. B. P. Adnyana, “Image enhancement using morphological contrast enhancement for video based image analysis,” in 2016 International Conference on Data and Software Engineering (ICoDSE), (2016), pp. 1–6.

49. A. Kus, “Illumination-related errors in limited-angle optical diffraction tomography,” Appl. Opt. 56(33), 9247–9256 (2017). [CrossRef]  

50. A. Kus, W. Krauze, P. L. Makowski, and M. Kujawinska, “Holographic tomography: hardware and software solutions for 3D quantitative biomedical imaging (Invited paper),” ETRI Journal 41, 61 (2019). [CrossRef]  

51. “Segment 3-D grayscale or RGB volumetric images - MATLAB,” MATLAB Central File Exchange, https://www.mathworks.com/help/images/ref/volumesegmenter-app.html.

52. M. Mazur and W. Krauze, “Volumetric segmentation of biological cells and subcellular structures for optical diffraction tomography images - dataset,” Zenodo (2023). https://doi.org/10.5281/zenodo.8188948.

Data availability

Data underlying the results presented in this paper are available in [52]. The code used to produce the results is available on a GitHub repository in [34].

52. M. Mazur and W. Krauze, “Volumetric segmentation of biological cells and subcellular structures for optical diffraction tomography images - dataset,” Zenodo (2023). https://doi.org/10.5281/zenodo.8188948.

34. M. Mazur, “ODT-SAS,” Github, 2023, https://github.com/biopto/ODT-SAS.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1.
Fig. 1. Scheme of ODT-SAS algorithm. (A) Steps for 3D biological cell segmentation. (B) Steps for 3D organelles segmentation: nucleoli (left) and lipid structures (right).
Fig. 2.
Fig. 2. Schematic 2D cross-section representation of the preprocessing stage in 3D segmentation of a whole biological cell. (A) Original image. (B-D) Images generated as a result of three preprocessing steps: (B) thresholding, (C) morphological filtration and (D) artifacts removal. The regions in the red boxes (right column) include noise and artifacts (caused by the limitation of the field of view by the galvanometric mirror) that the ODT suffers from. The contrast of these regions has been artificially enhanced to increase the visibility of the noise reduction.
Fig. 3.
Fig. 3. Sequential steps of the spurious foreground markers reduction presented in 2D cross-sections of (A1) masked ODT image, (B1) masked ODT image after 3D Gaussian filtering and (C1) masked ODT image after 3D Gaussian filtering and morphological opening by reconstruction. (A2, B2, C2) Images with superimposed foreground markers (yellow area) obtained for A1, B1, C1 images, respectively.
Fig. 4.
Fig. 4. Three stages of the 3D nucleoli segmentation: (A) semi-automatic nucleoli markers detection, (B) nucleoli ROI determination and (C) 3D Watershed segmentation.
Fig. 5.
Fig. 5. Comparison of Cellpose and ODT-SAS in the efficiency of 3D biological cell segmentation. The resulting cell masks obtained through both methods are presented in three-dimensional space (first row) and in two-dimensional space (second and third row) by superimposing the boundaries of masks (red contour) on selected XY and ZY cross-sections of ODT images: (a-b,e-f,i-j) boundaries of biological cell obtained through Cellpose and (c-d,g-h,k-l) through ODT-SAS.
Fig. 6.
Fig. 6. The effectiveness of ODT-SAS in biological cell separation compared to Cellpose. The first three columns show selected XY cross-sections of ODT image, and the last column presents the 3D volume. The results obtained through Cellpose (second row) and through ODT-SAS (third row) are presented by superimposing generated masks on the original ODT image (first row).
Fig. 7.
Fig. 7. The results of the 3D segmentation of nucleoli through ODT-SAS in comparison to manual segmentation presented in selected 2D cross-sections of three tomographic reconstructions: (A), (B), (C). The boundaries of the obtained nucleoli masks are marked in yellow.
Fig. 8.
Fig. 8. The effectiveness of ODT-SAS in 3D lipid structures (LS) segmentation in the case of three reconstructions. The results are presented in 2D cross-sections of original reconstruction (first and second column) with superimposed LS masks obtained through manual segmentation (third column) and those obtained through ODT-SAS (fourth column).

Tables (3)

Tables Icon

Table 1. The Cellpose and ODT-SAS parameters selected to segment three reconstruction from Fig. 5: reconstruction A (a-d), B (e-h) and C (i-l).

Tables Icon

Table 2. The volumes of the nucleoli masks from Fig. 7 obtained through manual segmentation and ODT-SAS.

Tables Icon

Table 3. The volumes of the lipid structures masks from Fig. 8 obtained through manual segmentation and ODT-SAS.

Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.