Open Access
Add to BibSonomy
Abstract
Binarization is a key process in microscopy cell counting and cytometry analysis that is performed before segmentation to identify a cell within the background. We test the performances of 16 global and 9 local ImageJ thresholding algorithms on both experimental and synthetic confocal images of Escherichia coli and Staphylococcus aureus, evaluating the misclassification errors according to standard pattern recognition parameters. Some thresholding algorithms, such as Otsu, outperform other approaches, with respect to a pixel-by-pixel analysis. Overall, we found that the Bernsen local thresholding furnishes the best results also with respect to cell counting and morphology analysis.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Automated cell counting on fluorescence-stained bacteria images is a fast and accurate method for microbial quantification, that is rapidly replacing conventional approaches, such as colony counts or manual hemocytometer cell numbering [1]. The enumeration of colony forming units (CFU) is one of the most used methods to evaluate cell viability, since it is highly sensitive to small concentrations; however, it presents several drawbacks such as low accuracy and large time consume. One colony may arise from a single or a cluster of many cells, and it may take up to 48 h depending on the growth media and conditions. In addition, manual approaches are highly prone to intra and inter-user variability. Flow cytometry is maybe the most accurate method to determine the cell number, but the instrumentation is quite complex and expensive, and it requires training to operate and analyze data. On the other hand, automated cell counting can detect, identify and count single microbial cells in a very short time, and many approaches have been proposed in literature to evaluate the number of fluorescence-stained cells in a confocal image [2–8].
Most cell counting approaches rely on thresholding, that is a fundamental step to obtain binarized images, where pixels are classified into background and foreground objects, i.e. detected cells. In microbial image processing, the effectiveness of the thresholding is also related to the cell shape, which is a fundamental characteristic to determine the effectiveness of binarization.
The research on automated cell counting on microscopic images is still attracting an intense interest, and many effective approaches have been recently proposed, that can be implemented also using convolutional neural networks [9–11]. However, many of these approaches are quite difficult to implement, if the numerical code is not available.
Different software tools for image processing are available in open source programs, and ImageJ is one of the most popular among microbiologists [12]. Cell counting is often the last phase of a complex microbial analysis, and researchers look for a simple and direct approach that can be easily employed, and sometime optimized for different purposes. This is the reason why, instead of proposing a new binarization approach, we focus on the existing ImageJ binarization tools, that are freely available and very popular. We analyze the performances of the 16 global and 9 local ImageJ thresholding algorithms on confocal fluorescence microscopic images of Escherichia coli (E. coli) and Staphylococcus aureus (S. aureus) that have rod and round cell shapes, respectively, and are often employed as test examples [7]. We consider both cocci, that form clusters, and bacilli in chains, which are difficult to single out. The accuracy of each approach is quantitatively evaluated as a function of different assessment criteria. Since in experimental cytometric images the ground truth is unknown, all the thresholding algorithms are tested also on synthetic images.
We found that the global thresholding methods Intermode, IsoData, Moments and Otsu slightly outperforms the others. In addition, since confocal images are often non uniformly illuminated, some local thresholding algorithms are more efficient for cell counting. In particular, local Bernsen, Otsu and Phansalkar methods achieve the best performances, if the region of interest (ROI) where the threshold is evaluated, is suitably selected.
2. Materials and methods
2.1 Cells culture and fluorescence microscopy
E. coli MG1655 and S. aureus ATCC25923 were grown for 18 h in tryptic soy broth and diluted in saline solution to a final concentration measured as optical density (OD) at 600 nm OD600=1 (measured with the BioPhotometer basic Eppendorf spectrophotometer). The bacterial culture has been spotted and air-dried on a glass coverslip covered with agarose 0.5%. The cells were stained adding 1μg/ml 4’,6-diamidino-2-phenylindole (DAPI), that penetrates the cell membrane, binds the adenine–thymine regions of dsDNA and emits at 458 nm. The fluorescence images of Fig. 1 have been acquired using a laser scanning confocal microscope (Leica SP5), with a 63× oil immersion objective. A stack of ten 1024×1024 pixels images have been acquired and processed for each bacterial species.
Fig. 1. Starting from the left-hand side: experimental, simulated and ground-truth images for S.aureus (first row) and E. coli (second row).
2.2 Synthetic images
We use the SIMCEP simulating platform, implemented in MATLAB, to generate the synthetic images, because it presents a modular structure that allows us to extend the functionalities, according to different applications [13,14]. The standard version of this software creates only round cells, such those corresponding to S. aureus. Therefore, a new shape model was added for E. coli bacilli simulation, using a morphological dilation approach [15].
Sixtheen 950×950 pixel images have been numerically generated, changing the simulation parameters, as reported in Table 1. We have considered different number of cells (N. cells) composing the population, that are randomly uniformly distributed. In addition, it is possible that cells form a number of clusters (N. cluster), and we use different probability values to assign a cell to a cluster (Cluster probab.) In this case, the cell is randomly positioned around the cluster center, according to a normal distribution, whose variance (Cluster varian.) can be varied. The cells may overlap or not, according to a binary parameter (Overlap), the minimum cell radius is expressed in pixels (Cell size), and the shape (Cell shape) can be round or rod-like.
Table 1. Parameters of the Synthetic Images
The ground-truth images (called also ‘digital phantoms’, with binary pixel values) of Fig. 1 were successively distorted by Gaussian filtering (standard deviation 3 pixels), also adding Gaussian and Poisson noises, to take into account the point spread function (PSF) blurring, as well as the detector and photon noises. The influence of PSF blurring and additive Gaussian noise has been also separately evaluated on two additional synthetic images of 350 and 700 cells.
The image processing pipeline represented in Fig. 2 is applied to both synthetic and experimental images, and starts from an 8-bit grayscale conversion, since only the blue channel is analysed. All the images are pre-processed before applying the thresholding algorithms, starting with the contrast enhancement using histogram equalization, that is required especially in the darker background regions or within cells with low emission. We consider 0.3% of saturated pixels, that is the ImageJ standard value, or we manually vary the contrast using ImageJ Adjust > Brightness/Contrast command. In addition, the use of a Gaussian filter with variance 2 pixels highlights the cells and reduces the noise, that can be also reduced with the Process > Noise > Despeckle command. We observe that image pre-processing is a fundamental step required to reduce the noise, and that local thresholding approaches are much less effective if applied directly on a noisy image.
2.3 Thresholding evaluation
We have considered 16 global ImageJ thresholding algorithms: Huang, Huang2, Intermode, IsoData, Li, MaxEntropy, Mean, MinError, Minimum, Moments, Otsu, Percentile, RenyiEntropy, Shanbhag, Triangle, and Yen. They are based on different approaches, related to different image properties: as an example, Li and MaxEntropy select the threshold value according to the entropy (information content) of the image. On the other hand, IsoData refers to the image histogram, while Triangle is a geometric method. A detailed survey of the thresholding approaches is reported in Ref. 16.
Otsu global threshold algorithm was developed in 1979 and is one of the most used binarization method [17]. It refers to a bi-modal image histogram that separates foreground and background pixels: the optimal threshold value that separates the two classes is chosen by applying an intra-class variance minimization and inter-class variance maximization.
On the other hand, local algorithms set a different threshold over a sliding ROI, that is moved along the image. We have considered 9 ImageJ local thresholding approaches: Bernsen,
Contrast, Mean, Median, MidGrey, Niblack, Otsu, Phansalkar, Sauvola. The local threshold is often evaluated by statistical operations, such as mean or median.
Bernsen locally adaptive binarization method fixes the threshold as the mean value (midgray) of the minimum and maximum intensities, if the contrast is larger than a contrast threshold decided by the user [18]. In the case that the local contrast is below the contrast threshold, the ROI is considered to consist only of one class of pixels, and the pixel is identified as a cell or background, depending on the midgrey value, that is compared with the average value within the ROI. We show that the performances of almost all local binarization approaches are very sensitive to the ROI radius.
To obtain the ground-truth of the experimental images, we performed a user-biased assessment [19], involving 10 users who select the darkest pixel in every image that he/she considers still belonging to the foreground and, thus recognized as a cell. The mean value over a 3×3 area around the selected pixel is evaluated, to avoid over-sensitivity, and the threshold has been set as the mean value obtained among the 10 users, for each image. The binarized images using this approach constitute the “manual ground-truth” of the experimental images.
The global and local thresholding approaches are evaluated comparing pixel-by-pixel the ground-truth and the binarized images. Four detection results are possible, weather a cell and a background pixel are correctly detected (true positive – TP and true negative – TN, respectively). On the other hand, if a cell or a background pixel in the ground-truth are not correctly classified in the binarized image, they result in a false negative (FN) and false positive (FP), respectively. We evaluate the thresholding algorithms according to the following detection parameters, that are commonly used in pattern recognition approaches [19].
The Relative Quality parameter is maximum if each cell pixel is properly detected, without any misclassification for both cell pixels (FN = 0) and background pixels (FP = 0). The Sensitivity (or TP rate), is the efficiency in correctly detecting the cell pixels. On the other hand, the Specificity describes how likely the background pixels are detected, and 1-Specificity corresponds to the FP rate. The Sensitivity and 1-Specificity parameters are the axis of the receiver operating characteristic (ROC) space, which is commonly used to compare the performance of different classification algorithms. The optimal threshold algorithm corresponds to the point (0,1) (FP rate=0 and TP rate=1).
2.4 Cell counting and morphology evaluation
The binarized images obtained after thresholding are then processed by watershed segmentation, to separate touching cells, followed by an automated edge detection algorithm. In this way, we can count the number of cells and perform a morphology evaluation considering the parameters Cell Area, Perimeter and Circularity = 4πArea/Perimeter2.
3. Results
3.1 Global thresholding
The 16 global thresholding algorithms have been applied to both synthetic and confocal images, after pre-processing, and the mean and the standard deviation of the Relative Quality parameter are reported in Fig. 3 and 4, respectively. We observe that numerical and experimental results are consistent, and the best approaches are Intermodes, Moments, IsoData and Otsu, with a Quality Parameter larger than 74% over synthetic images and 83% on confocal images. On the other hand, some approaches, as Percentile and Shanbhag should be avoided as they Relative Quality parameter is always less than 30%. In particular, the Shanbhag approach calculates the total entropy of each gray-level pixel and find the threshold that maximizes it [20]. It is evident that this approach is not effective on cell images, that are also affected by random noise. We also observe that the Relative Quality parameter is lower when evaluated on synthetic images. This effect can be explained because, in this case, the ground-truth is the SIMCEP image before the Gaussian filtering that simulates the blurring due to the imaging PSF. On the other hand, in the evaluation of the performances of confocal images, we used a “manual ground-truth,” that can be not completely accurate.
Fig. 3. Mean and standard deviation of the Relative Quality parameter of global thresholding algorithms applied to synthetic images.
Fig. 4. Mean and standard deviation of the Relative Quality parameter of global thresholding algorithms applied to experimental confocal images.
The most efficient algorithms are Moments, Intermodes, IsoData and Otsu, that are represented in the ROC space of Fig. 5 and 6 by points close to the optimum point (0,1). Otsu is the thresholding method that presents the best Sensitivity (0.930) and Specificity (0.996) values on confocal images. In Fig. 5, the marks corresponding to Otsu and Intermodes, Huang and Mean almost overlap for synthetic images. In Fig. 6, the marks corresponding to Isodata and Otsu, Li and Huang almost overlap for confocal images.
3.2 Local thresholding
The ImageJ local thresholding algorithms have been applied to both synthetic and confocal images, and the corresponding Relative Quality parameters are shown in Fig. 7 and 8, respectively. The threshold is locally selected in a ROI with a user-defined radius. We used a ROI radii of 3 pixels, that corresponds to the variance of the Gaussian PSF, and 15 that is the standard value in ImageJ. We observe that some algorithms are very sensitive to the choice of the ROI radius, that is the fundamental parameter for local thresholding, and that Bernsen, Phansalkar, Otsu and Midgrey algorithms outperform the others, with a Relative Quality parameter larger than 71% for synthetic images. On the other hand, some algorithms, such as Median, are quite ineffective, because they erode or excessively enlarge the cell and the corresponding Relative Quality is less than 36%. The standard deviation of the Relative Quality parameter is almost the double, when local thresholding algorithms are applied to confocal images.
Fig. 7. Mean and standard deviation of the Relative Quality parameter of local thresholding algorithms applied to synthetic images.
Fig. 8. Mean and standard deviation of the Relative Quality parameter of local thresholding algorithms applied to experimental confocal images.
4. Cell counting and morphology analysis
We have selected global Otsu and the local Bernsen thresholding methods as among the best ImageJ approaches for image binarization. To give an extensive evaluation of their performances, we have considered two additional synthetic images with 350 and 750 cells, respectively. The ground-truth images have been Gaussian filtered to simulate a PSF with increasing standard deviation (blur size) and the thresholding results achieved using global Otsu and local Bernsen algorithms are shown in Fig. 9, where ROI radii of 3, 4, 6 and 15 pixels have been considered. The thresholding approaches are very sensitive to the blurring effect, and the Relative Quality parameter is larger for larger number of cells, because it increases with the TP parameter, if the total number of pixels in the image does not vary (see Eq. (1)). We also observe that local Bernsen thresholding outperforms the Otsu global approach, and that the ROI radius, that determines the area where the threshold is evaluated, should be selected smaller or equal to the PSF standard deviation.
Fig. 9. Relative Quality parameter of local Bernsen (with different ROI radii) and global Otsu thresholding algorithms applied to synthetic images of (a) 350 and (b) and 700 cells. Blurring effect using a Gaussian filtering with different values of standard deviation has been included.
In addition, we add Gaussian noise to the synthetic image with 350 cells, to obtain final images with signal to noise ratios (SNR) of 10, 5, 2 and 0 dB. Without pre-processing (Fig. 10 (a)), the performances of Otsu algorithm are better, with respect to Bernsen approach, because the threshold is selected over a larger area (whole image), and therefore, it is more robust to noise. This is the reason why the Relative Quality is higher for larger ROI radii, when the local Bernsen thresholding is applied. After pre-processing, the noise is effectively reduced, and Otsu approach is equally performing on images with different SNR (Fig. 10(b)). As already evidenced in Fig. 9, Bernsen approach is better performing for smaller ROI radius values, because in this case, the noise does not affect the choice of the threshold parameter.
Fig. 10. Relative Quality parameter of local Bernsen (with different ROI radii) and global Otsu thresholding algorithms applied to synthetic images of 350 cells. Additive Gaussian noise has been applied with different SNR values. (a) without pre-processing and (b) using pre-processing.
Finally, to analyze the effectiveness in bacterial cell counting, we have considered 8 synthetic images of Table 1 (Images 1, 2, 3, 12, 13, 14, 15 and 16), and the thresholding has been followed by watershed and edge detection algorithms. The cell counting and morphology parameters are evaluated using the ImageJ commands of Fig. 2, and the results are reported in Table 2, and it is evident that both methods provide a satisfactory evaluation of the cell number, even though Bernsen local algorithm outperforms Otsu approach.
Table 2. Number of Cells
A cell is recognized as the same cell in both binarized and ground-truth images, if the distance between the cell centers in the two images is smaller than 3 pixels. In this case, we compare the Cell Area, Perimeter and Circularity, whose average deviations are reported in Fig. 11. The morphology parameters obtained in the synthetic images processed with Bernsen local threshold algorithm show a stable trend, with acceptable average deviations from the corresponding ground-truth values. On the contrary, Otsu global method presents large deviations of the morphology metrics that reaches values up of 90% for the Cell Area parameter. We also observe that the images 12, 13 and 14, that contain rod-shaped bacteria, present the highest deviation in the Cell Circularity parameter. In fact, Otsu thresholding approach elongates and enlarges the cells, and this effect led to artefactual agglomerates.
Fig. 11. Average deviation of the Area, Perimeter and Circularity parameters of synthetic images, with respect to the digital phantoms.
5. Conclusions
We have evaluated the performances of 16 global and 9 local ImageJ thresholding algorithms on both confocal fluorescence microscopic and synthetic images, obtained with the SIMCEP simulator. We have considered E. coli and S. aureus bacteria that have different shape and clustering organization.
The performances of the binarization approaches have been quantitatively pixel-by-pixel evaluated, by comparing ground-truth and binarized images and evaluating the Relative Quality, Sensitivity and Specificity parameters, that measure the misclassification errors. We have found that the global Otsu and the local Bernsen algorithms provide the best results. In addition, local thresholding approaches are very sensitive to the ROI dimension, that should be selected as a function of the size of the PSF. Regarding cell counting and morphology analysis, Bernsen is the best performing algorithm, because it preserves the cell shape and borders. However, it is possible to further optimize Bernsen approach, modifying the process for threshold selection [21].
Disclosures
The Authors declare that there are no conflicts of interest.
References
2. J. Lojk, U. Čibej, D. Karlaš, L. Šajn, and M. Pavlin, “Comparison of two automatic cell-counting solutions for fluorescent microscopic images,” J. Microsc. 260(1), 107–116 (2015). [CrossRef]
3. A. A. Dima, J. T. Elliott, J. J. Filliben, M. Halter, A. Peskin, J. Bernal, M. Kociolek, M. C. Brady, H. C. Tang, and A. L. Plant, “Comparison of segmentation algorithms for fluorescence microscopy images of cells,” Cytometry, Part A 79A(7), 545–559 (2011). [CrossRef]
4. M. G. Forero, K. Kato, and A. Hidalgo, “Automatic cell counting in vivo in the larval nervous system of Drosophila,” J. Microsc. 246(2), 202–212 (2012). [CrossRef]
5. D. Hodneland, N. V. Burkoreshtliev, T. Eichler, X. Tai, S. Furke, A. Lundervold, and H. Gerdes, “A unified framework for automated 3-D segmentation of surface-stained living cells and a comprehensive segmentation evaluation,” IEEE Trans. Med. Imag. 28(5), 720–738 (2009). [CrossRef]
6. E. Gelasca, B. Obara, K. D. Fedorov, B. Kvilekval, and Manjunath, “Biosegmentation benchmark for evaluation of bioimage analysis methods,” BMC Bioinformatics 10(1), 368 (2009). [CrossRef]
7. M. Klinger-Strobel, H. Suesse, D. Fischer, M. W. Pletz, and O. Makarewicz, “A novel computerized cell count algorithm for biofilm analysis,” PLoS One 11(5), e0154937 (2016). [CrossRef]
8. J. Brocher, “Qualitative and quantitative evaluation of two new histogram limiting binarization algorithms,” Int. J. Image Proc.8(2), 30–48 (2014).
9. B. Zieliński, A. Plichta, K. Misztal, P. Spurek, M. Brzychczy-Wøoch, and D. Ochońska, “Deep learning approach to bacterial colony classification,” PLoS One 12(9), e0184554 (2017). [CrossRef]
10. W. Xie, J. A. Noble, and A. Zisserman, “Microscopy cell counting and detection with fully convolutional regression networks,” Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 6(3), 283–292 (2018). [CrossRef]
11. D. Riccio, N. Brancati, M. Frucci, and D. Gragnaniello, “A new unsupervised approach for segmenting and counting cells in high-throughput microscopy image sets,” IEEE J. Biomed. Health Inform. 23(1), 437–448 (2019). [CrossRef]
12. https://imagej.nih.gov/ij/
13. A. Lehmussola, P. Ruusuvuori, J. Selinummi, and H. Huttunen, “Computational framework for simulating fluorescence microscope images with cell populations,” IEEE Trans. Med. Imaging 26(7), 1010–1016 (2007). [CrossRef]
14. http://www.cs.tut.fi/sgn/csb/simcep/tool.html.
15. R. Gonzalez and R. Woods, “Image enhancement in the spatial domain,” in Digital Image Processing3rd ed., (Prentice Hall, 2008), chap. 4.
16. M. Sezgin and B. Sankur, “Survey over image thresholding techniques and quantitative performance evaluation,” J. Electron. Imaging 13(1), 146–168 (2004). [CrossRef]
17. N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Sys., Man., Cyber. 9(1), 62–66 (1979). [CrossRef]
18. J. Bernsen, “Dynamic thresholding of gray-level images,” Proc. 8th Int. Conf. on Pattern Recognition, Paris, 1251–1255 (1986).
19. J. Brocher, “Qualitative and quantitative evaluation of two new histogram limiting binarization algorithms,” Int. J. Image Process. 8(2), 30–48 (2014).
20. A. G. Shanbhag, “Utilization of information measure as a means of image thresholding,” Graph. Models Image Process 56(5), 414–419 (1994). [CrossRef]
21. L. Nichele, V. Persichetti, M. Lucidi, and G. Cincotti, “Thresholding algorithms for microbial cell counting,” Proc. International Conference on Optical Transparent Networks (ICTON) (2019).
