Sub-micro scale cell segmentation using deep learning

Volkan Müjdat Tiryaki, Virginia M. Ayres, Ijaz Ahmed, David I. Shreiber

Research output: Contribution to journalArticlepeer-review


Automated cell segmentation is key for rapid and accurate investigation of cell responses. As instrumentation resolving power increases, clear delineation of newly revealed cellular features at the submicron through nanoscale becomes important. Reliance on the manual investigation of myriad small features retards investigation; however, use of deep learning methods has great potential to reveal cell features both at high accuracy and high speed, which may lead to new discoveries in the near term. In this study, semantic cell segmentation systems were investigated by implementing fully convolutional neural networks called U-nets for the segmentation of astrocytes cultured on poly-l-lysine-functionalized planar glass. The network hyperparameters were determined by changing the number of network layers, loss functions, and input image modalities. Atomic force microscopy (AFM) images were selected for investigation as these are inherently nanoscale and are also dimensional. AFM height, deflection, and friction images were used as inputs separately and together, and the segmentation performances were investigated on five-fold cross-validation data. Transfer learning methods, including VGG16, VGG19, and Xception, were used to improve cell segmentation performance. We find that AFM height images inherit more discriminative features than AFM deflection and AFM friction images for cell segmentation. When transfer-learning methods are applied, statistically significant segmentation performance improvements are observed. Segmentation performance was compared to classical image processing algorithms and other algorithms in use by considering both AFM and electron microscopy segmentation. An accuracy of 0.9849, Matthews correlation coefficient of 0.9218, and Dice's similarity coefficient of 0.9306 were obtained on the AFM test images. Performance evaluations show that the proposed system can be successfully used for AFM cell segmentation with high precision.

Original languageAmerican English
Pages (from-to)507-520
Number of pages14
JournalCytometry Part A
Issue number6
StatePublished - Jun 2022

ASJC Scopus subject areas

  • Pathology and Forensic Medicine
  • Histology
  • Cell Biology


  • astrocyte
  • atomic force microscopy
  • cell culture
  • cell morphology


Dive into the research topics of 'Sub-micro scale cell segmentation using deep learning'. Together they form a unique fingerprint.

Cite this