Quantitative Spatial Analysis on Whole Slide Images Using U-Net

Sanghoon Lee; Yanjun Zhao; Mohamed Masoud; Saeid Belkasim

doi:doi:10.11648/j.cbb.20200802.18

| Peer-Reviewed

Quantitative Spatial Analysis on Whole Slide Images Using U-Net

Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim

Published in Computational Biology and Bioinformatics (Volume 8, Issue 2)

Received: 22 October 2020 Accepted: 18 November 2020 Published: 4 December 2020

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.

Published in	Computational Biology and Bioinformatics (Volume 8, Issue 2)
DOI	10.11648/j.cbb.20200802.18
Page(s)	90-96
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2020. Published by Science Publishing Group

Keywords

Cancer, Whole Slide Images, Spatial Analysis, U-Net, Machine Learning

References

[1]	Pantanowitz, L., Sinard, J. H., Henricks, W. H., Fatheree, L. A., Carter, A. B., Contis, L., Beckwith, B. A., Evans, A. J., Lal, A. and Parwani, A. V., 2013. Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Archives of Pathology and Laboratory Medicine, 137 (12), pp. 1710-1722.
[2]	Abels, E. and Pantanowitz, L., 2017. Current state of the regulatory trajectory for whole slide imaging devices in the USA. Journal of pathology informatics, 8.
[3]	Komura, D. and Ishikawa, S., 2018. Machine learning methods for histopathological image analysis. Computational and structural biotechnology journal, 16, pp. 34-42.
[4]	Lee, S., Amgad, M., Chittajallu, D. R., McCormick, M., Pollack, B. P., Elfandy, H., Hussein, H., Gutman, D. A. and Cooper, L. A., 2020. HistomicsML2. 0: Fast interactive machine learning for whole slide imaging data. arXiv preprint arXiv: 2001.11547.
[5]	Nalisnik, M., Amgad, M., Lee, S., Halani, S. H., Vega, J. E. V., Brat, D. J., Gutman, D. A. and Cooper, L. A., 2017. Interactive phenotyping of large-scale histology imaging data with HistomicsML. Scientific reports, 7 (1), pp. 1-12.
[6]	Lee, S., Amgad, M., Masoud, M., Subramanian, R., Gutman, D. and Cooper, L., 2019, November. An Ensemble-based Active Learning for Breast Cancer Classification. In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 2549-2553). IEEE.
[7]	Cruz-Roa, A., Gilmore, H., Basavanhally, A., Feldman, M., Ganesan, S., Shih, N. N., Tomaszewski, J., González, F. A. and Madabhushi, A., 2017. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent. Scientific reports, 7, p. 46450.
[8]	LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015 May 28; 521 (7553): 436-44. doi: 10.1038/nature14539. PMID: 26017442.
[9]	Litjens G, Sánchez CI, Timofeeva N, Hermsen M, Nagtegaal I, Kovacs I, Hulsbergen-van de Kaa C, Bult P, van Ginneken B, van der Laak J. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep. 2016 May 23; 6: 26286. doi: 10.1038/srep26286. PMID: 27212078; PMCID: PMC4876324.
[10]	Hou L, Samaras D, Kurc TM, Gao Y, Davis JE, Saltz JH. Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2016 Jun-Jul; 2016: 2424-2433. doi: 10.1109/CVPR.2016.266. PMID: 27795661; PMCID: PMC5085270.
[11]	Sun W, Zheng B, Qian W. Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Comput Biol Med. 2017 Oct 1; 89: 530-539. doi: 10.1016/j.compbiomed.2017.04.006. Epub 2017 Apr 13. PMID: 28473055.
[12]	Murthy V, Hou L, Samaras D, Kurc TM, Saltz JH. Center-Focusing Multi-task CNN with Injected Features for Classification of Glioma Nuclear Images. IEEE Winter Conf Appl Comput Vis. 2017 Mar; 2017: 834-841. doi: 10.1109/WACV.2017.98. Epub 2017 May 15. PMID: 29881826; PMCID: PMC5988234.
[13]	Xu Y, Jia Z, Wang LB, Ai Y, Zhang F, Lai M, Chang EI. Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinformatics. 2017 May 26; 18 (1): 281. doi: 10.1186/s12859-017-1685-x. PMID: 28549410; PMCID: PMC5446756.
[14]	Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Silva, V. W. K., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S. and Fuchs, T. J., 2019. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature medicine, 25 (8), pp. 1301-1309.
[15]	Kothari, S., Phan, J. H., Stokes, T. H. and Wang, M. D., 2013. Pathology imaging informatics for quantitative analysis of whole-slide images. Journal of the American Medical Informatics Association, 20 (6), pp. 1099-1108.
[16]	Topol, E J., 2019. High-performance medicine: the convergence of human and artificial intelligence. Nature medicine, 25 (1), pp. 44-56.
[17]	Vandenberghe, M. E., Scott, M. L., Scorer, P. W., Söderberg, M., Balcerzak, D. and Barker, C., 2017. Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer. Scientific reports, 7, p. 45938.
[18]	Joyce JA. Therapeutic targeting of the tumor microenvironment. Cancer cell. 7 (6): 513-20, 2005.
[19]	Lorusso G, Ruegg C. The tumor microenvironment and its contribution to tumor evolution toward metastasis. Histochemistry and cell biology. 130 (6): 1091-103, 2008.
[20]	Yu KH, Zhang C, Berry GJ, Altman RB, Ré C, Rubin DL, Snyder M. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun. 2016 Aug 16; 7: 12474. doi: 10.1038/ncomms12474. PMID: 27527408; PMCID: PMC4990706.
[21]	Ehteshami Bejnordi B, Balkenhol M, Litjens G, Holland R, Bult P, Karssemeijer N, van der Laak JA. Automated Detection of DCIS in Whole-Slide H&E Stained Breast Histopathology Images. IEEE Trans Med Imaging. 2016 Sep; 35 (9): 2141-2150. doi: 10.1109/TMI.2016.2550620. Epub 2016 Apr 5. PMID: 27076354.
[22]	Tomczak, K., Czerwińska, P., and Wiznerowicz, M.: ‘The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge’, Contemp Oncol (Pozn), 2015, 19, (1A), pp. A68-A77.
[23]	Amgad, M., Elfandy, H., Hussein, H., Atteya, L. A., Elsebaie, M. A., Abo Elnasr, L. S., Sakr, R. A., Salem, H. S., Ismail, A. F., Saad, A. M. and Ahmed, J., 2019. Structured crowdsourcing enables convolutional segmentation of histology images. Bioinformatics, 35 (18), pp. 3461-3467.
[24]	Long, J., Shelhamer, E. and Darrell, T., 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431-3440).
[25]	Ronneberger, O., Fischer, P. and Brox, T., 2015, October. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.
[26]	Jana M, Sar N. Modeling of hotspot detection using cluster outlier analysis and Getis-Ord Gi* statistic of educational development in upper-primary level, India. Modeling Earth Systems and Environment. 2 (2): 60, 2016.
[27]	Junior GB, Paiva AC, Silva AC, de Oliveira AC. Classification of breast tissues using Getis-Ord statistics and support vector machines. Intelligent Decision Technologies. 3 (4): 197-205, 2009.

Cite This Article

Plain Text BibTeX RIS

APA Style

Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim. (2020). Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Computational Biology and Bioinformatics, 8(2), 90-96. https://doi.org/10.11648/j.cbb.20200802.18

Copy | Download

ACS Style

Sanghoon Lee; Yanjun Zhao; Mohamed Masoud; Saeid Belkasim. Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Comput. Biol. Bioinform. 2020, 8(2), 90-96. doi: 10.11648/j.cbb.20200802.18

Copy | Download

AMA Style

Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim. Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Comput Biol Bioinform. 2020;8(2):90-96. doi: 10.11648/j.cbb.20200802.18

Copy | Download

@article{10.11648/j.cbb.20200802.18,
  author = {Sanghoon Lee and Yanjun Zhao and Mohamed Masoud and Saeid Belkasim},
  title = {Quantitative Spatial Analysis on Whole Slide Images Using U-Net},
  journal = {Computational Biology and Bioinformatics},
  volume = {8},
  number = {2},
  pages = {90-96},
  doi = {10.11648/j.cbb.20200802.18},
  url = {https://doi.org/10.11648/j.cbb.20200802.18},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.cbb.20200802.18},
  abstract = {Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.},
 year = {2020}
}

Copy | Download

TY  - JOUR
T1  - Quantitative Spatial Analysis on Whole Slide Images Using U-Net
AU  - Sanghoon Lee
AU  - Yanjun Zhao
AU  - Mohamed Masoud
AU  - Saeid Belkasim
Y1  - 2020/12/04
PY  - 2020
N1  - https://doi.org/10.11648/j.cbb.20200802.18
DO  - 10.11648/j.cbb.20200802.18
T2  - Computational Biology and Bioinformatics
JF  - Computational Biology and Bioinformatics
JO  - Computational Biology and Bioinformatics
SP  - 90
EP  - 96
PB  - Science Publishing Group
SN  - 2330-8281
UR  - https://doi.org/10.11648/j.cbb.20200802.18
AB  - Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.
VL  - 8
IS  - 2
ER  -

Copy | Download

Author Information

Sanghoon Lee

Department of Computer Sciences and Electrical Engineering, Marshall University, Huntington, United States of America
Yanjun Zhao

Department of Computer Science, Troy University, Troy, United States of America
Mohamed Masoud

Department of Neurology, Emory University, Atlanta, United States of America
Saeid Belkasim

Department of Computer Science, Georgia State University, Atlanta, United States of America

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim. (2020). Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Computational Biology and Bioinformatics, 8(2), 90-96. https://doi.org/10.11648/j.cbb.20200802.18

Copy | Download

ACS Style

Sanghoon Lee; Yanjun Zhao; Mohamed Masoud; Saeid Belkasim. Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Comput. Biol. Bioinform. 2020, 8(2), 90-96. doi: 10.11648/j.cbb.20200802.18

Copy | Download

AMA Style

Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim. Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Comput Biol Bioinform. 2020;8(2):90-96. doi: 10.11648/j.cbb.20200802.18

Copy | Download

@article{10.11648/j.cbb.20200802.18,
  author = {Sanghoon Lee and Yanjun Zhao and Mohamed Masoud and Saeid Belkasim},
  title = {Quantitative Spatial Analysis on Whole Slide Images Using U-Net},
  journal = {Computational Biology and Bioinformatics},
  volume = {8},
  number = {2},
  pages = {90-96},
  doi = {10.11648/j.cbb.20200802.18},
  url = {https://doi.org/10.11648/j.cbb.20200802.18},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.cbb.20200802.18},
  abstract = {Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.},
 year = {2020}
}

Copy | Download

TY  - JOUR
T1  - Quantitative Spatial Analysis on Whole Slide Images Using U-Net
AU  - Sanghoon Lee
AU  - Yanjun Zhao
AU  - Mohamed Masoud
AU  - Saeid Belkasim
Y1  - 2020/12/04
PY  - 2020
N1  - https://doi.org/10.11648/j.cbb.20200802.18
DO  - 10.11648/j.cbb.20200802.18
T2  - Computational Biology and Bioinformatics
JF  - Computational Biology and Bioinformatics
JO  - Computational Biology and Bioinformatics
SP  - 90
EP  - 96
PB  - Science Publishing Group
SN  - 2330-8281
UR  - https://doi.org/10.11648/j.cbb.20200802.18
AB  - Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.
VL  - 8
IS  - 2
ER  -

Copy | Download