Machine and Deep Learning Methods Enable the Accurate and Efficient Segmentation, Grading, Diagnosis and Prognosis of Brain Tumors
Mohsen Tabatabaei1, Zeinab Khazaei2, Kamran Tavakol3*, Andrea Tavakol4
1PhD, Health Information Management, Office of Vice Chancellor for Research, Arak University of Medical Sciences, Arak, Iran
2PhD Student, School of Management and Economics, Science & Research Branch, Azad University. Tehran, Iran
3Professor Emeritus, College of Medicine, Howard University. Washington, DC, USA
4College of Information Studies, University of Maryland, College Park, MD, USA
*Corresponding Author: Dr. Kamran Tavakol, Professor Emeritus, College of Medicine, Howard University, Washington, DC, USA
Received: 15 March 2020; Accepted: 02 April 2020; Published: 09 April 2020
Citation: Mohsen Tabatabaei, Zeinab Khazaei, Kamran Tavakol, Andrea Tavakol. Machine and Deep Learning Methods Enable the Accurate and Efficient Segmentation, Grading, Diagnosis and Prognosis of Brain Tumors. Journal of Radiology and Clinical Imaging 3 (2020): 058-063.View / Download Pdf Share at Facebook
Early diagnosis and management of brain tumors are crucial and save patient from adverse effects or even death. Computer science advances have been made such that artificial intelligence (AI) brings enhancement to the classification, segmentation, grading and staging of brain tumors while assisting in the prediction of the diagnosis, treatment, future care and the prognosis, based on the analysis of MRI data. This review is aimed at enhancing the technical awareness of radiology residents, practitioners and physicians in general medicine, neurology and neurosurgery. Also, researchers from computer science, neuroscience and other relevant areas will find this review very helpful. We searched for relevant articles in Google Scholar, Scopus, and Pubmed, using these keywords: artificial intelligence, machine learning, deep learning, brain tumors, magnetic resonance imaging, MRI, and radiological image analyses. Out of 138 articles, 17 were selected, which described various AI algorithms to analyze brain tumor MRI data. This review presents the significant findings and conclusions of the selected articles, and the role of machine and deep learning methods in the classification, segmentation, diagnosis and prognosis of brain tumors. These AI methods enable radiologists to perform accurate, rapid and efficient mapping, classification, segmentation, grading and staging of brain tumors while predicting the diagnosis, treatment, future care and the clinical prognosis of patients with brain tumors.
Brain tumors; Artificial intelligence; Magnetic resonance imaging; Machine & deep learning methods; Grading and staging analyses
Brain tumors articles, Artificial intelligence articles, Magnetic resonance imaging articles, Machine & deep learning methods articles, Grading and staging analyses articles
AI - Artificial intelligence; FLAIR - Fluid-attenuated inversion recovery; BraTS - Brain tumor segmentation; iGTV - Infiltrative gross tumor volume; CNN - Convolutional neural network; MRTA - MRI texture analysis; CNS - Central nervous system; RescueNet - Cyclic unpaired encoder-decoder network for segmentation; CT - Computer tomography; ROC - Receiver operating characteristic; DCSNN - Deep convolutional symmetric neural network; rsFC - Resting-state functional connectivity; DSC - Dice similarity coefficient; VGG - Visual geometry group; DTI - Diffusion tensor imaging; iGTV - Infiltrative gross tumor volume
Brain tumors occur worldwide and impair the proper function of the central nervous system (CNS) by damaging the tissue and increasing the intracranial pressure, which may lead to death. Gliomas are the 10th most common form of tumors affecting children and adults alike . Fortunately, it is one of the most curable cancers if detected early . Therefore, the early diagnosis of brain tumors is crucial and can save patients from adverse effects or death . Magnetic resonance imaging (MRI) is used to assess tumors and evaluate the treatment success in the clinical practice. Therefore, accurately segmenting the brain tumors from MRI data is the key to clinical diagnosis and treatment planning. The processing of a large quantity of MRI data per patient is time consuming and cannot be achieved in a short time. However, the analysis of data from brain tumor MRI is a demanding research area and helpful to radiologists for rapid and valid analysis of brain tumors .
The application of artificial intelligence (AI) has led to novel discoveries across multiple disciplines including medicine [3, 4]. Much credible evidence have emerged in medical literature, suggesting that computer science can bring significant improvements to clinical medicine including radiology. Future collaboration among computer scientists and medical practitioners are critical to maximize the benefits of AI technology in the management of various diseases, especially neurological disorders and brain tumors. In recent years, AI advances have been made that enhance the diagnosis and management of brain tumors [1, 2].
Novel AI approaches, such as machine and deep learning methods, have been developed and continue to evolve for the segmentation, classification and grading of brain tumors. In radiology, these methods provide much insight into the diagnosis, treatment, follow-up and prognosis with the potential to bring significant efficiency to the clinical practice [1-5]. The following review of the most recent articles is an attempt to enhance the technical awareness of not only junior radiology residents and the practitioners, but also physicians in general medicine, neurology and neurosurgery. Also, researchers from computer science, neuroscience and other relevant areas will find this review very helpful.
For the purpose of this review, major search engines, such as Google Scholar, Scopus, Pubmed, and Medline, were searched, using the following keywords: artificial intelligence, machine learning, deep learning, brain tumors, magnetic resonance imaging, MRI, and radiological image analyses. Initially, 138 most recent peer-reviewed, original research and reviews on relevant topics were identified by the authors. After a careful review of the contents, 17 articles were selected [1-17], all of which describing various AI algorithms to analyze datasets from the MRI data of human brain tumors, in addition to the radiologists’ interpretations. This review briefly presents the significant findings and conclusions of the 17 articles, and the role of AI methods in the classification, segmentation, diagnosis and prognosis of brain tumors based on MRI data.
3. Literature Review
The radiology practice is undergoing a new development by taking advantage of some computer science components, such as artificial intelligence [1-5]. Machine and deep learning models are demanding techniques that assist radiologists in the efficient and valid analysis of brain tumor images. The profound task of classifying and segmenting brain tumors can be achieved through various artificial intelligence methods. Specifically, machine and deep learning techniques and algorithms, data mining and hybridization methods have enabled radiologists to obtain efficient segmentation and classification of brain tumors, compared to the interpretation of images from standard MRI [1-5].
The analysis of MRI and computer tomography (CT) data by a 5-year review of the studies conducted between 2014 and 2019 has suggested that most doctors prefer MRI over CT for staging human brain tumors . For this purpose, a combination of deep learning and meta-heuristic methods provide for the crucial task of segmenting and classifying brain tumors . These non-invasive techniques obviate the need for a surgical procedure to achieve similar clinical findings.
4. Machine Learning
4.1 Tumor classification
In 2015, an early machine learning study  used four algorithms for the automatic detection and classification of gliomas into benign or malignant group, based on MRI data. It found that a satisfactory area under the receiver operating characteristic (ROC) curves (>0.80) was achieved in three of the algorithms. The study further predicted better classification performance if the data were combined with those from other diagnostic modalities. At the time, this method held the promise of replacing the invasive methods of tumor grading through biopsy and histopathological analysis .
4.2 Predicting tumor grade
Later in 2016, a study  examined the diagnostic accuracy of grading glioma tumors by assessing the heterogeneity and texture analysis based on preoperative MRI data from 95 patients with low- or high-grade gliomas. The low and high grades were best discriminated with the sensitivity and specificity of 93% and 81%, respectively. This study concluded that assessing heterogeneity of gliomas based on MRI data improved the accuracy of grading and provided for a rapid treatment decision .
4.3 Predicting tumor stage
A study conducted in 2019  used convolutional neural network (CNN) algorithm, which serves as a machine learning or a deep learning method. It classifies MRI data from human brain tumors at various stages, compared with those obtained from such methods as Random Forest and K-Nearest Neighbors. The CNN was determined as the best tumor classifying method with an average accuracy of 98% and validation accuracy of 71% .
4.4 Prognostic advantages
Another study conducted in 2019, reports that a multivariate machine learning model, i.e., resting-state functional connectivity (rsFC) predicted the severity of aphasia in 126 patients with left-sided low- or high-grade gliomas, affecting the brain’s speech centers . The findings advance our knowledge regarding large-scale language network reorganization in patients with gliomas, and shed light on why the resection of the areas critical to language processing, may not necessarily induce aphasia .
5. Deep Learning
Deep learning technique facilitates accurate segmenting, classifying, grading, serially mapping tumor shape and texture, and predicting the survival of patients with brain tumors based on MRI and CT datasets [4, 5]. It further provides for better insight into brain tumors’ nature both qualitatively and quantitatively, and enables enhanced future care and clinical prognosis .
5.1 Radiation mapping
A study used diffusion tensor imaging (DTI) scans, a specific deep learning model, to estimate the infiltrative gross tumor volume (iGTV) in 33 patients with brain gliobastomas, compared to healthy standard GTV . The study concluded that free water corrected DTI scans can assist the radiologist to define the infiltrative tumor areas and correctly plan the radiation therapy dosing .
5.2 Clinical advantages and challenges
There have been considerable advances in the prediction of molecular biomarkers, and the role of both machine and deep learning methods in the identification of diffuse glioma features, based on patients’ MRI data . Also, there are few examples of molecular markers for other brain tumors . In 2019, a study compared the application of three CNN methods used for the segmentation of the MRI data for 44 patients with brain glioblastoma between two institutions . This study found that the performance of the model decreased considerably when it was applied to data from different institutions compared to those for a single institution. This was evident for the segmentation of the whole tumor and its components. The study concluded that there was a strong effect when data for training CNN models were chosen from multiple institutions, the elucidation of the reasons await further research .
5.3 Paired Vs unpaired data
A study published in 2019  developed deep learning algorithms, termed end-to-end network architecture or RescueNet, for the unpaired segmentation of brain tumors from MRI data. It eliminated the need for paired data to train the network and the results were tested on BraTS 2015 & 2017 datasets. This network provided huge advantages and improved the automated algorithms to assist radiologists in analyzing unpaired MRI data .
5.4 Limited data
Prior to the above study , another work  introduced an automated algorithm for brain tumor segmentation. It used a limited MRI data (T1 & FLAIR) per patient, by which the tumor texture and abnormality maps were constructed. In this research, Random Forests classification and dedicated Voxel clustering delineated the tumor compartments, achieving good results for the segmentation of the enhanced brain tissue and the tumor core, and the identification of total abnormal regions .
5.5 Influence of data symmetry
Following a non-symmetrical segmentation approach  in 2018, another method was developed in 2019, termed deep convolutional symmetrical neural network (DCSNN) . This study was more realistic, accurate and efficient than the former one  since it considered the symmetry as an important factor for the correct segmentation of glioma tumors. This method outperformed the complex mapping tasks and resolved the symmetry issue with the non-symmetrical methods . The symmetrical method was validated based on BraTS-2015 database and the results evaluated by dice similarity coefficient (DSC) metrics, with the average DSC being 0.852. This method takes only 10.8 seconds to segment a brain tumor dataset .
5.6 Automated segmentation
Another study conducted in 2019  provides a modified deep learning and end-to-end segmentation method, originally derived from CNN. It can automatically generate the segmentation maps slice by slice. The method, validated through BraTS 2015-2017, is currently a competitive, state-of-the-art segmentation approach .
5.7 Tumor progression and recurrence
A more recent study in 2019  has examined multiple MRI sequences from the histologically confirmed, high grade gliomas of 55 patients to better identify the tumor progression. The MRI sequence that achieved the highest accuracy was used to develop the deep learning model. The results support the studies that suggest that machine and deep learning approaches are clinically preferred to correctly detect the progression and recurrence of such tumors .
Even with proper acquisition of brain tumor images, the accurate and reliable segmentation of brain tumors is a complicated task. Future research on deep learning should be focused on discovering better ways of identifying and understanding the predictive features for the clinical decision making, diagnosis, treatment and prognosis of various brain tumors .
The exciting role of artificial intelligence in the advancement of radiology practice is rapidly evolving based on the analysis of MRI data. Numerous machine and deep learning methods enable radiologists to perform accurate, rapid and efficient mapping, classification, segmentation, grading and staging of brain tumors while predicting the diagnosis, treatment, future care and prognosis of such cases.
The authors are grateful to Dr. Manoochehr Kelarestaghi, Assistant Professor, Kharazmi University, Karaj, Iran for for his excellent review of the manuscript. We appreciate the Office of Vice Chancellor for Research, Arak University of Medical Sciences, Arak, Iran, and the School of Management and Economics, Azad University’s the Science and Research Branch, Tehran, Iran, for their support of this study. Funding was provided by the first and second authors.
Conflict of Interests
The authors had no conflict of interest with any entity during the conduction of this study.
- Ravishankar P, Smith DA, Avril S, et al. Uterine carcinosarcoma: a primer for radiologists. Abdominal Radiology 44 (2019): 2874-2885.
- Sagebiel TL, Bhosale PR, Patnana M, et al. Uterine Carcinosarcomas. Seminars in Ultrasound, CT and MRI. 40 (2019): 295-301.
- Denschlag D, Ulrich UA. Uterine carcinosarcomas - Diagnosis and management. Oncology Research and Treatment 41 (2018): 675-679.
- Zhao F, Xu Y, Zhang H, et al. Ultrasonographic findings of uterine carcinosarcoma. Gynecol Obstet Invest 84 (2019): 277-282.
- Hosh M, Antar S, Nazzal A, et al. Uterine Sarcoma: Analysis of 13,089 Cases Based on Surveillance, Epidemiology, and End Results Database. Int J Gynecol Cancer 26 (2016).
- Epstein E, Fischerova D, Valentin L, et al. Ultrasound characteristics of endometrial cancer as defined by International Endometrial Tumor Analysis (IETA) consensus nomenclature: prospective multicenter study. Ultrasound Obstet Gynecol 51 (2018): 818-828.
- Skogen K, Schulz A, Dormagen JB, Ganeshan B, Helseth E, et al. Diagnostic performance of texture analysis on MRI in grading cerebral gliomas. Eur J Radiol 85 (2016): 824-829.
- Yuan B, Zhang N, Yan J, et al. Resting-state functional connectivity predicts individual language impairment of patients with left hemispheric gliomas involving language network. NeuroImage Clin 24 (2019): 102023.
- Peeken JC, Molina-Romero M, Diehl C, Menze BH, Straube C, et al. Deep learning derived tumor infiltration maps for personalized target definition in Glioblastoma radiotherapy. Radiotherapy and Oncology 138 (2019): 166-172.
- Korfiatis P, Erickson B. Deep learning can see the unseeable: Predicting molecular markers from MRI of brain gliomas. Clin Radiol 74 (2019): 367-373.
- AlBadawy EA, Saha A, Mazurowski MA. Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing. Medical Physics 45 (2018): 1150-1158.
- Ghaffari M, Sowmya A, Oliver R. Automated brain tumor Segmentation using multimodal brain scans: A survey based on models submitted to the BraTS 2012-18 challenges. IEEE Reviews in Biomed Eng 13 (2020): 156-168.
- Bonte S, Goethals I, Van Holen R. Machine learning-based brain tumor segmentation on limited data using local texture and abnormality. Computers in biology and medicine 98 (2018): 39-47.
- Hussain S, Anwar SM, Majid M. Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing. 282 (2018): 248-261.
- Chen H, Qin Z, Ding Y, et al. Brain tumor segmentation with deep convolutional symmetric neural network. Neurocomputing (2019).
- Li H, Li A, Wang M. A novel end-to-end brain tumor segmentation method using improved fully convolutional networks. Computers in Biology and Medicine 108 (2019): 150-160.
- Bacchi S, Zerner T, Dongas J, et al. Deep learning in the detection of high-grade glioma recurrence using multiple MRI sequences: A pilot study (2019).
- Bacchi S, Zerner T, Dongas J, et al. Deep learning in the detection of high-grade glioma recurrence using multiple MRI sequences: A pilot study. J Clin Neurosci 70 (2019): 11-13.