Combination of Graph and Convolutional Networks for Brain Tumor Segmentation from Multi-Modal MR Images In Clinical Applications
,
Abstract
Purpose: Brain tumors are very important for the overall health of humans, which happen due to the uncontrolled increase and duplication of abnormal cells. Therefore, brain tumor segmentation is a very important step in medical diagnosis and can help in early tumor detection, treatment planning, and tumor progression follow-ups. To solve the problems related to manual segmentation such as time-cost, inaccuracy and subjectivity, automatic segmentation with deep learning methods is presented. This study aimed to develop an automatic brain tumor segmentation based on the combination of convolutional and graph neural networks to overcome the shortcomings of each network when they are used individually.
Materials and Methods: The main goal of this study is to propose a novel architecture for brain tumor segmentation from multi-modal MR images and comparison of the results with related SOTA studies. The novel architecture uses a simple Convolutional Neural Network (CNN) and Graph Neural Network (GNN) sequentially. In the first stage, the volumetric 3D image with a combination of all modalities is fed to the simple convolutional network. After retrieving the feature representation of the CNN, a graph model is created and fed to the GNN. The CNN will help to capture local information of patches and GNN will retrieve the global information available in the data which together can provide promising results.
Results: The proposed model used for the segmentation of the BraTS2021 dataset showed the average Dice score of 0.86 and the average Hausdorff of 17.94. The results showed that the combination of CNN and GNN can the performance of the task at hand. Also, the heatmaps extracted can show the importance of adding the GNN into the CNN.
Conclusion: New and creative advancements in artificial intelligence and its applications for medical image segmentation are very promising. We proposed a hybrid network of CNN and GNN to capture local and global information and combine them in a way such that we can recreate an acceptable segmented result which is justified with Dice score and Hausdorff metrics quantitatively. The proposed methodology performed better in comparison with the other related methods. Also, the activation heatmaps confirm the reliability of the approach qualitatively.
1- M Ravinder et al., "Enhanced brain tumor classification using graph convolutional neural network architecture." Scientific Reports, Vol. 13 (No. 1), p. 14938, (2023).
2- Steven Lawrence Fernandes, U John Tanik, V Rajinikanth, and K Arvind Karthik, "A reliable framework for accurate brain image examination and treatment planning based on early diagnosis support for clinicians." Neural Computing and Applications, Vol. 32pp. 15897-908, (2020).
3- VVS Sasank and S Venkateswarlu, "An automatic tumour growth prediction based segmentation using full resolution convolutional network for brain tumour." Biomedical Signal Processing and Control, Vol. 71p. 103090, (2022).
4- Kabirat Sulaiman Ayomide, Teh Noranis Mohd Aris, and Maslina Zolkepli, "Improving Brain Tumor Segmentation in MRI Images through Enhanced Convolutional Neural Networks." International Journal of Advanced Computer Science and Applications, Vol. 14 (No. 4), (2023).
5- Qihang Ma et al., "DGRUnit: Dual graph reasoning unit for brain tumor segmentation." Computers in biology and medicine, Vol. 149p. 106079, (2022).
6- Javaria Amin, Muhammad Sharif, Mudassar Raza, Tanzila Saba, and Muhammad Almas Anjum, "Brain tumor detection using statistical and machine learning method." Computer methods and programs in biomedicine, Vol. 177pp. 69-79, (2019).
7- Komal Sharma, Akwinder Kaur, and Shruti Gujral, "Brain tumor detection based on machine learning algorithms." International Journal of Computer Applications, Vol. 103 (No. 1), (2014).
8- Mohamed A Naser and M Jamal Deen, "Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images." Computers in biology and medicine, Vol. 121p. 103758, (2020).
9- S Somasundaram and R Gobinath, "Current trends on deep learning models for brain tumor segmentation and detection–a review." in 2019 International conference on machine learning, big data, cloud and parallel computing (COMITCon), (2019): IEEE, pp. 217-21.
10- Tirivangani Magadza and Serestina Viriri, "Deep learning for brain tumor segmentation: a survey of state-of-the-art." Journal of Imaging, Vol. 7 (No. 2), p. 19, (2021).
11- Arkapravo Chattopadhyay and Mausumi Maitra, "MRI-based brain tumor image detection using CNN based deep learning method." Neuroscience Informatics, p. 100060, (2022).
12- Sérgio Pereira, Adriano Pinto, Victor Alves, and Carlos A Silva, "Brain tumor segmentation using convolutional neural networks in MRI images." IEEE transactions on medical imaging, Vol. 35 (No. 5), pp. 1240-51, (2016).
13- Sajid Iqbal, M Usman Ghani, Tanzila Saba, and Amjad Rehman, "Brain tumor segmentation in multi‐spectral MRI using convolutional neural networks (CNN)." Microscopy research and technique, Vol. 81 (No. 4), pp. 419-27, (2018).
14- Aida Allahverdi, Siavash Akbarzadeh, Alireza Khorrami Moghaddam, and Armin Allahverdy, "Differentiating tumor and edema in brain magnetic resonance images using a convolutional neural network." Frontiers in Biomedical Technologies, Vol. 5 (No. 1-2), pp. 44-50, (2018).
15- Xuan Chen, Jun Hao Liew, Wei Xiong, Chee-Kong Chui, and Sim-Heng Ong, "Focus, segment and erase: an efficient network for multi-label brain tumor segmentation." in Proceedings of the european conference on computer vision (ECCV), (2018), pp. 654-69.
16- Haozhe Jia, Weidong Cai, Heng Huang, and Yong Xia, "H^ 2 2 NF-Net for Brain Tumor Segmentation Using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part II 6, (2021): Springer, pp. 58-68.
17- Zeyu Jiang, Changxing Ding, Minfeng Liu, and Dacheng Tao, "Two-stage cascaded u-net: 1st place solution to brats challenge 2019 segmentation task." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Revised Selected Papers, Part I 5, (2020): Springer, pp. 231-41.
18- Pavel Dvořák and Bjoern Menze, "Local structure prediction with convolutional neural networks for multimodal brain tumor segmentation." in Medical Computer Vision: Algorithms for Big Data: International Workshop, MCV 2015, Held in Conjunction with MICCAI 2015, Munich, Germany, October 9, 2015, Revised Selected Papers 18, (2016): Springer, pp. 59-71.
19- Konstantinos Kamnitsas et al., "Ensembles of multiple models and architectures for robust brain tumour segmentation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: Third International Workshop, BrainLes 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Revised Selected Papers 3, (2018): Springer, pp. 450-62.
20- Yixin Wang et al., "Modality-pairing learning for brain tumor segmentation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I 6, (2021): Springer, pp. 230-40.
21- Andriy Myronenko, "3D MRI brain tumor segmentation using autoencoder regularization." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4, (2019): Springer, pp. 311-20.
22- Lutao Dai, Tengfei Li, Hai Shu, Liming Zhong, Haipeng Shen, and Hongtu Zhu, "Automatic brain tumor segmentation with domain adaptation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4, (2019): Springer, pp. 380-92.
23- Yading Yuan, "Automatic brain tumor segmentation with scale attention network." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I 6, (2021): Springer, pp. 285-94.
24- Mohammad Havaei et al., "Brain tumor segmentation with deep neural networks." Medical image analysis, Vol. 35pp. 18-31, (2017).
25- Saddam Hussain, Syed Muhammad Anwar, and Muhammad Majid, "Brain tumor segmentation using cascaded deep convolutional neural network." in 2017 39th annual International Conference of the IEEE engineering in medicine and biology Society (EMBC), (2017): IEEE, pp. 1998-2001.
26- Jie Chang et al., "A mix-pooling CNN architecture with FCRF for brain tumor segmentation." Journal of Visual Communication and Image Representation, Vol. 58pp. 316-22, (2019).
27- Neha Sharma, Vibhor Jain, and Anju Mishra, "An analysis of convolutional neural networks for image classification." Procedia computer science, Vol. 132pp. 377-84, (2018).
28- Ashish Vaswani et al., "Attention is all you need." Advances in neural information processing systems, Vol. 30(2017).
29- Alexey Dosovitskiy et al., "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929, (2020).
30- Ze Liu et al., "Swin transformer: Hierarchical vision transformer using shifted windows." in Proceedings of the IEEE/CVF international conference on computer vision, (2021), pp. 10012-22.
31- Wenxuan Wang, Chen Chen, Meng Ding, Hong Yu, Sen Zha, and Jiangyun Li, "Transbts: Multimodal brain tumor segmentation using transformer." in Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, (2021): Springer, pp. 109-19.
32- Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger R Roth, and Daguang Xu, "Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 272-84.
33- Yun Jiang, Yuan Zhang, Xin Lin, Jinkun Dong, Tongtong Cheng, and Jing Liang, "SwinBTS: A method for 3D multimodal brain tumor segmentation using swin transformer." Brain Sciences, Vol. 12 (No. 6), p. 797, (2022).
34- Qiran Jia and Hai Shu, "Bitr-unet: a cnn-transformer combined network for mri brain tumor segmentation." in International MICCAI Brainlesion Workshop, (2021): Springer, pp. 3-14.
35- Yao Zhang et al., "mmformer: Multimodal medical transformer for incomplete multimodal learning of brain tumor segmentation." in Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part V, (2022): Springer, pp. 107-17.
36- Zhaohu Xing, Lequan Yu, Liang Wan, Tong Han, and Lei Zhu, "Nestedformer: Nested modality-aware transformer for brain tumor segmentation." in Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part V, (2022): Springer, pp. 140-50.
37- Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie, "A convnet for the 2020s." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2022), pp. 11976-86.
38- Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini, "The graph neural network model." IEEE transactions on neural networks, Vol. 20 (No. 1), pp. 61-80, (2008).
39- Thomas N Kipf and Max Welling, "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907, (2016).
40- Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio, "Graph attention networks." arXiv preprint arXiv:1710.10903, (2017).
41- Will Hamilton, Zhitao Ying, and Jure Leskovec, "Inductive representation learning on large graphs." Advances in neural information processing systems, Vol. 30(2017).
42- Islem Gammoudi, Raja Ghozi, and Mohamed Ali Mahjoub, "Hybrid Architecture for 3D Brain Tumor Image Segmentation Based on Graph Neural Network Pooling." in Advances in Computational Collective Intelligence: 14th International Conference, ICCCI 2022, Hammamet, Tunisia, September 28–30, 2022, Proceedings, (2022): Springer, pp. 337-51.
43- Camillo Saueressig, Adam Berkley, Elliot Kang, Reshma Munbodh, and Ritambhara Singh, "Exploring graph-based neural networks for automatic brain tumor segmentation." in From Data to Models and Back: 9th International Symposium, DataMod 2020, Virtual Event, October 20, 2020, Revised Selected Papers 9, (2021): Springer, pp. 18-37.
44- Dhrumil Patel, Dhruv Patel, Rudra Saxena, and Thangarajah Akilan, "Multi-class Brain Tumor Segmentation using Graph Attention Network." arXiv preprint arXiv:2302.05598, (2023).
45- Rouholla Bagheri, Jalal Haghighat Monfared, and Mohammad Reza Montazeriyoun, "Brain tumor segmentation using graph coloring approach in magnetic resonance images." Journal of Medical Signals and Sensors, Vol. 11 (No. 4), p. 285, (2021).
46- Ujjwal Baid et al., "The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification." arXiv preprint arXiv:2107.02314, (2021).
47- Bjoern H Menze et al., "The multimodal brain tumor image segmentation benchmark (BRATS)." IEEE transactions on medical imaging, Vol. 34 (No. 10), pp. 1993-2024, (2014).
48- Spyridon Bakas et al., "Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features." Scientific data, Vol. 4 (No. 1), pp. 1-13, (2017).
49- Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba, "Learning deep features for discriminative localization." in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), pp. 2921-29.
50- Gaurav Singh and Ashish Phophalia, "Multimodal Brain Tumor Segmentation Using Modified UNet Architecture." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 295-305.
51- Camillo Saueressig, Adam Berkley, Reshma Munbodh, and Ritambhara Singh, "A joint graph and image convolution network for automatic brain tumor segmentation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 356-65.
52- Linmin Pei and Yanling Liu, "Multimodal brain tumor segmentation using a 3D ResUNet in BraTS 2021." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 315-23.
53- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, "Deep residual learning for image recognition." in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), pp. 770-78.
54- Karen Simonyan and Andrew Zisserman, "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556, (2014).
55- Christian Szegedy et al., "Going deeper with convolutions." in Proceedings of the IEEE conference on computer vision and pattern recognition, (2015), pp. 1-9.
56- Zhaoyu Hu, Leyin Li, An Sui, Guoqing Wu, Yuanyuan Wang, and Jinhua Yu, "An efficient R-Transformer network with dual encoders for brain glioma segmentation in MR images." Biomedical Signal Processing and Control, Vol. 79p. 104034, (2023).
57- Ze Liu et al., "Swin transformer v2: Scaling up capacity and resolution." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (2022), pp. 12009-19.
2- Steven Lawrence Fernandes, U John Tanik, V Rajinikanth, and K Arvind Karthik, "A reliable framework for accurate brain image examination and treatment planning based on early diagnosis support for clinicians." Neural Computing and Applications, Vol. 32pp. 15897-908, (2020).
3- VVS Sasank and S Venkateswarlu, "An automatic tumour growth prediction based segmentation using full resolution convolutional network for brain tumour." Biomedical Signal Processing and Control, Vol. 71p. 103090, (2022).
4- Kabirat Sulaiman Ayomide, Teh Noranis Mohd Aris, and Maslina Zolkepli, "Improving Brain Tumor Segmentation in MRI Images through Enhanced Convolutional Neural Networks." International Journal of Advanced Computer Science and Applications, Vol. 14 (No. 4), (2023).
5- Qihang Ma et al., "DGRUnit: Dual graph reasoning unit for brain tumor segmentation." Computers in biology and medicine, Vol. 149p. 106079, (2022).
6- Javaria Amin, Muhammad Sharif, Mudassar Raza, Tanzila Saba, and Muhammad Almas Anjum, "Brain tumor detection using statistical and machine learning method." Computer methods and programs in biomedicine, Vol. 177pp. 69-79, (2019).
7- Komal Sharma, Akwinder Kaur, and Shruti Gujral, "Brain tumor detection based on machine learning algorithms." International Journal of Computer Applications, Vol. 103 (No. 1), (2014).
8- Mohamed A Naser and M Jamal Deen, "Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images." Computers in biology and medicine, Vol. 121p. 103758, (2020).
9- S Somasundaram and R Gobinath, "Current trends on deep learning models for brain tumor segmentation and detection–a review." in 2019 International conference on machine learning, big data, cloud and parallel computing (COMITCon), (2019): IEEE, pp. 217-21.
10- Tirivangani Magadza and Serestina Viriri, "Deep learning for brain tumor segmentation: a survey of state-of-the-art." Journal of Imaging, Vol. 7 (No. 2), p. 19, (2021).
11- Arkapravo Chattopadhyay and Mausumi Maitra, "MRI-based brain tumor image detection using CNN based deep learning method." Neuroscience Informatics, p. 100060, (2022).
12- Sérgio Pereira, Adriano Pinto, Victor Alves, and Carlos A Silva, "Brain tumor segmentation using convolutional neural networks in MRI images." IEEE transactions on medical imaging, Vol. 35 (No. 5), pp. 1240-51, (2016).
13- Sajid Iqbal, M Usman Ghani, Tanzila Saba, and Amjad Rehman, "Brain tumor segmentation in multi‐spectral MRI using convolutional neural networks (CNN)." Microscopy research and technique, Vol. 81 (No. 4), pp. 419-27, (2018).
14- Aida Allahverdi, Siavash Akbarzadeh, Alireza Khorrami Moghaddam, and Armin Allahverdy, "Differentiating tumor and edema in brain magnetic resonance images using a convolutional neural network." Frontiers in Biomedical Technologies, Vol. 5 (No. 1-2), pp. 44-50, (2018).
15- Xuan Chen, Jun Hao Liew, Wei Xiong, Chee-Kong Chui, and Sim-Heng Ong, "Focus, segment and erase: an efficient network for multi-label brain tumor segmentation." in Proceedings of the european conference on computer vision (ECCV), (2018), pp. 654-69.
16- Haozhe Jia, Weidong Cai, Heng Huang, and Yong Xia, "H^ 2 2 NF-Net for Brain Tumor Segmentation Using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part II 6, (2021): Springer, pp. 58-68.
17- Zeyu Jiang, Changxing Ding, Minfeng Liu, and Dacheng Tao, "Two-stage cascaded u-net: 1st place solution to brats challenge 2019 segmentation task." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Revised Selected Papers, Part I 5, (2020): Springer, pp. 231-41.
18- Pavel Dvořák and Bjoern Menze, "Local structure prediction with convolutional neural networks for multimodal brain tumor segmentation." in Medical Computer Vision: Algorithms for Big Data: International Workshop, MCV 2015, Held in Conjunction with MICCAI 2015, Munich, Germany, October 9, 2015, Revised Selected Papers 18, (2016): Springer, pp. 59-71.
19- Konstantinos Kamnitsas et al., "Ensembles of multiple models and architectures for robust brain tumour segmentation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: Third International Workshop, BrainLes 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Revised Selected Papers 3, (2018): Springer, pp. 450-62.
20- Yixin Wang et al., "Modality-pairing learning for brain tumor segmentation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I 6, (2021): Springer, pp. 230-40.
21- Andriy Myronenko, "3D MRI brain tumor segmentation using autoencoder regularization." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4, (2019): Springer, pp. 311-20.
22- Lutao Dai, Tengfei Li, Hai Shu, Liming Zhong, Haipeng Shen, and Hongtu Zhu, "Automatic brain tumor segmentation with domain adaptation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4, (2019): Springer, pp. 380-92.
23- Yading Yuan, "Automatic brain tumor segmentation with scale attention network." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I 6, (2021): Springer, pp. 285-94.
24- Mohammad Havaei et al., "Brain tumor segmentation with deep neural networks." Medical image analysis, Vol. 35pp. 18-31, (2017).
25- Saddam Hussain, Syed Muhammad Anwar, and Muhammad Majid, "Brain tumor segmentation using cascaded deep convolutional neural network." in 2017 39th annual International Conference of the IEEE engineering in medicine and biology Society (EMBC), (2017): IEEE, pp. 1998-2001.
26- Jie Chang et al., "A mix-pooling CNN architecture with FCRF for brain tumor segmentation." Journal of Visual Communication and Image Representation, Vol. 58pp. 316-22, (2019).
27- Neha Sharma, Vibhor Jain, and Anju Mishra, "An analysis of convolutional neural networks for image classification." Procedia computer science, Vol. 132pp. 377-84, (2018).
28- Ashish Vaswani et al., "Attention is all you need." Advances in neural information processing systems, Vol. 30(2017).
29- Alexey Dosovitskiy et al., "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929, (2020).
30- Ze Liu et al., "Swin transformer: Hierarchical vision transformer using shifted windows." in Proceedings of the IEEE/CVF international conference on computer vision, (2021), pp. 10012-22.
31- Wenxuan Wang, Chen Chen, Meng Ding, Hong Yu, Sen Zha, and Jiangyun Li, "Transbts: Multimodal brain tumor segmentation using transformer." in Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, (2021): Springer, pp. 109-19.
32- Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger R Roth, and Daguang Xu, "Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 272-84.
33- Yun Jiang, Yuan Zhang, Xin Lin, Jinkun Dong, Tongtong Cheng, and Jing Liang, "SwinBTS: A method for 3D multimodal brain tumor segmentation using swin transformer." Brain Sciences, Vol. 12 (No. 6), p. 797, (2022).
34- Qiran Jia and Hai Shu, "Bitr-unet: a cnn-transformer combined network for mri brain tumor segmentation." in International MICCAI Brainlesion Workshop, (2021): Springer, pp. 3-14.
35- Yao Zhang et al., "mmformer: Multimodal medical transformer for incomplete multimodal learning of brain tumor segmentation." in Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part V, (2022): Springer, pp. 107-17.
36- Zhaohu Xing, Lequan Yu, Liang Wan, Tong Han, and Lei Zhu, "Nestedformer: Nested modality-aware transformer for brain tumor segmentation." in Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part V, (2022): Springer, pp. 140-50.
37- Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie, "A convnet for the 2020s." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2022), pp. 11976-86.
38- Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini, "The graph neural network model." IEEE transactions on neural networks, Vol. 20 (No. 1), pp. 61-80, (2008).
39- Thomas N Kipf and Max Welling, "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907, (2016).
40- Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio, "Graph attention networks." arXiv preprint arXiv:1710.10903, (2017).
41- Will Hamilton, Zhitao Ying, and Jure Leskovec, "Inductive representation learning on large graphs." Advances in neural information processing systems, Vol. 30(2017).
42- Islem Gammoudi, Raja Ghozi, and Mohamed Ali Mahjoub, "Hybrid Architecture for 3D Brain Tumor Image Segmentation Based on Graph Neural Network Pooling." in Advances in Computational Collective Intelligence: 14th International Conference, ICCCI 2022, Hammamet, Tunisia, September 28–30, 2022, Proceedings, (2022): Springer, pp. 337-51.
43- Camillo Saueressig, Adam Berkley, Elliot Kang, Reshma Munbodh, and Ritambhara Singh, "Exploring graph-based neural networks for automatic brain tumor segmentation." in From Data to Models and Back: 9th International Symposium, DataMod 2020, Virtual Event, October 20, 2020, Revised Selected Papers 9, (2021): Springer, pp. 18-37.
44- Dhrumil Patel, Dhruv Patel, Rudra Saxena, and Thangarajah Akilan, "Multi-class Brain Tumor Segmentation using Graph Attention Network." arXiv preprint arXiv:2302.05598, (2023).
45- Rouholla Bagheri, Jalal Haghighat Monfared, and Mohammad Reza Montazeriyoun, "Brain tumor segmentation using graph coloring approach in magnetic resonance images." Journal of Medical Signals and Sensors, Vol. 11 (No. 4), p. 285, (2021).
46- Ujjwal Baid et al., "The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification." arXiv preprint arXiv:2107.02314, (2021).
47- Bjoern H Menze et al., "The multimodal brain tumor image segmentation benchmark (BRATS)." IEEE transactions on medical imaging, Vol. 34 (No. 10), pp. 1993-2024, (2014).
48- Spyridon Bakas et al., "Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features." Scientific data, Vol. 4 (No. 1), pp. 1-13, (2017).
49- Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba, "Learning deep features for discriminative localization." in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), pp. 2921-29.
50- Gaurav Singh and Ashish Phophalia, "Multimodal Brain Tumor Segmentation Using Modified UNet Architecture." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 295-305.
51- Camillo Saueressig, Adam Berkley, Reshma Munbodh, and Ritambhara Singh, "A joint graph and image convolution network for automatic brain tumor segmentation." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 356-65.
52- Linmin Pei and Yanling Liu, "Multimodal brain tumor segmentation using a 3D ResUNet in BraTS 2021." in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, (2022): Springer, pp. 315-23.
53- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, "Deep residual learning for image recognition." in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), pp. 770-78.
54- Karen Simonyan and Andrew Zisserman, "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556, (2014).
55- Christian Szegedy et al., "Going deeper with convolutions." in Proceedings of the IEEE conference on computer vision and pattern recognition, (2015), pp. 1-9.
56- Zhaoyu Hu, Leyin Li, An Sui, Guoqing Wu, Yuanyuan Wang, and Jinhua Yu, "An efficient R-Transformer network with dual encoders for brain glioma segmentation in MR images." Biomedical Signal Processing and Control, Vol. 79p. 104034, (2023).
57- Ze Liu et al., "Swin transformer v2: Scaling up capacity and resolution." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (2022), pp. 12009-19.
| Files | ||
| Issue | Vol 12 No 3 (2025) | |
| Section | Original Article(s) | |
| DOI | https://doi.org/10.18502/fbt.v12i3.19178 | |
| Keywords | ||
| Brain Tumor Segmentation Convolutional Neural Network Graph Neural Network Magnetic Resonance Imaging | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
How to Cite
1.
Vatanpour M, Haddadnia J, Salmani Bajestani S. Combination of Graph and Convolutional Networks for Brain Tumor Segmentation from Multi-Modal MR Images In Clinical Applications. Frontiers Biomed Technol. 2025;12(3):547-556.
