Convolutional neural network model for segmentation of magnetic resonance images with brain tumors

Accurate and fast segmentation of magnetic resonance (MR) images with volumetric brain formations, such as glioblastomas and meningiomas, helps plan surgical and radiation treatment, increase the safety and radicality of surgical intervention, which in turn allows to increase the overall life expectancy of patients and the disease-free period. It is required for the development and evaluation of the effectiveness of a deep convolutional neural network with transmission connections for automatic segmentation of volumetric brain formations (meningiomas and glioblastomas) on postoperative MR images, as well as analysis of its accuracy in comparison with manual expert segmentation and existing methods.

Material and methods. The paper considers the creation of an architecture inspired by the SegResNet model, training on BraTS2024-GLI and BraTS2024-MEN-RT, describes a method for compiling a training sample that reduces class imbalance, and analyzes the results in comparison with participants in the BraTS 2024 competition.

Results and discussion. The developed model was trained and tested on two datasets of postoperative images of glioblastoma and meningioma. Several metrics are analyzed to compare the model with the methods described in the literature, as well as to evaluate it in the context of the variability of manual segmentation by different experts. The model achieves a Sorensen coefficient 0.8299 for meningioma segmentation and 0.7028 for glioblastoma contrast-accumulating region segmentation. In addition, the segmentation of the model provides an accurate estimate of the volume of the tumor area, as evidenced by the high values of the coefficient of intra-class correlation – 0.9661 for meningioma and 0.8339 for glioblastoma. Overall, the developed model requires fewer resources for learning and getting results.

Conclusions. Model performs segmentation at least at the expert level, but with significantly less variability, especially when assessing the volume of the tumor after surgical treatment.

1. Ostrom Q.T., Price M., Neff C., Cioffi G., Waite K.A., Kruchko C., Barnholtz-Sloan J.S. CBTRUS Statistical Report: primary brain and other central nervous system tumors diagnosed in the United States in 2015– 2019. Neuro Oncol. 2022;24(Supplement_5):v1–v95. doi: 10.1093/neuonc/noac202

2. Clinical guidelines. Primary tumors of the central nervous system. 2020. Available at: https://oncologyassociation.ru/wp-content/uploads/2020/09/pervichnye_opuholi_cns.pdf [In Russian].

3. Idu A.A., Bogaciu N.S., Ciurea A.V. Brain imaging and morphological plasticity in glioblastoma: a literature review. J. Med. Life. 2023;16(3):344–347. doi: 10.25122/jml-2022-0201

4. Hughes J.D., Fattahi N., van Gompel J., Arani A., Meyer F., Lanzino G., Link M.J., Ehman R., Huston J. Higher-resolution magnetic resonance elastography in meningiomas to determine intratumoral consistency. Neurosurgery. 2015;77(4):653–659. doi: 10.1227/NEU.0000000000000892

5. Yeh F.C., IrimiaA., Bastos D.C.D.A., GolbyA.J. Tractography methods and findings in brain tumors and traumatic brain injury. NeuroImage. 2021;245:118651. doi: 10.1016/j.neuroimage.2021.118651

6. Krivoshapkin A.L., Sergeev G.S., Gaytan A.S., Kalneus L.E., Kurbatov V.P., Abdullaev O.A., Salim N., Bulanov D.V., Simonovich A.E. Automated volumetric analysis of postoperative magnetic resonance imaging predicts survival in patients with glioblastoma. World Neurosurg. 2019;126:e1510–e1517. doi: 10.1016/j.wneu.2019.03.142

7. Helland R.H., Ferles A., Pedersen A., Kommers I., Ardon H., Barkhof F., Bello L., Berger M.S., Dunås T., Nibali M.C., … Bouget D. Segmentation of glioblastomas in early post-operative multi-modal MRI with deep neural networks. Sci. Rep. 2023;13(1):18897. doi: 10.1038/s41598-023-45456-x

8. Krivoshapkin A.L., Sergeev G.S., Kalneus L.E., Gaytan A.S., Murtazin V.I., Kurbatov V.P., Volkov A.M., Kostromskaya D.V., Pyatov S.M., Amelin M.E., Duishobaev A.R. New software for preoperative diagnostics of meningeal tumor histologic types. World Neurosurg. 2016;90:123–132. doi: 10.1016/j.wneu.2016.02.084

9. Davis M. Glioblastoma: overview of disease and treatment. Clin. J. Oncol. Nurs. 2016;20(5):S2–S8. doi: 10.1188/16.CJON.S1.2-8

10. Nanda A., Bir S.C., Maiti T.K., Konar S.K., Missios S., Guthikonda B. Relevance of Simpson grading system and recurrence-free survival after surgery for World Health Organization Grade I meningioma. J. Neurosurg. 2017;126(1):201–211. doi: 10.3171/2016.1.JNS151842

11. Wen P.Y., van den Bent M., Youssef G., Cloughesy T.F., Ellingson B.M., Weller M., Galanis E., Barboriak D.P., De Groot J., Gilbert M.R., … ChangS.M. RANO 2.0: Update to the response assessment in neuro-oncology criteria for highand low-grade gliomas in adults. J. Clin. Oncol. 2023;41(33):5187–5199. doi: 10.1200/JCO.23.01059

12. Berntsen E.M., Stensjøen A.L., Langlo M.S., Simonsen S.Q., Christensen P., Moholdt V.A., Solheim O. Volumetric segmentation of glioblastoma progression compared to bidimensional products and clinical radiological reports. Acta Neurochir. (Wien). 2020;162(2):379–387. doi: 10.1007/s00701-019-04110-0

13. Visser M., Müller D.M.J., van Duijn R.J.M., Smits M., Verburg N., Hendriks E.J., Nabuurs R.J.A., Bot J.C.J., Eijgelaar R.S., Witte M., … de Munck J.C. Inter-rater agreement in glioma segmentations on longitudinal MRI. Neuroimage Clin. 2019;22:101727. doi: 10.1016/j.nicl.2019.101727

14. Sahoo P.K., Soltani S., Wong A.K.C. A survey of thresholding techniques. Computer Vision, Graphics, and Image Processing. 1988;41(2):233–260. doi: 10.1016/0734-189X(88)90022-9

15. Manousakas I.N., Undrill P.E., Cameron G.G., Redpath T.W. Split-and-merge segmentation of magnetic resonance medical images: performance evaluation and extension to three dimensions. Comput. Biomed. Res. 1998;31(6):393–412. doi: 10.1006/cbmr.1998.1489

16. Bezdek J.C., Hall L.O., Clarke L.P. Review of MR image segmentation techniques using pattern recognition. Med. Phys. 1993;20(4):1033–1048. doi: 10.1118/1.597000

17. Ashburner J., Friston K.J. Unified segmentation. Neuroimage. 2005;26(3):839–851. doi: 10.1016/j.neuroimage.2005.02.018

18. Dale A.M., Fischl B., Sereno M.I. Cortical surface-based analysis. Neuroimage. 1999;9(2):179–194. doi: 10.1006/nimg.1998.0395

19. Smith S.M., Jenkinson M., Woolrich M.W., Beckmann C.F., Behrens T.E.J., Johansen-Berg H., Bannister P.R., De Luca M., Drobnjak I., … Matthews P.M. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage. 2004;23:S208–S219. doi: 10.1016/j.neuroimage.2004.07.051

20. Otsu N. A threshold selection method from gray-level histograms. IEEE Trans. Syst., Man, Cybern. 1979;9(1):62–66. doi: 10.1109/TSMC.1979.4310076

21. Cordova J.S., Schreibmann E., Hadjipanayis C.G., Guo Y., Shu H.K.G., Shim H., Holder C.A. Quantitative tumor segmentation for evaluation of extent of glioblastoma resection to facilitate multisite clinical trials. Transl. Oncol. 2014;7(1):40–W5. doi: 10.1593/tlo.13835

22. Odland A., Server A., Saxhaug C., Breivik B., Groote R., Vardal J., Larsson C., Bjørnerud A. Volumetric glioma quantification: comparison of manual and semi-automatic tumor segmentation for the quantification of tumor growth. Acta. Radiol. 2015;56(11):1396– 1403. doi: 10.1177/0284185114554822

23. Zeng K., Bakas S., Sotiras A., Akbari H., Rozycki M., Rathore S., Pati S., Davatzikos C. Segmentation of gliomas in pre-operative and post-operative multimodal magnetic resonance imaging volumes based on a hybrid generative-discriminative framework. Brainlesion. 2016;10154:184–194. doi: 10.1007/978-3-31955524-9_18

24. Cireşan D.C., Giusti A., Gambardella L.M., Schmidhuber J. Deep neural networks segment neuronal membranes in electron microscopy images. Advances in Neural Information Processing Systems: proc. conf., Lake Tahoe Nevada, December 3–6, 2012. Red Hook: Curran Associates Inc, 2012. 2843–2851.

25. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Lecture Notes in Computer Science. New York: Springer International Publishing, 2015; 234–241. doi: 10.1007/978-3-319-24574-4_28

26. Isensee F., Jäger P.F., Full P.M., Vollmuth P., Maier-Hein K.H. nnU-Net for Brain Tumor Segmentation. In: Lecture Notes in Computer Science. New York: Springer International Publishing, 2021; 118–132. doi: 10.1007/978-3-030-72087-2_11

27. LaBella D., Adewole M., Alonso-Basanta M., Altes T., Anwar S.M., Baid U., Bergquist T., Bhalerao R., Chen S., Chung V., … Calabrese E. The ASNRMICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: intracranial meningioma. Published online May 12, 2023. doi: 10.48550/arXiv.2305.07642

28. de Verdier M.C., Saluja R., Gagnon L., LaBella D., Baid U., Tahon N.H., Foltyn-Dumitru M., Zhang J., Alafif M., Baig S., … Rudie J.D. The 2024 brain tumor segmentation (BraTS) challenge: glioma segmentation on post-treatment MRI. Published online 2024. doi: 10.48550/ARXIV.2405.18368

29. LaBella D., Schumacher K., Mix M., Leu K., McBurney-Lin S., Nedelec P., Villanueva-Meyer J., Shapey J., Vercauteren T., Chia K., … Calabrese E. Brain Tumor Segmentation (BraTS) Challenge 2024: meningioma radiotherapy planning automated segmentation. Published online 2024. doi: 10.48550/ARXIV.2405.18383

30. Rohlfing T., Zahr N.M., Sullivan E.V., Pfefferbaum A. The SRI24 multichannel atlas of normal adult human brain structure. Hum. Brain. Mapp. 2010;31(5):798–819. doi: 10.1002/hbm.20906

31. Isensee F., Schell M., Pflueger I., Brugnara G., Bonekamp D., Neuberger U., Wick A., Schlemmer H., Heiland S., Wick W., Bendszus M., Maier-Hein K.H., Kickingereder P. Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain. Mapp. 2019;40(17):4952–4964. doi: 10.1002/hbm.24750

32. Myronenko A. 3D MRI brain tumor segmentation using autoencoder regularization. In: Lecture Notes in Computer Science. New York: Springer International Publishing, 2019; P. 311–320. doi: 10.1007/978-3-03011726-9_28

33. Kingma D.P., Ba J. Adam: A method for stochastic optimization. Published online January 30, 2017. doi: 10.48550/arXiv.1412.6980

34. Smith L.N., Topin N. Super-convergence: very fast training of neural networks using large learning rates. Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications: proc. conf., Baltimore, April 14–18, 2019. Bellingham: SPIE, 2019; 1100612. doi: 10.1117/12.2520589

35. Lin T.Y., Goyal P., Girshick R., He K., Dollar P. Focal Loss for Dense Object Detection. IEEE International Conference on Computer Vision: proc. conf., Venice, October 22–29, 2017. Piscataway: IEEE, 2017; 2999–3007. doi: 10.1109/ICCV.2017.324

36. Sudre C.H., Li W., Vercauteren T., Ourselin S., Jorge Cardoso M. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Lecture Notes in Computer Science. New York: Springer International Publishing, 2017; 240–248. doi: 10.1007/978-3-319-67558-9_28

37. McGraw K.O., Wong S.P. Forming inferences about some intraclass correlation coefficients. Psychol. Methods. 1996;1(1):30–46. doi: 10.1037/1082989X.1.1.30

38. Mall P.K., Singh P.K., Srivastav S., Narayan V., Paprzycki M., Jaworska T., Ganzha M. A comprehensive review of deep neural networks for medical image processing: Recent developments and future opportunities. Healthc. Anal. (N.Y.). 2023;4(1):100216. doi: 10.1016/j.health.2023.100216

39. Ferreira A., Jesus T., Puladi B., Kleesiek J., Alves V., Egger J. Improved multi-task brain tumour segmentation with synthetic data augmentation. Published online December 2, 2024. doi: 10.48550/arXiv.2411.04632

40. Roy S., Koehler G., Ulrich C., Baumgartner M., Petersen J., Isensee F., Jäger P.F., Maier-Hein K.H. MedNeXt: transformer-driven scaling of convnets for medical image segmentation. In: Lecture Notes in Computer Science. Cham: Springer, 2023. P. 405–415. doi: 10.1007/978-3-031-43901-8_39