The impact of brain computed tomography metadata on the performance of diagnostic artificial intelligence services: a pilot study

The aim of the study is to evaluate the impact of metadata added to the training dataset on the performance of an artificial intelligence system aimed at working with diagnostic brain images.
Material and Methods. An expanded set of computed tomography scans of the brain with and without signs of intracranial haemorrhage, supplemented with clinical and technical parameters, was used as a basis. From the dataset, 176 studies were selected (106 for training, 70 for testing), which were specially prepared for input into the neural network: they were segmented with the region of interest (the brain) highlighted, normalized, and the background was removed. The ResNet10 neural network architecture, which is capable of analyzing 3D medical images, was used as the underlying neural network. It was combined with another neural network to form the ResNet10_Meta architecture, which, using a One-HotEnconding (OHE) approach, connects other metadata to analyze images directly. Four parameters were selected as metadata: ‘Manufacturer’, ‘SliceThickness’, ‘PatientAge’, and ‘XrayTubeCurrent’.
Results. Twelve experiments were conducted to train the neural network with one, two, or all of the additional parameters added alternately. The model with the addition of the ‘XTube’ parameter showed the highest specificity (82.3 %, 95 % confidence interval (95 % CI) [69.6–94.9]), outperforming the Baseline model without additional metadata (specificity 79.4 %, 95 % CI [66.0–92.8]). The ‘XTube + SliceT’ model with the addition of multiple metadata showed comparatively higher sensitivity (69.7 %, 95 % CI [54.5–84.9]) relative to Baseline model. The model with the addition of all metadata ‘All params’ did not show significant improvements. However, all differences found were statistically insignificant (p > 0.05).
Conclusions. Our data demonstrated that there were no statistically significant differences in the performance of a neural network analyzing diagnostic images without or with metadata added to the training dataset. However, this study is pilot and was conducted on a limited sample, so it remains to be seen whether commercially available artificial intelligence tools can be completely insensitive to image specifications.

1. Kwee T.C., Kwee R.M. Workload of diagnostic radiologists in the foreseeable future based on recent scientific advances: growth expectations and role of artificial intelligence. Insights Imaging. 2021;12(1):88–101. doi: 10.1186/s13244-021-01031-4

2. Arzamasov K., Vasilev Y., Vladzymyrskyy A., Omelyanskaya O., Shulkin I., Kozikhina D., Goncharova I., Gelezhe P., Kirpichev Y., Bobrovskaya T., Andreychenko A. An international non-inferiority study for the benchmarking of ai for routine radiology cases: chest X-ray, fluorography and mammography. Healthcare (Basel). 2023;11(12):1684. doi: 10.3390/healthcare11121684

3. Solovev A.V., Vasilev Yu.A., Sinitsyn V.E., Petryaykin A.V., Vladzimirskiy A.V., Shulkin I.M., Sharova D.E., Semenov D.S. Improving aortic aneurysm detection with artificial intelligence based on chest computed tomography data. Digital Diagnostics. 2024;5(1):29–40. [In Russian]. doi: 10.17816/DD569388

4. Bizzo B.C., Almeida R.R., Michalski M.H., Alkasab T.K. Artificial Intelligence and Clinical Decision Support for Radiologists and Referring Providers. J. Am. Coll. Radiol. 2019;16(9 Pt B):1351–1356. doi: 10.1016/j.jacr.2019.06.010

5. Jha S. Value of triage by artificial intelligence. Acad. Radiol. 2020;27(1):153–155. doi: 10.1016/j.acra.2019.11.002

6. Alukaev D., Kiselev S., Mustafaev T., Ainur A., Ibragimov B., Vrtovec T. A deep learning framework for vertebral morphometry and Cobb angle measurement with external validation. Eur. Spine J. 2022;31(8):2115–2124. doi: 10.1007/s00586-022-07245-4

7. Hamyoon H., Yee Chan W., Mohammadi A., Yusuf Kuzan T., Mirza-Aghazadeh-Attari M., Leong W.L., Murzoglu Altintoprak K., Vijayananthan A, Rahmat K., Ab Mumin N., … Abbasian Ardakani A. Artificial intelligence, BI-RADS evaluation and morphometry: A novel combination to diagnose breast cancer using ultrasonography, results from multi-center cohorts. Eur. J. Radiol. 2022;157:110591. doi: 10.1016/j.ejrad.2022.110591

8. Tariq A., Purkayastha S., Padmanaban G.P., Krupinski E., Trivedi H., Banerjee I., Gichoya J.W. Current clinical applications of artificial intelligence in radiology and their best supporting evidence. J. Am. Coll. Radiol. 2020;17(11):1371–1381. doi: 10.1016/j.jacr.2020.08.018

9. Alwosheel A., van Cranenburgh S., Chorus C.G. Is your dataset big enough? Sample size requirements when using artificial neural networks for discrete choice analysis. J. Choice Mod. 2018;28:167–182. doi: 10.1016/j.jocm.2018.07.002

10. Vasilev Yu.A., Bobrovskaya T.M., Arzamasov K.M., Chetverikov S.F., Vladzimirskiy A.V., Omelyanskaya O.V., Andreychenko A.E., Pavlov N.A., Anishchenko L.N. Medical datasets for machine learning: fundamental principles of standartization and systematization. Menedzher zdravookhraneniya = Manager of Health Care. 2023;(4):28–41. [In Russian]. doi: 10.21045/1811-0185-2023-4-28-41

11. Park S.H., Han K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology. 2018;286(3):800–809. doi: 10.1148/radiol.2017171920

12. Zech J.R., Badgeley M.A., Liu M., Costa A.B., Titano J.J., Oermann E.K. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 2018;15(11):e1002683. doi: 10.1371/journal.pmed.1002683

13. Vasilev Yu.A., Vladzimirskiy A.V., Omelyanskaya O.V., Reshetnikov R.V., Blokhin I.A., Kodenko M.R., Nanova O.G. Review of meta-analyses on the use of artificial intelligence in radiology. Meditsinskaya vizualizatsiya = Medical Visualization. 2024;28(3):22–41. [In Russian]. doi: 10.24835/1607-0763-1425

14. Morozov S., Vladzimirskiy A., Ledikhova N., Andreychenko A., Arzamasov K., Omelyanskaya O., Reshetnikov R., Gelezhe P., Blokhin I., Turavilova E., … Bondarchuk D. Diagnostic accuracy of artificial intelligence for analysis of 1.3 million medical imaging studies: the Moscow experiment on computer vision technologies. medRxiv. Preprint. Available at: https://doi.org/10.1101/2023.08.31.23294896

15. Seyam M., Weikert T., Sauter A., Brehm A., Psychogios M.N., Blackham K.A. Utilization of artificial intelligence-based intracranial hemorrhage detection on emergent noncontrast CT images in clinical workflow. Radiol. Artif. Intell. 2022;4(2):e210168. doi: 10.1148/ryai.210168

16. Voter A.F., Meram E., Garrett J.W., Yu J.J. Diagnostic accuracy and failure mode analysis of a deep learning algorithm for the detection of intracranial hemorrhage. J .Am. Coll. Radiol. 2021;18(8):1143–1152. doi: 10.1016/j.jacr.2021.03.005

17. Zia A., Fletcher C., Bigwood S., Ratnakanthan P., Seah J., Lee R., Kavnoudias H., Law M. Retrospective analysis and prospective validation of an AI-based software for intracranial haemorrhage detection at a high-volume trauma centre. Sci. Rep. 2022;12(1):19885. doi: 10.1038/s41598-022-24504-y

18. Schaller S., Wildberger J.E., Raupach R., Niethammer M., Klingenbeck-Regn K., Flohr T. Spatial domain filtering for fast modification of the tradeoff between image sharpness and pixel noise in computed tomography. IEEE Trans. Med. Imaging. 2003;22(7):846–853. doi: 10.1109/TMI.2003.815073

19. Wang Y., de Bock G.H., van Klaveren R.J., van Ooyen P., Tukker W., Zhao Y., Dorrius M.D., Proença R.V., Post W.J., Oudkerk M. Volumetric measurement of pulmonary nodules at low-dose chest CT: effect of reconstruction setting on measurement variability. Eur. Radiol. 2010;20(5):1180–1197. doi: 10.1007/s00330-009-1634-9

20. Behrendt F.F., Das M., Mahnken A.H., Kraus T., Bakai A., Stanzel S., Günther R.W., Wildberger J.E. Computer-aided measurements of pulmonary emphysema in chest multidetector-row spiral computed tomography: effect of image reconstruction parameters. J. Comput. Assist. Tomogr. 2008;32(6):899–904. doi: 10.1097/RCT.0b013e31815ade64

21. Yeo M., Tahayori B., Kok H.K., Maingard J., Kutaiba N., Russell J., Thijs V., Jhamb A., Chandra R.V., Brooks M., Barras C.D., Asadi H. Review of deep learning algorithms for the automatic detection of intracranial hemorrhages on computed tomography head imaging. J. Neurointerv. Surg. 2021;13(4):369–378. doi: 10.1136/neurintsurg-2020-017099

22. Khoruzhaya A.N., Bobrovskaya T.M., Kozlov D.V., Kuligovskiy D., Novik V.P., Arzamasov K.M., Kremneva E.I. Expanded brain Ct dataset for the development of AI systems for intracranial hemorrhage detection and classification. Data. 2024;9(2):30–36. doi: 10.3390/data9020030

23. Cai J.C., Akkus Z., Philbrick K.A., Boonrod A., Hoodeshenas S., Weston A.D., Rouzrokh P., Conte G.M., Zeinoddini A., Vogelsang D.C., Huang Q., Erickson B.J. Fully automated segmentation of head ct neuroanatomy using deep learning. Radiol. Artif. Intell. 2020;2(5):e190183. doi: 10.1148/ryai.2020190183

24. Kundisch A., Hönning A., Mutze S., Kreissl L., Spohn F., Lemcke J., Sitz M., Sparenberg P., Goelz L. Deep learning algorithm in detecting intracranial hemorrhages on emergency computed tomographies. PLoS One. 2021;16(11):e0260560. doi: 10.1371/journal.pone.0260560

25. Schutte K., Brulport F., Harguem-Zayani S., Schiratti J.B., Ghermi R., Jehanno P., Jaeger A., Alamri T., Naccache R., Haddag-Miliani L., … Lassau N. An artificial intelligence model predicts the survival of solid tumour patients from imaging and clinical data. Eur. J. Cancer. 2022;174:90–98. doi: 10.1016/j.ejca.2022.06.055

26. Experiment on the use of innovative technologies in the field of computer vision to analyse medical images and further application in the health care system of Moscow. Available at: https://mosmed.ai/ai/ [In Russian].

27. van Asch C.J., Luitse M.J., Rinkel G.J., van der Tweel I., Algra A., Klijn C.J. Incidence, case fatality, and functional outcome of intracerebral haemorrhage over time, according to age, sex, and ethnic origin: a systematic review and meta-analysis. Lancet Neurol. 2010;9(2):167–176. doi: 10.1016/S1474-4422(09)70340-0

28. McLouth J., Elstrott S., Chaibi Y., Quenet S., Chang P.D., Chow D.S., Soun J.E. Validation of a deep learning tool in the detection of intracranial hemorrhage and large vessel occlusion. Front Neurol. 2021;12:656112. doi: 10.3389/fneur.2021.656112