Investigation of the capabilities of algorithms for automated quality assurance of DICOM metadata of chest X-ray examinations

Purpose. To evaluate the quality of filling DICOM tags responsible for the orientation, scanning area and photometric interpretation of the image, as well as to develop and test algorithms for automatically determining the true values of these tags for chest x-rays and fluorograms.

Materials and methods. To assess the quality of filling DICOM tags, were used 1885 studies obtained from the Unified Radiological Information Service of the Unified Medical Information and Analysis System (ERIS EMIAS). For training and validation of algorithms for automatic determination of the true values of tags, were used datasets of radiographs in standard frontal and lateral projections, from open databases and from ERIS EMIAS (12,920 studies in total). The deep neural network architecture VGG 19 was chosen as the basis for creating algorithms.

Results. We found that the frequency of missing values in DICOM tags can range from 6 to 75%, depending on the tag. At the same time, up to 70% of filled tag values have errors. We obtained next models: a model for determining the anatomical area of x-ray examination, a model for determining the projection on the chest x-ray, a model for determining the photometric interpretation of the image. All of the obtained algorithms have high classification quality indicators. The AUC for each of the obtained models was more than 0.99.

Conclusions. Our study shows that a large number of studies in diagnostic practice contain incorrect values of DICOM tags, which can critically affect the implementation of software based on artificial intelligence technology in clinical practice. Our obtained algorithms can be integrated into the development process of such software and used in the preprocessing of images before their analysis.

1. McDonald R.J., Schwartz K.M., Eckel L.J. et al. The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad. Radiol. 2015; 22 (9): 1191–1198. https://doi.org/10.1016/j.acra.2015.05.007

2. van Leeuwen K.G., de Rooij M., Schalekamp S. et al. How does artificial intelligence in radiology improve efficiency and health outcomes? Pediatr. Radiol. 2022; 52 (11): 2087–2093. https://doi.org/10.1007/s00247-021-05114-8

3. Chetlen A.L., Chan T.L., Ballard D.H. et al. Addressing Burnout in Radiologists. Acad. Radiol. 2019; 26 (4): 526–533. https://doi.org/10.1016/j.acra.2018.07.001

4. Hosny A., Parmar Ch., Quackenbush J. et al. Artificial intelligence in radiology. Nat. Rev. Cancer. 2018; 18 (8): 500–510. https://doi.org/10.1038/s41568-018-0016-5

5. Rubin D.L. Artificial Intelligence in Imaging: The Radiologist’s Role. J. Am. Coll. Radiol. 2019; 16 (9): 1309–1317. https://doi.org/10.1016/j.jacr.2019.05.036

6. Savadjiev P., Chong J., Dohan A. et al. Demystification of AI-driven medical image interpretation: past, present and future. Eur. Radiol. 2019; 29 (3): 1616–1624. https://doi.org/10.1007/s00330-018-5674-x

7. Acosta J.N., Falcone G.J., Rajpurkar P. The Need for Medical Artificial Intelligence That Incorporates Prior Images. Radiology. 2022; 304 (2): 283–288. https://doi.org/10.1148/radiol.212830

8. Pavlov N.A., Andreychenko A.E., Vladzymyrskyy A.V. et al. Reference medical datasets (MosMedData) for independent external evaluation of algorithms based on artificial intelligence in diagnostics. Digital Diagnostics. 2021; 2 (1): 49–66. https://doi.org/10.17816/DD60635 (In Russian)

9. Willemink M.J., Koszek W.A., Hardell C. et al. Preparing Medical Imaging Data for Machine Learning. Radiology. 2020; 295 (1): 4–15. https://doi.org/10.1148/radiol.2020192224

10. Park S.H., Han K. Methodologic Guide for Evaluating Clinical Performance and Effect of Artificial Intelligence Technology for Medical Diagnosis and Prediction. Radiology. 2018; 286 (3): 800–809. https://doi.org/10.1148/radiol.2017171920

11. European Society of Radiology (ESR). What the radiologist should know about artificial intelligence – an ESR white paper. Insights. Imaging. 2019; 10 (1): 44. https://doi.org/10.1186/s13244-019-0738-2

12. Borisov A.A., Semenov S.S., Arzamasov K.M. Transfer Learning for automated search for defects on chest X-rays. Medical Visualization. 2023; 27 (1): 158–169. https://doi.org/10.24835/1607-0763-1243 (In Russian)

13. Juszczyk J., Badura P., Czajkowska J. et al. Automated size-specific dose estimates using deep learning image processing. Medical Image Analysis. 2021; 68: 101898. https://doi.org/10.1016/j.media.2020.101898

14. Keshavamurthy K.N., Elnajjar P., El-Rowmeim A. et al. Application of Deep Learning Techniques for Characterization of 3D Radiological Datasets – A Pilot Study for Detection of Intravenous Contrast in Breast MRI. Proc. SPIE Int. Soc. Opt. Eng. 2019; 10954: 109540X. https://doi.org/10.1117/12.2513809

15. DICOM standart // URL: https://www.dicomstandard.org/ (дата обращения 10.01.2023)

16. CheXpert Dataset //URL: https://stanfordmlgroup.github.io/competitions/chexpert/ (дата обращения 23.12.2022)

17. Chest X-rays dataset // URL: https://www.kaggle.com/datasets/raddar/chest-xrays-indiana-university (дата обращения 26.12.2022)

18. Chest X-Ray Images (Pneumonia)// URL: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia (дата обращения 20.12.2022)

19. NIH ChestX-ray14 //URL: https://nihcc.app.box.com/v/ChestXray-NIHCC (дата обращения 20.12.2022)

20. Han B., Du J., Jia Y. et al. Zero-Watermarking Algorithm for Medical Image Based on VGG19 Deep Convolution Neural Network. J. Healthc. Eng. 2021; 2021: 5551520. https://doi.org/10.1155/2021/5551520

21. Karacı A. VGGCOV19-NET: automatic detection of COVID-19 cases from X-ray images using modified VGG19 CNN architecture and YOLO algorithm. Neural. Comput. Appl. 2022; 34 (10): 8253–8274. https://doi.org/10.1007/s00521-022-06918-x

22. ROC analysis tool of Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow City Health Department // URL: https://roc-analysis.mosmed.ai/.

23. Mustra M., Delac K., Grgic M. et al. Overview of the DICOM standard. ELMAR, 2008. 50th International Symposium. Zadar, Croatia: 39–44. ISBN 978-1-4244-3364-3

24. Gueld M.O., Kohnen M., Keysers D. et al. Quality of DICOM header information for image categorization. Proc. SPIE 4685. Medical Imaging 2002: PACS and Integrated Medical Information Systems: Design and Evaluation. https://doi.org/10.1117/12.467017

25. Santosh K.C., Wendling L. Angular relational signature-based chest radiograph image view classification. Med. Biol. Eng. Comput. 2018; 56 (8): 1447–1458. https://doi.org/10.1007/s11517-018-1786-3

26. Urinbayev K., Orazbek Y., Nurambek Y. et al. End-to-End Deep Diagnosis of X-ray Images. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) in conjunction with the 43rd Annual Conference of the Canadian Medical and Biological Engineering Society. https://doi.org/10.1109/EMBC44109.2020.9175208