Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice

Viraj Kulkarni; Manish Gawali; Amit Kharat

doi:10.2196/28776

journal article Sep 09, 2021

Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice

Viraj Kulkarni

Manish Gawali

Amit Kharat

JMIR Medical Informatics Vol. 9 No. 9 pp. e28776 · JMIR Publications Inc.

View at Publisher Save 10.2196/28776

Abstract

The use of machine learning to develop intelligent software tools for the interpretation of radiology images has gained widespread attention in recent years. The development, deployment, and eventual adoption of these models in clinical practice, however, remains fraught with challenges. In this paper, we propose a list of key considerations that machine learning researchers must recognize and address to make their models accurate, robust, and usable in practice. We discuss insufficient training data, decentralized data sets, high cost of annotations, ambiguous ground truth, imbalance in class representation, asymmetric misclassification costs, relevant performance metrics, generalization of models to unseen data sets, model decay, adversarial attacks, explainability, fairness and bias, and clinical validation. We describe each consideration and identify the techniques used to address it. Although these techniques have been discussed in prior research, by freshly examining them in the context of medical imaging and compiling them in the form of a laundry list, we hope to make them more accessible to researchers, software developers, radiologists, and other stakeholders.

Topics

No keywords indexed for this article. Browse by subject →

References

123

[1]

10.1136/bmj.j4683

[2]

10.1007/s11604-008-0259-2

[3]

10.1038/s41568-018-0016-5

[4]

Maretíc, Z Dermatologica (1986) 10.1159/000249312

[5]

Deep Learning in Medical Image Analysis

Dinggang Shen, Guorong Wu, Heung-Il Suk

Annual Review of Biomedical Engineering 10.1146/annurev-bioeng-071516-044442

[6]

10.1080/01431169508954507

[7]

10.1093/jnci/90.23.1801

[8]

10.4236/ojmi.2015.53018

[9]

Learning from Imbalanced Data

Haibo He, E.A. Garcia

IEEE Transactions on Knowledge and Data Engineerin... 10.1109/tkde.2008.239

[10]

PoochEBallesterPBarrosRCan we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classificationarXiv.org20202021-08-16http://arxiv.org/abs/1909.01940

[11]

10.1007/bf00116900

[12]

SzegedyCZarembaWSutskeverIBrunaJErhanDGoodfellowIFergusRIntriguing properties of neural networksarXiv.org20142021-08-16https://arxiv.org/abs/1312.6199

[13]

SteinhardtJKohPLiangPCertified defenses for data poisoning attacksarXiv.org20172021-08-16https://arxiv.org/abs/1706.03691

[14]

Biggio, B Machine Learning and Knowledge Discovery in Databases (2013)

[15]

10.1016/j.cmpb.2020.105608

[16]

10.1126/science.aax2342

[17]

10.1148/radiol.2018180547

[18]

10.1109/cvpr.2017.369

[19]

10.1016/j.media.2020.101797

[20]

JohnsonAPollardTGreenbaumNLungrenMDengCPengYLuZMarkRBerkowitzSHorngSMIMIC-CXR-JPG, a large publicly available database of labeled chest radiographsarXiv.org20192021-08-16http://arxiv.org/abs/1901.07042

[21]

10.1186/s40537-019-0197-0

[22]

10.1016/j.media.2019.101552

[23]

10.1186/1471-2458-14-1144

[24]

Voigt, P The EU General Data Protection Regulation (GDPR) (2017) 10.1007/978-3-319-57959-7

[25]

10.1056/nejmlim035027

[26]

10.1016/j.clsr.2013.07.010

[27]

McMahanHMooreERamageDHampsonSArcasBCommunication-efficient learning of deep networks from decentralized dataProceedings of the 20th International Conference on Artificial Intelligence and Statistics201720th International Conference on Artificial Intelligence and StatisticsFort Lauderdale, Florida, USAMay 9-11, 2017

[28]

VepakommaPGuptaOSwedishTRaskarRSplit learning for health: Distributed deep learning without sharing raw patient dataarXiv.org20182021-08-16http://arxiv.org/abs/1812.00564

[29]

10.1007/978-3-030-80432-9_34

[30]

10.1109/worlds450073.2020.9210355

[31]

10.1148/radiol.2017171920

[32]

10.1148/radiol.2018171093

[33]

10.18653/v1/2020.emnlp-main.117

[34]

10.1148/radiol.16142770

[35]

10.1016/j.media.2019.03.009

[36]

FeyjieAAzadRPedersoliMKauffmanCAyedIDolzJSemi-supervised few-shot learning for medical image segmentationarXiv.org20202021-08-16http://arxiv.org/abs/2003.08462

[37]

10.1016/j.patcog.2018.02.017

[38]

10.1007/s13244-018-0596-3

[39]

OlatunjiTYaoLCovingtonBRhodesAUptonACaveats in generating medical imaging labels from radiology reportsarXiv.org20192021-08-16http://arxiv.org/abs/1905.02283

[40]

10.1016/j.jacr.2018.05.037

[41]

10.1186/1748-717x-7-32

[42]

10.4103/0971-6203.44472

[43]

CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison

Jeremy Irvin, Pranav Rajpurkar, Michael Ko et al.

Proceedings of the AAAI Conference on Artificial I... 10.1609/aaai.v33i01.3301590

[44]

10.1186/1471-2334-12-31

[45]

The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans

Samuel G. Armato, Geoffrey McLennan, Luc Bidaut et al.

Medical Physics 10.1118/1.3528204

[46]

10.1148/radiol.2019191293

[47]

10.1145/1553374.1553488

[48]

Survey on deep learning with class imbalance

Justin M. Johnson, Taghi M. Khoshgoftaar

Journal of Big Data 10.1186/s40537-019-0192-5

[49]

10.1016/j.ipm.2010.11.007

[50]

10.1007/s11192-018-2865-9

Showing 50 of 123 references

Cited By

26

A classical–quantum convolutional neural network for detecting pneumonia from chest radiographs

Viraj Kulkarni, Sanjesh Pawale · 2023

Neural Computing and Applications

Metrics

26

Citations

123

References

Details

Published: Sep 09, 2021
Vol/Issue: 9(9)
Pages: e28776

Authors

Cite This Article

Viraj Kulkarni, Manish Gawali, Amit Kharat (2021). Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice. JMIR Medical Informatics, 9(9), e28776. https://doi.org/10.2196/28776

Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice

You May Also Like