Deep convolutional neural network (CNN) model optimization techniques—Review for medical imaging

Ghazanfar Latif; Jaafar Alghazo; Majid Ali Khan; Ghassen Ben Brahim; Khaled Fawagreh; Nazeeruddin Mohammad

doi:10.3934/math.2024998

journal article Jan 01, 2024

Deep convolutional neural network (CNN) model optimization techniques—Review for medical imaging

Ghazanfar Latif

Jaafar Alghazo

Majid Ali Khan Ghassen Ben Brahim

Khaled Fawagreh Nazeeruddin Mohammad

AIMS Mathematics Vol. 9 No. 8 pp. 20539-20571 · American Institute of Mathematical Sciences (AIMS)

View at Publisher Save 10.3934/math.2024998

Abstract

The field of artificial intelligence (AI) and machine learning (ML) has been expanding and is explored by researchers in various fields. In medical diagnosis, for instance, the field of AI/ML is being explored because if medical diagnostic devices are built and designed with a backend of AI/ML, then the benefits would be unprecedented. Automated diagnostic tools would result in reduced health care costs, diagnosis without human intervention, overcoming human errors, and providing adequate and affordable medical care to a wider portion of the population with portions of the actual cost. One domain where AI/ML can make an immediate impact is medical imaging diagnosis (MID), namely the classification of medical images, where researchers have applied optimization techniques aiming to improve image classification accuracy. In this paper, we provide the research community with a comprehensive review of the most relevant studies to date on the use of deep CNN architecture optimization techniques for MID. As a case study, the application of these techniques to COVID-19 medical images were made. The impacts of the related variables, including datasets and AI/ML techniques, were investigated in detail. Additionally, the significant shortcomings and challenges of the techniques were touched upon. We concluded our work by affirming that the application of AI/ML techniques for MID will continue for many years to come, and the performance of the AI/ML classification techniques will continue to increase.

Topics

No keywords indexed for this article. Browse by subject →

References

86

[1]

V. Sharma, M. G. Dastidar, S. Sutradhar, V. Raj, K. De Silva, S. Roy, A step toward better sample management of COVID-19: On-spot detection by biometric technology and artificial intelligence, COVID-19 Sustain, Develop. Goals, 2022 (2022), 349–380. https://doi.org/10.1016/B978-0-323-91307-2.00017-1 10.1016/b978-0-323-91307-2.00017-1

[2]

G. Latif, H. Morsy, A. Hassan, J. Alghazo, Novel coronavirus and common pneumonia detection from CT scans using deep learning-based extracted features, Viruses, 14 (2022), 1667. https://doi.org/10.3390/v14081667 10.3390/v14081667

[3]

A. Islam, T. Rahim, M. Masuduzzaman, S. Y. Shin, A blockchain-based artificial intelligence-empowered contagious pandemic situation supervision scheme using internet of drone things, IEEE Wirel. Commun., 28 (2021), 166–173. https://doi.org/10.1109/MWC.001.2000429 10.1109/mwc.001.2000429

[4]

T. Rahim, M. A. Usman, S. Y. Shin, A survey on contemporary computer-aided tumor, polyp, and ulcer detection methods in wireless capsule endoscopy imaging, Comput. Med. Imag. Grap., 85 (2020), 101767. https://doi.org/10.1016/j.compmedimag.2020.101767 10.1016/j.compmedimag.2020.101767

[5]

G. Latif, DeepTumor: Framework for brain MR image classification, segmentation and tumor detection, Diagnostics, 12 (2022), 2888. https://doi.org/10.3390/diagnostics12112888 10.3390/diagnostics12112888

[6]

T. Rahim, S. A. Hassan, S. Y. Shin, A deep convolutional neural network for the detection of polyps in colonoscopy images, Biomed. Signal Proces., 68 (2021), 102654. https://doi.org/10.1016/j.bspc.2021.102654 10.1016/j.bspc.2021.102654

[7]

A. Bashar, G. Latif, G. Ben Brahim, N. Mohammad, J. Alghazo, COVID-19 pneumonia detection using optimized deep learning techniques, Diagnostics, 11 (2021), 1972. https://doi.org/10.3390/diagnostics11111972 10.3390/diagnostics11111972

[8]

E. Hussain, M. Hasan, M. A. Rahman, I. Lee, T. Tamanna, M. Z. Parvez, CoroDet: A deep learning based classification for COVID-19 detection using chest X-ray images, Chaos Soliton. Fract., 142 (2021), 110495. https://doi.org/10.1016/j.chaos.2020.110495 10.1016/j.chaos.2020.110495

[9]

G. Latif, G. Ben Brahim, D. N. F. A. Iskandar, A. Bashar, J. Alghazo, Glioma tumors' classification using deep-neural-network-based features with SVM classifier, Diagnostics, 12 (2022), 1018. https://doi.org/10.3390/diagnostics12041018 10.3390/diagnostics12041018

[10]

I. Iqbal, M. Younus, K. Walayat, M. U. Kakar, J. Ma, Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images, Comput. Med. Imag. grap, 88 (2021), 101843. https://doi.org/10.1016/j.compmedimag.2020.101843 10.1016/j.compmedimag.2020.101843

[11]

I. Iqbal, K. Walayat, M. U. Kakar, J. Ma, Automated identification of human gastrointestinal tract abnormalities based on deep convolutional neural network with endoscopic images, Intell. Syst. Appl., 16 (2022), 200149. https://doi.org/10.1016/j.iswa.2022.200149 10.1016/j.iswa.2022.200149

[12]

V. Shah, R. Keniya, A. Shridharani, M. Punjabi, J. Shah, N. Mehendale, Diagnosis of COVID-19 using CT scan images and deep learning techniques, Emerg. Radiol., 28 (2021), 497–505. https://doi.org/10.1007/s10140-020-01886-y 10.1007/s10140-020-01886-y

[13]

M. M. Rahaman, C. Li, Y. Yao, K. Frank, M. A. Rahman, Q. Wang, et al., Identification of COVID-19 samples from chest X-Ray images using deep learning: A comparison of transfer learning approaches, J. X-Ray Sci. Technol., 28 (2020), 821–839. https://doi.org/10.3233/XST-200715 10.3233/xst-200715

[14]

A. S. Al-Waisy, S. Al-Fahdawi, M. A. Mohammed, K. H. Abdulkareem, S. A. Mostafa, M. S. Maashi, et al., COVID-CheXNet: Hybrid deep learning framework for identifying COVID-19 virus in chest X-rays images, Soft Comput., 27 (2020), 2657–2672. https://doi.org/10.1007/s00500-020-05424-3 10.1007/s00500-020-05424-3

[15]

Y. Chang, X. Jing, Z. Ren, B. Schuller, CovNet: A transfer learning framework for automatic COVID-19 detection from crowd-sourced cough sounds, Front. Digit. Health, 3 (2022), 799067. https://doi.org/10.3389/fdgth.2021.799067 10.3389/fdgth.2021.799067

[16]

M. Elpeltagy, H. Sallam, Automatic prediction of COVID-19 from chest images using modified ResNet50, Multimed. Tools Appl., 80 (2021), 26451–26463. https://doi.org/10.1007/s11042-021-10783-6 10.1007/s11042-021-10783-6

[17]

R. K. Patel, M. Kashyap, Automated diagnosis of COVID stages from lung CT images using statistical features in 2-dimensional flexible analytic wavelet transform, Biocybern. Biomed. Eng., 42 (2022), 829–841. https://doi.org/10.1016/j.bbe.2022.06.005 10.1016/j.bbe.2022.06.005

[18]

D. K. Redie, A. E. Sirko, T. M. Demissie, S. S. Teferi, V. K. Shrivastava, O. P. Verma, et al., Diagnosis of COVID-19 using chest X-ray images based on modified DarkCovidNet model, Evol Intell., 16 (2022), 729–738. https://doi.org/10.1007/s12065-021-00679-7 10.1007/s12065-021-00679-7

[19]

F. Özyurt, Efficient deep feature selection for remote sensing image recognition with fused deep learning architectures, J. Supercomput, 76 (2020), 8413–8431. https://doi.org/10.1007/s11227-019-03106-y 10.1007/s11227-019-03106-y

[20]

Receptive fields, binocular interaction and functional architecture in the cat's visual cortex

D. H. Hubel, T. N. Wiesel

The Journal of Physiology 10.1113/jphysiol.1962.sp006837

[21]

Y. LeCun, Y. Bengio, Convolutional networks for images, speech, and time series, In: The handbook of brain theory and neural networks, 1995.

[22]

G. Latif, J. Alghazo, L. Alzubaidi, M. N. Nasser, Y. Alghazo, Deep convolutional neural network for recognition of unified multi-language handwritten numerals, In: 2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR), 2018. <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1109/ASAR.2018.8480289">https://doi.org/10.1109/ASAR.2018.8480289</ext-link> 10.1109/asar.2018.8480289

[23]

ImageNet classification with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton

Communications of the ACM 10.1145/3065386

[24]

Going deeper with convolutions

Christian Szegedy, Wei Liu, Yangqing Jia et al.

2015 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2015.7298594

[25]

S. Alghamdi, M. Alabkari, F. Aljishi, G. Latif, A. Bashar, Lung cancer detection from LDCT images using deep convolutional neural networks, In: International Conference on Communication, Computing and Electronics Systems, Singapore: Springer, 733 (2021), 363–374. <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1007/978-981-33-4909-4_27">https://doi.org/10.1007/978-981-33-4909-4_27</ext-link>

[26]

D. A. Alghmgham, G. Latif, J. Alghazo, L. Alzubaidi, Autonomous traffic sign (ATSR) detection and recognition using deep CNN, Procedia Comput. Sci., 163 (2019), 266–274. https://doi.org/10.1016/j.procs.2019.12.108 10.1016/j.procs.2019.12.108

[27]

G. Latif, N. Mohammad, R. AlKhalaf, R. AlKhalaf, J. Alghazo, M. Khan, An automatic arabic sign language recognition system based on deep CNN: An assistive system for the deaf and hard of hearing, Int. J. Comput. Digital Syst., 9 (2020), 715–724. http://doi.org/10.12785/ijcds/090418 10.12785/ijcds/090418

[28]

B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, A. Oliva, Learning deep features for scene recognition using places database, In: NIPS'14: Proceedings of the 27th International Conference on Neural Information Processing Systems, 1 (2014), 487–495.

[29]

M. M. Butt, G. Latif, D. N. F. A. Iskandar, J. Alghazo, A. H. Khan, Multi-channel convolutions neural network based diabetic retinopathy detection from fundus images, Procedia Comput. Sci., 163 (2019), 283–291. https://doi.org/10.1016/j.procs.2019.12.110 10.1016/j.procs.2019.12.110

[30]

D. C. Cireşan, U. Meier, J. Masci, L. Gambardella, J. Schmidhuber, Flexible, high performance convolutional neural networks for image classification, In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, 2011, 1237–1242. <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-210">https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-210</ext-link>

[31]

G. Lokku, G. H. Reddy, M. N. G. Prasad, OPFaceNet: Optimized face recognition network for noise and occlusion affected face images using hyperparameters tuned convolutional neural network, Appl. Soft Comput., 117 (2022), 108365. https://doi.org/10.1016/j.asoc.2021.108365 10.1016/j.asoc.2021.108365

[32]

S. Y. Kim, Z. W. Geem, G. Han, Hyperparameter optimization method based on harmony search algorithm to improve performance of 1D CNN human respiration pattern recognition system, Sensors, 20 (2020), 3697. https://doi.org/10.3390/s20133697 10.3390/s20133697

[33]

G. Latif, K. Bouchard, J. Maitre, A. Back, L. P. Bedard, Deep-learning-based automatic mineral grain segmentation and recognition, Minerals, 12 (2022), 455. https://doi.org/10.3390/min12040455 10.3390/min12040455

[34]

Invariant Scattering Convolution Networks

Joan Bruna, S. Mallat

IEEE Transactions on Pattern Analysis and Machine... 10.1109/tpami.2012.230

[35]

S. Lawrence, C. L. Giles, A. C. Tsoi, What size neural network gives optimal generalization? Convergence properties of backpropagation, In: Digital Repository at the University of Maryland, 1998.

[36]

L. Wan, M. Zeiler, S. Zhang, Y. Cun, R. Fergus, Regularization of neural networks using dropconnect, In: ICML'13: Proceedings of the 30th International Conference on International Conference on Machine Learning, 28 (2013), 1058–1066.

[37]

Q. Xu, M. Zhang, Z. Gu, G. Pan, Overfitting remedy by sparsifying regularization on fully-connected layers of CNNs, Neurocomputing, 328 (2019), 69–74. https://doi.org/10.1016/j.neucom.2018.03.080 10.1016/j.neucom.2018.03.080

[38]

S. R. Dubey, S. K. Singh, B. B. Chaudhuri, Activation functions in deep learning: A comprehensive survey and benchmark, Neurocomputing, 503 (2022), 92–108. https://doi.org/10.1016/j.neucom.2022.06.111 10.1016/j.neucom.2022.06.111

[39]

S. Akbar, M. Peikari, S. Salama, S. Nofech-Mozes, A. Martel, The transition module: A method for preventing overfitting in convolutional neural networks, Comput. Methods Biomech. Biomed. Eng.: Imaging Vis., 7 (2019), 260–265. https://doi.org/10.1080/21681163.2018.1427148 10.1080/21681163.2018.1427148

[40]

H. Wu, X. Gu, Towards dropout training for convolutional neural networks, Neural Networks, 71 (2015), 1–10. https://doi.org/10.1016/j.neunet.2015.07.007 10.1016/j.neunet.2015.07.007

[41]

Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network

Marios Anthimopoulos, Stergios Christodoulidis, Lukas Ebner et al.

IEEE Transactions on Medical Imaging 10.1109/tmi.2016.2535865

[42]

J. Chen, Y. Shen, The effect of kernel size of CNNs for lung nodule classification, In: 2017 9th International Conference on Advanced Infocomm Technology (ICAIT), 2017,340–344. <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1109/ICAIT.2017.8388942">https://doi.org/10.1109/ICAIT.2017.8388942</ext-link> 10.1109/icait.2017.8388942

[43]

B. Chen, W. Deng, J. Du, Noisy softmax: Improving the generalization ability of DCNN via postponing the early softmax saturation, In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 4021–4030. <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1109/CVPR.2017.428">https://doi.org/10.1109/CVPR.2017.428</ext-link> 10.1109/cvpr.2017.428

[44]

Greedy Layer-Wise Training of Deep Networks

Yoshua Bengio, Pascal Lamblin, Dan Popovici et al.

Advances in Neural Information Processing Systems... 10.7551/mitpress/7503.003.0024

[45]

Deep Residual Learning for Image Recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.90

[46]

S. Han, J. Pool, J. Tran, W. J. Dally, Learning both weights and connections for efficient neural network, In: NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems, 1 (2015), 1135–1143.

[47]

P. Ochs, A. Dosovitskiy, T. Brox, T. Pock, On iteratively reweighted algorithms for nonsmooth nonconvex optimization in computer vision, SIAM J. Imaging Sci., 8 (2015), 331–372. https://doi.org/10.1137/140971518 10.1137/140971518

[48]

P. Murugan, S. Durairaj, Regularization and optimization strategies in deep convolutional neural network, 2017, arXiv: 1712.04711.

[49]

J. Snoek, O. Rippel, K. Swersky, R. Kiros, N. Satish, N. Sundaram, et al., Scalable Bayesian optimization using deep neural networks, In: ICML'15: Proceedings of the 32nd International Conference on International Conference on Machine Learning, 37 (2015), 2171–2180.

[50]

D. Cheng, Y. Gong, S. Zhou, J. Wang, N. Zheng, Person re-identification by multi-channel parts-based CNN with improved triplet loss function, In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 1335–1344. <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1109/CVPR.2016.149">https://doi.org/10.1109/CVPR.2016.149</ext-link> 10.1109/cvpr.2016.149

Showing 50 of 86 references

Metrics

7

Citations

86

References

Details

Published: Jan 01, 2024
Vol/Issue: 9(8)
Pages: 20539-20571

Authors

G

Ghazanfar Latif