Explainable Vision Transformer with Self-Supervised Learning to Predict Alzheimer’s Disease Progression Using 18F-FDG PET

Uttam Khatri; Goo-Rak Kwon

doi:10.3390/bioengineering10101225

journal article Open Access Oct 20, 2023

Explainable Vision Transformer with Self-Supervised Learning to Predict Alzheimer’s Disease Progression Using 18F-FDG PET

Uttam Khatri

Goo-Rak Kwon

Bioengineering Vol. 10 No. 10 pp. 1225 · MDPI AG

View at Publisher Save 10.3390/bioengineering10101225

Abstract

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder that affects millions of people worldwide. Early and accurate prediction of AD progression is crucial for early intervention and personalized treatment planning. Although AD does not yet have a reliable therapy, several medications help slow down the disease’s progression. However, more study is still needed to develop reliable methods for detecting AD and its phases. In the recent past, biomarkers associated with AD have been identified using neuroimaging methods. To uncover biomarkers, deep learning techniques have quickly emerged as a crucial methodology. A functional molecular imaging technique known as fluorodeoxyglucose positron emission tomography (18F-FDG-PET) has been shown to be effective in assisting researchers in understanding the morphological and neurological alterations to the brain associated with AD. Convolutional neural networks (CNNs) have also long dominated the field of AD progression and have been the subject of substantial research, while more recent approaches like vision transformers (ViT) have not yet been fully investigated. In this paper, we present a self-supervised learning (SSL) method to automatically acquire meaningful AD characteristics using the ViT architecture by pretraining the feature extractor using the self-distillation with no labels (DINO) and extreme learning machine (ELM) as classifier models. In this work, we examined a technique for predicting mild cognitive impairment (MCI) to AD utilizing an SSL model which learns powerful representations from unlabeled 18F-FDG PET images, thus reducing the need for large-labeled datasets. In comparison to several earlier approaches, our strategy showed state-of-the-art classification performance in terms of accuracy (92.31%), specificity (90.21%), and sensitivity (95.50%). Then, to make the suggested model easier to understand, we highlighted the brain regions that significantly influence the prediction of MCI development. Our methods offer a precise and efficient strategy for predicting the transition from MCI to AD. In conclusion, this research presents a novel Explainable SSL-ViT model that can accurately predict AD progress based on 18F-FDG PET scans. SSL, attention, and ELM mechanisms are integrated into the model to make it more predictive and interpretable. Future research will enable the development of viable treatments for neurodegenerative disorders by combining brain areas contributing to projection with observed anatomical traits.

Topics

No keywords indexed for this article. Browse by subject →

References

63

[1]

Ottoy "Association of short-term cognitive decline and MCI-to-AD dementia conversion with CSF, MRI, amyloid- and 18F-FDG-PET imaging" NeuroImage Clin. (2019) 10.1016/j.nicl.2019.101771

[2]

(2023, September 10). World Alzheimer Report 2021: Journey through the Diagnosis of Dementia. Available online: https://www.alzint.org/u/World-Alzheimer-Report-2021.pdf.

[3]

(2023). 2023 Alzheimer’s disease facts and figures. Alzheimer’s Dement., 19, alz.13016.

[4]

Silveira, M., and Marques, J. (2010, January 23–26). Boosting Alzheimer Disease Diagnosis Using PET Images. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey. 10.1109/icpr.2010.626

[5]

Hampel "The Amyloid-β Pathway in Alzheimer’s Disease" Mol. Psychiatry (2021) 10.1038/s41380-021-01249-0

[6]

NIA‐AA Research Framework: Toward a biological definition of Alzheimer's disease

Clifford R. Jack, David A. Bennett, Kaj Blennow et al.

Alzheimer's & Dementia 2018 10.1016/j.jalz.2018.02.018

[7]

Yu, W., Lei, B., Shen, Y., Wang, S., Liu, Y., Feng, Z., Hu, Y., and Ng, M.K. (2021). Morphological feature visualization of Alzheimer’s disease via Multidirectional Perception GAN. arXiv.

[8]

Mecocci "The impact of aging in dementia: It is time to refocus attention on the main risk factor of dementia" Ageing Res. Rev. (2021) 10.1016/j.arr.2020.101210

[9]

Liu "Alzheimer’s Disease Neuroimaging Initiative Classification of Alzheimer’s Disease by Combination of Convolutional and Recurrent Neural Networks Using FDG-PET Images" Front. Neuroinform. (2018) 10.3389/fninf.2018.00035

[10]

Nesteruk "Combined use of biochemical and volumetric biomarkers to assess the risk of conversion of mild cognitive impairment to Alzheimer’s disease" Folia Neuropathol. (2016) 10.5114/fn.2016.64815

[11]

Caminiti "FDG-PET and CSF biomarker accuracy in prediction of conversion to different dementias in a large multicentre MCI cohort" NeuroImage Clin. (2018) 10.1016/j.nicl.2018.01.019

[12]

Nuvoli "Qualitative and Quantitative Analyses of Brain 18Fluoro-Deoxy-Glucose Positron Emission Tomography in Primary Progressive Aphasia" Dement. Geriatr. Cogn. Disord. (2020) 10.1159/000504938

[13]

Liu "Inter-modality relationship constrained multi-modality multi-task feature selection for Alzheimer’s Disease and mild cognitive impairment identification" NeuroImage (2014) 10.1016/j.neuroimage.2013.09.015

[14]

Lu, S., Xia, Y., Cai, T.W., and Feng, D.D. (2015, January 25–29). Semi-supervised manifold learning with affinity regularization for Alzheimer’s disease identification using positron emission tomography imaging. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.

[15]

Deep Learning in Medical Image Analysis

Dinggang Shen, Guorong Wu, Heung-Il Suk

Annual Review of Biomedical Engineering 2017 10.1146/annurev-bioeng-071516-044442

[16]

Shaffer "Predicting Cognitive Decline in Subjects at Risk for Alzheimer Disease by Using Combined Cerebrospinal Fluid, MR Imaging, and PET Biomarkers" Radiology (2013) 10.1148/radiol.12120010

[17]

Massa "Added value of semiquantitative analysis of brain FDG-PET for the differentiation between MCI-Lewy bodies and MCI due to Alzheimer’s disease" Eur. J. Nucl. Med. Mol. Imaging (2022) 10.1007/s00259-021-05568-w

[18]

Lei "Diagnosis of early Alzheimer’s disease based on dynamic high order networks" Brain Imaging Behav. (2021) 10.1007/s11682-019-00255-9

[19]

Zuo, Q., Lei, B., Shen, Y., Liu, Y., Feng, Z., and Wang, S. (2021). Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer’s Disease Prediction. arXiv. 10.1007/978-3-030-88010-1_40

[20]

Hu, S., Yu, W., Chen, Z., and Wang, S. (2020, January 11–14). Medical Image Reconstruction Using Generative Adversarial Network for Alzheimer Disease Assessment with Class-Imbalance Problem. Proceedings of the 2020 IEEE 6th International Conference on Computer and Communications (ICCC), Chengdu, China. 10.1109/iccc51575.2020.9344912

[21]

Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis

Ravi Aggarwal, Viknesh Sounderajah, Guy Martin et al.

npj Digital Medicine 2021 10.1038/s41746-021-00438-z

[22]

Varoquaux "Machine learning for medical imaging: Methodological failures and recommendations for the future" NPJ Digit. Med. (2022) 10.1038/s41746-022-00592-y

[23]

Transparency of deep neural networks for medical image analysis: A review of interpretability methods

Zohaib Salahuddin, Henry C. Woodruff, Avishek Chatterjee et al.

Computers in Biology and Medicine 10.1016/j.compbiomed.2021.105111

[24]

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Mark Sandler, Andrew Howard, Menglong Zhu et al.

2018 IEEE/CVF Conference on Computer Vision and Pa... 10.1109/cvpr.2018.00474

[25]

Deep Residual Learning for Image Recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.90

[26]

Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.

[27]

Li "3-D CNN-Based Multichannel Contrastive Learning for Alzheimer’s Disease Automatic Diagnosis" IEEE Trans. Instrum. Meas. (2022) 10.1109/tim.2022.3218574

[28]

Zhu "Dual Attention Multi-Instance Deep Learning for Alzheimer’s Disease Diagnosis With Structural MRI" IEEE Trans. Med. Imaging (2021) 10.1109/tmi.2021.3077079

[29]

Acquarelli, J., van Laarhoven, T., Postma, G.J., Jansen, J.J., Rijpma, A., van Asten, S., Heerschap, A., Buydens, L.M.C., and Marchiori, E. (2022). Convolutional neural networks to predict brain tumor grades and Alzheimer’s disease with MR spectroscopic imaging data. PLoS ONE, 17. 10.1371/journal.pone.0268881

[30]

Kang, W., Lin, L., Zhang, B., Shen, X., and Wu, S. (2021). Multi-model and multi-slice ensemble learning architecture based on 2D convolutional neural networks for Alzheimer’s disease diagnosis. Comput. Biol. Med., 136. 10.1016/j.compbiomed.2021.104678

[31]

Park "Deep learning application for the classification of Alzheimer’s disease using 18F-flortaucipir (AV-1451) tau positron emission tomography" Sci. Rep. (2023) 10.1038/s41598-023-35389-w

[32]

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv.

[33]

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Advances in Neural Information Processing Systems, Curran Associates, Inc.

[34]

Hoang "Vision transformers for the prediction of mild cognitive impairment to Alzheimer’s disease progression using mid-sagittal sMRI" Front. Aging Neurosci. (2023) 10.3389/fnagi.2023.1102869

[35]

Lyu, Y., Yu, X., Zhu, D., and Zhang, L. (July, January 29). Classification of Alzheimer’s Disease via Vision Transformer: Classification of Alzheimer’s Disease via Vision Transformer. Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments, Corfu, Greece.

[36]

Sarraf, S., Sarraf, A., DeSouza, D.D., Anderson, J.A.E., and Kabia, M. (2023). The Alzheimer’s Disease Neuroimaging Initiative OViTAD: Optimized Vision Transformer to Predict Various Stages of Alzheimer’s Disease Using Resting-State fMRI and Structural MRI Data. Brain Sci., 13. 10.3390/brainsci13020260

[37]

Yin, Y., Jin, W., Bai, J., Liu, R., and Zhen, H. (2022, January 18–23). SMIL-DeiT:Multiple Instance Learning and Self-supervised Vision Transformer network for Early Alzheimer’s disease classification. Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy. 10.1109/ijcnn55064.2022.9892524

[38]

Zhu, J., Tan, Y., Lin, R., Miao, J., Fan, X., Zhu, Y., Liang, P., Gong, J., and He, H. (2022). Efficient self-attention mechanism and structural distilling model for Alzheimer’s disease diagnosis. Comput. Biol. Med., 147. 10.1016/j.compbiomed.2022.105737

[39]

Zhang, Z., and Khalvati, F. (2022). Introducing Vision Transformer for Alzheimer’s Disease classification task with 3D input. arXiv.

[40]

Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P.H., Buchatskaya, E., Doersch, C., Pires, B.A., Guo, Z.D., and Azar, M.G. (2020). Bootstrap your own latent: A new approach to self-supervised Learning. arXiv.

[41]

Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020). A Simple Framework for Contrastive Learning of Visual Representations. arXiv.

[42]

He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020). Momentum Contrast for Unsupervised Visual Representation Learning. arXiv. 10.1109/cvpr42600.2020.00975

[43]

Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021). Emerging Properties in Self-Supervised Vision Transformers. arXiv. 10.1109/iccv48922.2021.00951

[44]

Chen "Self-supervised learning for medical image analysis using image context restoration" Med. Image Anal. (2019) 10.1016/j.media.2019.101539

[45]

Tang, Y., Yang, D., Li, W., Roth, H., Landman, B., Xu, D., Nath, V., and Hatamizadeh, A. (2022). Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis. arXiv. 10.1109/cvpr52688.2022.02007

[46]

Dong "Glucose metabolism in the right middle temporal gyrus could be a potential biomarker for subjective cognitive decline: A study of a Han population" Alzheimer’s Res. Ther. (2021) 10.1186/s13195-021-00811-w

[47]

(2023, January 11). SPM—Statistical Parametric Mapping. Available online: https://www.fil.ion.ucl.ac.uk/spm/.

[48]

Lange "PETPVE12: An SPM toolbox for Partial Volume Effects correction in brain PET—Application to amyloid imaging with AV45-PET" NeuroImage (2017) 10.1016/j.neuroimage.2016.12.077

[49]

Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. (2021). Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. arXiv.

[50]

Qureshi, M.N.I., Min, B., Jo, H.J., and Lee, B. (2016). Multiclass Classification for the Differential Diagnosis on the ADHD Subtypes Using Recursive Feature Elimination and Hierarchical Extreme Learning Machine: Structural MRI Study. PLoS ONE, 11. 10.1371/journal.pone.0160697

Showing 50 of 63 references

Cited By

22

Multimodal Self-Supervised Learning for Early Alzheimer’s: Cross-Modal MRI–PET, Longitudinal Signals, and Site Invariance

Soumaya Belhaj Ali, Naglaa E. Ghannam · 2025

Diagnostics

Metrics

22

Citations

63

References

Details

Published: Oct 20, 2023
Vol/Issue: 10(10)
Pages: 1225
License: View

Authors

U

Uttam Khatri

Department of Information and Communication Engineering, Chosun University, 309 Pilmun-daero, Dong-gu, Gwangju 61452, Republic of Korea

G

Goo-Rak Kwon

Department of Information and Communication Engineering, Chosun University, 309 Pilmun-daero, Dong-gu, Gwangju 61452, Republic of Korea

Funding

National Research Foundation of Korea (NRF) Award: NRF-2021R1I1A3050703

Industry-University-Research Institute platform cooperation R&D Award: NRF-2021R1I1A3050703

Cite This Article

Uttam Khatri, Goo-Rak Kwon (2023). Explainable Vision Transformer with Self-Supervised Learning to Predict Alzheimer’s Disease Progression Using 18F-FDG PET. Bioengineering, 10(10), 1225. https://doi.org/10.3390/bioengineering10101225

Explainable Vision Transformer with Self-Supervised Learning to Predict Alzheimer’s Disease Progression Using 18F-FDG PET

You May Also Like