Time-series visual explainability for Alzheimer's disease progression detection for smart healthcare

Artificial intelligence (AI)-based diagnostic systems provide less error-prone and safer support to clinicians, enhancing the medical decision-making process. This study presents a smart and reliable healthcare framework for detecting Alzheimer's disease (AD) progression. Early detection of AD before the onset of clinical symptoms is the most crucial step in starting timely treatment. To predict the conversion of cognitively normal patients to those with AD, three-dimensional 3D magnetic resonance imaging (MRI) whole-brain neuroimaging methods have been extensively studied. However, depending on the 3D volume, this method is computationally expensive. To solve this problem, we used an approximate rank pooling method originally designed for video action recognition with a 3D MRI volume to obtain a compressed representation of multiple two-dimensional (2D) MRI slices. This study proposes a hybrid multimodal CNN-BiLSTM deep model for AD progression detection, in which the resulting dynamic 2D images are fused with cognitive features. Moreover, a novel explainable AI approach is proposed to provide visual explanations using the resulting longitudinal 2D dynamic images. Temporal explanations were provided by visualizing the affected brain regions captured using longitudinal 2D MRIs. By utilizing a sample of 1,692 subjects with multimodal data from the Alzheimer's Disease Neuroimaging Initiative dataset, our method was assessed using a 10-fold cross-validation process. The model achieved an area under the receiver operating characteristics curve (AUC) of 94% using longitudinal 2D three-time-step dynamic image data. The fusion of 2D dynamic images with cognitive features enhanced the performance by 2% in terms of the AUC. Patients who gradually develop AD, show changes in various brain regions. For such patients, our system highlights the critical role of the hippocampus, medial amygdala, caudal hippocampus, and lateral amygdala at the initial time steps. In the late stages of AD, the system detects abnormalities in extra brain regions such as the medial temporal gyrus, superior temporal gyrus, fusiform gyrus, and caudal hippocampus; indicating that patients have completely progressed to AD.