1 Introduction

AD is a kind of irreversible dementia and a prevalent neurodegenerative disorder that mainly affects older adults [5]. According to the World Health Organization (WHO), over 50 million people globally are living with dementia, and Alzheimer’s disease is the most common form, accounting for 60-70% of cases. According to statistics released by the International Alzheimer’s Association, the global number of AD patients is projected to reach 76 million by 2030 and 135 million by 2050 [24]. AD typically impairs an individual’s cognitive, behavioral, and emotional abilities. With the increasing life expectancy of modern society, the aging population has intensified, leading to a rise in the occurrence of AD and placing a considerable burden on communities [47]. The precise etiology of AD remains uncertain, and no efficacious medications exist for its cure, underscoring the significance of early diagnosis and medical intervention to decelerate the progression of the disease.

The brain’s anatomical structure in AD patients presents diffuse encephalatrophy, with the primary morphological changes characterized by narrowed gyrus and widened sulcus [43]. The most prominent atrophy locations are concentrated in the hippocampus, temporal, parietal, and prefrontal areas. Magnetic Resonance Imaging (MRI) can clearly find the lesions of encephalatrophy. MRI is a neuroimaging method that utilizes nuclear magnetic resonance phenomena to capture electromagnetic signals emitted by the human body in order to reconstruct its internal structure. Compared to other imaging approaches such as Computed Tomography (CT) and Positron Emission Tomography (PET), MRI stands as the most widely used noninvasive neuroimaging technology as it poses no radioactive tracers or harmful gamma rays [48]. Magnetization Prepared Rapid Gradient Echo (MP-RAGE), a form of T1-weighted structural MRI (sMRI), effectively portrays brain structures such as gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) and is proficient in detecting encephalatrophy in clinical settings. Figure 1 displays the MP-RAGE scans of AD patients with encephalatrophy.

Fig. 1
figure 1

The MP-RAGE scans of AD patients with encephalatrophy. (Subject ID: (a) 005_S_0221-22, (b) 006_S_0547-38)

Full size image
Fig. 2
figure 2

The framework of our proposed dual-branch Alzheimer’s disease diagnostic model based on distinguishing atrophic patch localization

Full size image

Advancements in computer vision technology have led researchers to explore feature extraction from neuroimaging to achieve computer-aided diagnosis (CAD) [1, 54, 63]. Due to its wide application in clinically diagnosing AD, MRI is a neuroimaging modality upon which many methods rely to construct CAD models. Besides MRI, several methods utilize different modalities of neuroimaging to build CAD models, such as fMRI (functional MRI) [49], PET (Positron Emission Tomography) [55], DTI (Diffusion Tensor Imaging) [39], EEG (Electroencephalogram) [40], and others. These diverse neuroimaging modalities present information from various perspectives of patients, aiding in a multifaceted understanding of disease diagnosis. The use of MRI for AD diagnosis is driven by its high spatial resolution, non-invasive nature, and capability to provide detailed brain images. MRI enables early detection of structural changes, quantitative analysis of brain regions, and identification of comorbid conditions without using ionizing radiation. These advantages make MRI a crucial tool for accurate and comprehensive assessment of AD, thereby facilitating timely intervention and effective monitoring of disease progression. Hence, our study proposed an MRI-based CAD model.

Early AD diagnosis through CAD can assist clinicians in identifying potential lesions and providing diagnostic recommendations, thus improving diagnostic efficiency and reducing misdiagnoses. Depending on the scale of input data for feature extraction, current computer-aided AD diagnosis methods can be separated into four classifications: subject-based methods [3, 8, 64], voxel-based methods [44, 62], ROI (region of interest)-based methods [23, 52], and patch-based methods [28, 42]. Subject-based methods typically use 2D slices [3, 64] or 3D images [8] of patients to extract features. The 2D slice methods often focus on core cerebral regions for feature extraction while neglecting other parts and the contextual correlation features between brain tissues. On the other hand, the 3D methods heavily rely on model design quality for diagnostic performance, and their interpretability is insufficient. Voxel-based methods usually extract voxel values from registered images as features through statistical analysis [4], directly measuring the density and volume of GM, WM, and CSF components [41]. However, voxel-based methods indiscriminately treat each voxel value, resulting in high feature dimensionality and complexity in extraction and calculations. ROI-based methods utilize the cerebral anatomical structure to extract features from specific brain regions related to diseases for diagnosis [56]. These methods often require prior knowledge or experienced clinicians’ input and lack consideration for cross-regional areas, frequently leading to suboptimal results. In the early stage of AD, structural changes often appear in local areas, such as the hippocampus and amygdala [47], rather than the whole brain or isolated voxels. Patch-based methods extract patches related to AD diagnosis from the entire image and extract features from the patches to capture cerebral disease-related patterns. These patch-based methods construct patch-level feature extraction models, rendering auxiliary diagnostic tasks more straightforward and efficient [34]. Simultaneously, patch-based methods overcome the limitations of anatomical regions and can identify features of cross-tissue lesion areas. However, the existing patch-based methods often depend on specific prior knowledge and lack adaptability to different patients. For different diagnostic tasks (e.g., AD vs. CN and sMCI vs. pMCI), existing methods frequently extract patches from the same locations [32] and build diagnostic networks to achieve CAD. Meanwhile, when constructing diagnostic networks, current studies often extract features from each patch separately [29, 42], failing to mine the contextual relationships between patches and resulting in the subpar diagnostic performance of patch-based methods. Consequently, patch-based methods are promising but face two main challenges:

  1. (1)

    One challenge confronting patch-based feature extraction methods is how to precisely locate distinguishing atrophic patches for various diagnostic tasks while pruning diagnostically irrelevant patches.

  2. (2)

    Developing a more accurate diagnostic model that simultaneously extracts and fuses inter- and intra-features of patches is another challenge in constructing a patch-based computer-aided AD diagnostic model.

To address these challenges, we propose a novel dual-branch Alzheimer’s disease diagnostic model based on distinguishing atrophic patch localization. The schematic of our proposed method is illustrated in Fig. 2. Initially, we introduce a data-driven method for distinguishing atrophic patch localization. This involves defining a distinguishing index for the patches and subsequently proposing a discontinuity voxel-based dynamic voxel wrapping algorithm to calculate the distinguishing index of various patches in the brain for disease diagnosis. Furthermore, we propose a selection and pruning algorithm to achieve more precise localization of distinguishing atrophic patches. To simultaneously extract inter- and intra-features from these distinguishing atrophic patches, we present a dual-branch diagnostic network structure. The intra-feature extraction branch employs a multi-layer parallel network structure designed to extract features from each patch individually and fuse the feature vectors. The inter-feature extraction branch integrates the multiple distinguishing atrophic patches by defining a spatial context mixing matrix, which is then input into a Convolutional Neural Network (CNN) feature extraction network to obtain mutual information between patches. Finally, the features from both branches are fused to produce the final diagnostic result. This approach ensures that both local and global context information is effectively utilized, enhancing the accuracy and reliability of Alzheimer’s disease diagnosis. The main contributions of our proposed method are as follows:

  • A data-driven patch localization method is proposed to locate atrophic patches with high discrimination for disease diagnosis. We present a Discontinuity Voxel-based Dynamic Voxel Wrapping (DV\(^2\)W) algorithm to calculate the various patches’ Distinguishing Index (DI). And we propose a patch selection and pruning algorithm DAPL (Distinguishing Atrophic Patch Localization) based on DI and Spatial Contact Ratio (SCR) to achieve distinguishing atrophic patch localization.

  • A dual-branch diagnostic network that integrates inter-features and intra-features of patches is proposed to diagnose diseases accurately. The intra-feature extraction branch constructs a parallel network structure to synchronously extract and fuse features from each patch. The inter-feature extraction branch integrates the patches and creates a Spatial Context Mixing (SCM) matrix, which is then utilized to construct a CNN network. The fusion of two components yields the final diagnostic result.

  • Experiments on real patient datasets demonstrate our diagnostic model’s superiority in two diagnostic tasks compared to existing state-of-the-art methods. The visualization of distinguishing atrophic patch locations can intuitively display areas that require attention for disease diagnosis and assist clinicians in quickly discovering and identifying potential lesion locations to enhance diagnostic efficiency.

The remaining sections are structured as follows: Section 2 provides a summary of current related works across four feature extraction strategies. Section 3 describes our proposed method and the dataset used. In Section 4, we present the experimental settings and results of our approach. Section 5 discusses the advancements and implications of our proposed method. Section 6 offers the conclusion.

2 Related work

In this section, we introduce feature extraction methods for four categories involved in computer-aided AD diagnosis: subject-based methods, voxel-based methods, ROI-based methods, and patch-based methods.

Subject-based methods usually utilize 3D neuroimages or 2D slices as input to construct a model for feature extraction and disease diagnosis. Cobbinah et al. [8] developed a diagnostic model with a soft attention mechanism and residual blocks, aligning different protocols of 3D MRI scans with the proposed adversarial autoencoder as input. Feng et al. [11] built a 3D CNN architecture for extracting feature representations from neuroimages. Subsequently, a fully stacked bidirectional long short-term memory (LSTM) network was suggested to capture spatially relevant deep-level information in the feature map, enhancing diagnostic performance. Wang et al. [57] input 3D MRI into an ensemble of DenseNet with five branches. Although the 3D methods can use all the information of neuroimages, the diagnostic network constructed usually has many parameters and high computational complexity. Meanwhile, the diagnostic performance is often unsatisfactory due to unavoidable image noise. Zhang et al. [64] input the 2D coronal gray matter slices into a feature extraction model with channel attention. Liu et al. [33] assembled a diagnostic network cascading CNN and recurrent neural network (RNN). CNN extracts feature from slices, while a bidirectional gate recurrent unit is employed to extract spatial information between different slices. The methods based on 2D slices usually select partial brain images as input data, often leading to information loss. In addition, using 2D slices often makes it challenging to mine spatial information, resulting in poor diagnostic performance.

Subject-based methods eliminate the need for prior knowledge or predefined lesion areas and data preprocessing. Deep learning advancements reduce manual feature design dependence, enhancing diagnostic accuracy through global and contextual information capture. However, performance relies heavily on network design and training optimization, facing challenges with complex medical images for clinical understanding.

Voxel-based methods typically require the registration of neuroimages into a public space. The processed images extract meaningful voxel values as features through statistical analysis methods. This method also measures the volume and density of GM, WM, and CSF. Zhang et al. [62] use MRI data and a voxel-based morphometry method (VBM) [4] to diagnose AD. Richhariya et al. [44] proposed a voxel-based feature selection and classification method - universum SVM-RFE. This method can achieve better accuracy by reducing the number of features. In addition, Takao et al. [53] also utilized VBM to resolve the dependability of longitudinal changes in GM volume. Voxel-based feature extraction methods usually deal with very high voxel dimensions, which makes feature extraction and selection more complex and carries the risk of overfitting. At the same time, neglecting the consideration of the correlation between voxels and the importance of voxels can make the diagnostic model unable to achieve satisfactory performance.

Voxel-based methods simplify MRI analysis but struggle with high-dimensional features, particularly with complex imaging modalities like fMRI and large datasets. They may overlook regional information crucial for disease discrimination, impacting diagnostic performance.

ROI-based methods need to utilize the prior cerebral morphological features to divide the anatomical structure of the human brain into different regions [56]. These approaches usually focus on localized tissue regions and specific parts of the brain rather than the whole brain. Suk et al. [52] constructed feature sequences based on the extracted features from different ROIs, then input them into the deep autoencoder model to obtain potential features and combined them with a hidden Markov model to build a state transition matrix to predict the state of the disease. Wang et al. [58] used the brain tissue extraction tool to obtain hippocampus slices and constructed a CNN model for feature extraction. Li et al. [26] extracted 93 ROI-based volume features from multimodal neuroimages and utilized principal component analysis to diagnose disease. Cui et al. [9] diagnosed the disease by analyzing ROIs’ shape and volume features, such as the hippocampus. Although the ROI-based methods have good interpretability, they require much prior knowledge. In addition, morphological abnormalities caused by neurological diseases do not constantly occur in pre-defined ROIs. They may also exist in locations across ROIs, which can make the performance of ROI-based methods unstable and affect diagnostic performance.

Fig. 3
figure 3

The demographic characteristics of all the subjects used in our study

Full size image

ROI-based methods in CAD offer interpretability for clinicians, reducing computational complexity compared to voxel-based methods. However, they rely on prior knowledge for ROI selection, impacting diagnostic performance. Neurological disorders may affect multiple ROIs, potentially destabilizing CAD model performance.

The early changes in the brain structure of AD often occur in parts or multiple regions of the brain. Patch-based methods divide the brain into patches with a specified size and extract features from these patches. These methods avoid the enormous computational burden of processing large 3D images and can more accurately obtain disease-related features [50, 66]. Lian et al. [28] proposed a one-stage patch selection and disease diagnostic model. A hierarchical fully convolutional neural network is proposed for feature extraction of multiple patches, and patches that impact diagnosis are selected through network pruning algorithms. Pan et al. [42] divided MRI into patches of the same size and isolated each patch into a constructed CNN feature extraction network to obtain features. The features were fused and input into the classifier for disease diagnosis. Suk et al. [51] cropped the images into multiple patches and input them into a Deep Boltzmann Machine for feature fusion. Finally, the Support Vector Machine was used for diagnosis. Liu et al. [32] constructed a multimodal cascaded network using MRI and PET(Positron Emission Computed Tomography) data. They input the patches corresponding to the positions of the two modalities into the network for feature extraction and disease diagnosis. In addition, [34] extracted features from the patches determined using prior knowledge, fused the subjects’ demographic information, and jointly input the fused features into the classifier for disease diagnosis.

Although patch-based methods have many advantages, such as being more targeted in processing objects and having lower computational complexity compared to subject-based and voxel-based methods, and having the same patch size compared to ROI-based methods, making it easier to construct feature extraction networks for different patches, there are still some challenges. The first challenge is that almost all patch-based methods do not select different patches for different diagnostic tasks in AD, and some models do not even perform feature selection. Instead, they directly divide the images into patches and input them into feature extraction networks, which affects the diagnosis performance. We propose a data-driven distinguishing atrophic patch localization method to select different patches with distinguishing for different diagnostic tasks. The second challenge is that existing patch-based methods often extract features independently of each patch for fusion diagnosis, lacking in mining the contextual relationships between patches. To this end, we propose a dual-branch diagnostic network that simultaneously mines inter-features and intra-features and fuses them into the classifier for diagnosis.

Fig. 4
figure 4

The workflow of our proposed diagnostic model

Full size image

3 Materials and methods

3.1 Dataset and data pre-processing

In this study, we investigate T1-weighted structural MR images of 1461 subjects, including 218 AD, 328 pMCI, 448 sMCI, and 467 CN. Mild cognitive impairment (MCI) is a condition situated between cognitive normal (CN) and AD and serves as a significant risk factor for dementia [16]. Clinically, MCI patients are classified into two subtypes based on whether they will progress into AD within 36 months of their MCI diagnosis: stable MCI (sMCI) and progressive MCI (pMCI). The demographic characteristics of these subjects are illustrated in Fig. 3. All these neuroimages are acquired from Alzheimer’s Disease Neuroimaging Initiative [36]. The slice thickness of all MRIs equals 1.2 mm. The sagittal scanning images are selected because they can clearly display the structure of the sulcus and gyrus. Some necessary preprocessing for these images is done. We utilize tool [17] for intensity correction, FreeSurfer [12] for skull stripping, and image registration to the MNI152 template [21].

3.2 Methodology

In clinical, physicians usually focus on multiple local locations of the brain when diagnosing Alzheimer’s disease through neuroimaging and combine these regions to determine the diagnostic results comprehensively. In order to design a more accurate AD computer-aided diagnostic model, we simulate the process of disease diagnosis by clinicians. Firstly, we propose a data-driven distinguishing atrophic patch localization method, which can locate patches with a high impact on disease diagnosis. Furthermore, we propose a dual-branch disease diagnostic model that extracts intra-features and inter-features from the selected patches and fuses the extracted features to obtain diagnostic results. The workflow of our proposed AD diagnostic model is shown in Fig. 4. The following two subsections provide a detailed introduction to the two main parts of our model.

3.2.1 Distinguishing atrophic patch localization

When diagnosing a specific disease, clinicians usually prioritize observing the pathological changes at specific locations in the neuroimaging based on experience [14]. We propose a method for CAD to identify the locations of these lesions on where they need to be focused. First, we divide the registered 3D MRI into N patches of the same size, then define Distinguishing Index (DI) for each patch. Let \(I_{X\times Y\times Z}\) represent a 3D MRI; \(l\left( i\right) =\left( l_x\left( i\right) ,l_y\left( i\right) ,l_z\left( i\right) \right) \) denotes a voxel in I, and \(i\in [1,N]\). \(v\left( l\left( i\right) \right) \) is the voxel value mapping function. \(P\left( l\left( i\right) \right) \) represents a patch with \(l\left( i\right) \) as the first voxel and \(l\times l\times l\) as the size. DI represents the ability of different patches to distinguish between different diagnostic tasks. We propose a Discontinuity Voxel-based Dynamic Voxel Wrapping (DV\(^2\)W) algorithm to calculate the DI of each patch \(P\left( l\left( i\right) \right) \) with specified diagnostic tasks. Meanwhile, we defined Spatial Contact Ratio (SCR) between different patches. Finally, based on DI and SCR, we propose a Distinguishing Atrophic Patch Localization (DAPL) algorithm to filter out M patches with influential distinguishing significance for diagnostic tasks. To calculate DI, we need to calculate the difference between two patches obtained from two classifications. Unlike calculating the differences in 3D matrices, it is easier to calculate the differences (or distances) between sequence data. Therefore, we consider transforming the patches of three-dimensional matrix structures into sequential data. Here we define the h-order outstretching image sequence.

Definition 1

(h-Order Outstretching Image Sequence) Given image \(I_{a\times b\times c}\), cut the image into multiple slices along the h direction, each slice being a two-dimensional matrix. Expand each matrix into a sequence and concatenate the sequences of each slice expansion. The flattened sequence is called the h-order outstretching sequence of the image I, represented as

$$\begin{aligned} Seq\left( I,h\right) =\left\{ seq\left( I,h\right) _1,seq\left( I,h\right) _2,\ldots ,seq\left( I,h\right) _k\right\} \end{aligned}$$
(1)

where \(k=a\times b\times c\) is the number of voxels in the image I. For a 3D MRI image, \(h\in H=\left\{ axial,\ sagittal,\ coronal\right\} \).

For two image sequences, calculating the degree of difference can be seen as calculating similarity. The standard calculation of similarity between sequences is based on Euclidean distance measurement and requires that the points in the two sequences correspond one-to-one. However, for images from two different types of diseases, voxel points cannot achieve bijection due to differences in the location and grade of brain atrophy, as well as errors in the registration process. Using traditional similarity calculation methods will not be able to effectively calculate the similarity (difference) between two image sequences. To achieve this, we propose a DV\(^2\)W algorithm based on discontinuity voxels to calculate the difference between image sequences. Before introducing the algorithm, we will provide some relevant definitions.

Definition 2

(Matching Function) Given two image sequences \(Seq\left( I_1,h\right) \) and \(Seq\left( I_2,h\right) \), The matching function \(\mathcal {F}_{Seq\left( I_1,h\right) ,Seq\left( I_2,h\right) }\left( f,g\right) \) contains two mappings, with the mapping rules being:

$$\begin{aligned} {\left\{ \begin{array}{ll} f:Seq\left( I_1,h\right) \rightarrow Seq\left( I_2,h\right) /\Lambda \\ g:Seq\left( I_2,h\right) \rightarrow Seq\left( I_1,h\right) /\Lambda \end{array}\right. } \end{aligned}$$
(2)

where \(\Lambda \) represents null.

Generally, the matching function satisfies monotonicity, that is:

  1. (1)

    If \(f\left( seq\left( I_1,h\right) _i\right) \rightarrow \ seq\left( I_2,h\right) _j\), then for all \(i^\prime >i\), \(f\left( seq\left( I_1,h\right) _{i^\prime }\right) \rightarrow seq\left( I_2,h\right) _{j^\prime }/\Lambda \), where \(j^\prime \) satisfies \(j^\prime >j\).

  2. (2)

    If \(g\left( seq\left( I_2,h\right) _i\right) \rightarrow \ seq\left( I_1,h\right) _j\), then for all \(i^\prime >i\), \(g\left( seq\left( I_2,h\right) _{i^\prime }\right) \rightarrow seq\left( I_1,h\right) _{j^\prime }/\Lambda \), where \(j^\prime \) satisfies \(j^\prime >j\).

Definition 3

(Discontinuity Voxel) Given two sequences \(Seq\left( I_1,h\right) \) and \(Seq\left( I_2,h\right) \), and the matching function \(\mathcal {F}_{Seq\left( I_1,h\right) ,Seq\left( I_2,h\right) }\left( f,g\right) \). If we have \(f\left( seq\left( I_1,h\right) _i\right) \rightarrow \Lambda \) or \(g\left( seq\left( I_2,h\right) _j\right) \rightarrow \Lambda \), then the voxel points \(seq\left( I_1,h\right) _i\) and \(seq\left( I_2,h\right) _j\) are called discontinuity voxels, abbreviated as DV.

Definition 4

(Voxel Distance) Let \(seq\left( I_1,h\right) _i\) and \(seq\left( I_2,h\right. \) \(\left. \right) _j\) be two voxel points of two image sequences, the voxel distance between \(seq\left( I_1,h\right) _i\) and \(seq\left( I_2,h\right) _j\) are defined as

$$\begin{aligned}&DIS\left( seq\left( I_1,h\right) _i,seq\left( I_2,h\right) _j\right) \nonumber \\&=\frac{\Vert v\left( seq\left( I_1,h\right) _i\right) -v\left( seq\left( I_2,h\right) _j\right) \Vert ^2_{2}}{\delta ^2+\Vert v\left( seq\left( I_1,h\right) _i\right) -v\left( seq\left( I_2,h\right) _j\right) \Vert ^2_{2}} \end{aligned}$$
(3)

where \(\delta \) is the critical threshold.

Definition 5

(Distinguishing Distance) Given two sequences \(Seq\left( I_1,h\right) \) and \(Seq\left( I_2,h\right) \), find an optimal matching function \(\mathcal {F}_{Seq\left( I_1,h\right) ,Seq\left( I_2,h\right) }\left( f,g\right) \) such that

$$\begin{aligned} {\begin{matrix} DS& \left( Seq\left( I_1,h\right) ,Seq\left( I_2,h\right) \right) = \\ & \min _\mathcal {F} (\sum _{\begin{matrix}seq\left( I_1,h\right) _i\in Seq\left( I_1,h\right) \\ f\left( seq\left( I_1,h\right) _i\right) \nrightarrow \Lambda \\ \end{matrix}}DIS\left( seq\left( I_1,h\right) _i,\right. \\ & \left. f\left( seq\left( I_1,h\right) _i\right) \right) +\sum _{\begin{matrix}seq\left( I_1,h\right) _i\in Seq\left( I_1,h\right) \\ f\left( seq\left( I_1,h\right) _i\right) \rightarrow \Lambda \\ \end{matrix}}\tau \\ & +\sum _{\begin{matrix}seq\left( I_2,h\right) _i\in Seq\left( I_2,h\right) \\ g\left( seq\left( I_2,h\right) _i\right) \nrightarrow \Lambda \\ \end{matrix}}DIS\left( seq\left( I_2,h\right) _i,\right. \\ & \left. g\left( seq\left( I_2,h\right) _i\right) \right) +\sum _{\begin{matrix}seq\left( I_2,h\right) _i\in Seq\left( I_2,h\right) \\ g\left( seq\left( I_2,h\right) _i\right) \rightarrow \Lambda \\ \end{matrix}}\tau ) \end{matrix}} \end{aligned}$$
(4)

then \(DS\left( Seq\left( I_1,h\right) ,Seq\left( I_2,h\right) \right) \) is called the distinguishing distance between images \(I_1\) and \(I_2\). \(\tau \) represents the maximum of voxel distance.

Definition 6

(Distinguishing Index) Given the image data of two classifications \(CL_1\) and \(CL_2\), the distinguishing index of patch \(P\left( l\left( i\right) \right) \) in the task of diagnosing \(CL_1\) and \(CL_2\) is defined as:

$$\begin{aligned} DI\left( l\left( i\right) \right)&\!=\!\frac{1}{S_\alpha {\cdot S}_\beta }\sum _{h\in H}\sum _{\alpha =1}^{S_\alpha }\sum _{\beta =1}^{S_\beta }\!DS\!\left( Seq\left( {P\left( l\left( i\right) \right) }_\alpha ^{{CL}_1},h\right) \!,\!\right. \nonumber \\&\left. Seq\left( {P\left( l\left( i\right) \right) }_\beta ^{{CL}_2},h\right) \right) \end{aligned}$$
(5)

where \(S_\alpha \) and \(S_\beta \) are the number of images in \(CL_1\) and \(CL_2\), respectively.

\(DI\left( l\left( i\right) \right) \) describes the diagnostic ability of a patch at a specified location \(l\left( i\right) \) in diagnosing \(CL_1\) and \(CL_2\) tasks, which is the degree of encephalatrophy at that location. The key to calculating DI is to calculate distinguishing distance, meaning we need to know the distinguishing distance between any two patches clearly.

Let’s consider the outstretching image sequences \(Seq\left( A,\right. \) \(\left. h\right) =\left\{ \rho _1,\rho _2,\ldots ,\rho _m\right\} \) and \(Seq\left( B,h\right) =\left\{ \theta _1,\theta _2,\ldots ,\theta _m\right\} \) of two image patches. When calculating distinguishing distance, each voxel in the two sequences has two possible states, continuity voxel, and discontinuity voxel. A voxel is defined as a continuity voxel state, abbreviated as CV when it can discover a corresponding voxel in another sequence. On the contrary, if the current voxel cannot find a corresponding voxel in another sequence, it is in a DV state. We use \(DS\left( \rho _i,\theta _j\right) \) to represent the distinguishing distance of the two sequences \(\left\{ \rho _1,\rho _2,\ldots ,\rho _i\right\} \) and \(\left\{ \theta _1,\theta _2,\ldots ,\theta _j\right\} \). The two voxels \(\rho _i\) and \(\theta _j\) have four possible states, that is (1) \(\rho _i\) is DV, and \(\theta _j\) is DV; (2) \(\rho _i\) is DV, and \(\theta _j\) is CV; (3) \(\rho _i\) is CV, and \(\theta _j\) is DV; (4) \(\rho _i\) is CV, and \(\theta _j\) is CV. So we can calculate \(DS\left( \rho _i,\theta _j\right) \) by the following equation:

$$\begin{aligned} DS\left( \rho _i,\theta _j\right)&=\min \left\{ {DS}_{DD}\left( \rho _i,\theta _j\right) ,{DS}_{DC}\left( \rho _i,\theta _j\right) ,\right. \nonumber \\&\left. {DS}_{CD}\left( \rho _i,\theta _j\right) , {DS}_{CC}\left( \rho _i,\theta _j\right) \right\} \end{aligned}$$
(6)

where \({DS}_{DD},{DS}_{DC},{DS}_{CD},{DS}_{CC}\) are the four state matrices of the current voxel pair \(\left( \rho _i,\theta _j\right) \). Below, we will provide specific calculation methods for these four matrices.

In order to better match voxels between two sequences, we propose a dynamic strategy. The cumulative distance of the current voxel pair \(\left( \rho _i,\theta _j\right) \) comes from \(\left( \rho _{i-1},\theta _j\right) \), \(\left( \rho _i,\theta _{j-1}\right) \), and \(\left( \rho _{i-1},\theta _{j-1}\right) \); similarly, the state matrices also come from these three parts. Let’s take \({DS}_{DD}\left( \rho _i,\theta _j\right) \) as an example to illustrate the calculation process. If the cumulative distance comes from \(\left( \rho _{i-1},\theta _j\right) \), since \(\theta _j\) is DV, only the state of \(\rho _{i-1}\) needs to be concerned, which can be separated into two situations: \({DS}_{DD}\left( \rho _{i-1},\theta _j\right) \) and \({DS}_{CD}\left( \rho _{i-1},\theta _j\right) \). If the cumulative distance comes from \(\left( \rho _i,\theta _{j-1}\right) \), since \(\rho _i\) is DV, only the state of \(\theta _{j-1}\) needs to be concerned, which can be separated into two situations: \({DS}_{DD}\left( \rho _i,\theta _{j-1}\right) \) and \({DS}_{DC}\left( \rho _i,\theta _{j-1}\right) \). If the cumulative distance comes from \(\left( \rho _{i-1},\theta _{j-1}\right) \), we only need to consider \(DS\left( \rho _{i-1},\theta _{j-1}\right) \). For the current step \(\left( \rho _i,\theta _j\right) \), the cumulative distance varies depending on the state to which it belongs. For the state (1), (2) and (3), we set the cumulative distance to \(\tau \). When the state is (4), the cumulative distance equals to \(DIS\left( \rho _i,\theta _j\right) \). The solving procedure of \({DS}_{DC}\left( \rho _i,\theta _j\right) \),\({DS}_{CD}\left( \rho _i,\theta _j\right) \), and \({DS}_{CC}\left( \rho _i,\theta _j\right) \) can be received similarly.

We calculate distinguishing distance biunique for the patches at location \(l\left( i\right) \) in two categories \(CL_1\) and \(CL_2\) of data samples and calculate the weighted average to obtain DI. The calculation methodology of DI is systematized into Algorithm 1.

Algorithm 1
figure f

Discontinuity voxel-based dynamic voxel wrapping algorithm.

Full size image

Algorithm 1 provides the calculation procedure of DI. Lines 5-17 describe the calculation method of distinguishing distance. By calculating distinguishing distance pairwise between patches at location \(l\left( i\right) \) from two categories \(CL_1\) and \(CL_2\) and performing weighted averaging, DI can be obtained.

We require the selection of patches to have a strong distinguishing ability on the one hand and to have smaller overlapping parts between patches on the other hand. An enormous overlap can increase redundancy by extracting similar features by the diagnostic network. Therefore, we propose the concept of spatial contact ratio to measure the degree of overlap between two patches.

Definition 7

(Spatial Contact Ratio) Let \(l\left( i\right) =\left( l_x\left( i\right) ,l_y\left( i\right) ,l_z\left( i\right) \right) \) and \(l\left( j\right) =\left( l_x\left( j\right) ,l_y\left( j\right) ,l_z\left( j\right) \right) \) be two voxel locations of MRI, SCR is defined as the proportion of voxels that overlap two patches \(P\left( l\left( i\right) \right) \) and \(P\left( l\left( j\right) \right) \) with the size \(l\times l\times l\), and the calculation equation is as follows:

$$\begin{aligned} SCR\left( P\left( l\left( i\right) \right) ,P\left( l\left( j\right) \right) \right) =\frac{\prod _{u\in \left\{ x,y,z\right\} } ReLU\left( 0,l-\left| l_u\left( i\right) -l_u\left( j\right) \right| \right) }{l\times l\times l} \end{aligned}$$
(7)

where ReLU [37] is the linear rectification function.

By calculating SCR, we can obtain the capacity of voxel overlap between any two patches. When selecting atrophic patches, to avoid neural networks learning too many repetitive features, we need to control the repetition of the selected patches.

The DI allows different patches to distinguish diagnostic tasks, and the SCR can effectively measure voxel overlap. Based on DI and SCR, we propose a distinguishing atrophic patch localization (DAPL) algorithm to filter and prune N partitioned patches. We first arrange these N patches in DI order from high to low. Then we start filtering from the first patch, and the patch that enters the selection queue must ensure that the SCR value of all patches in the current queue is more petite than \(\varepsilon \). Otherwise, the patch will not be retained in the selection queue. Traverse all patches, and the top-M patches remaining in the selection queue are the distinguishing atrophic patches we have located. The above calculation process is summarized as Algorithm 2.

Algorithm 2
figure g

Distinguishing atrophic patch localization algorithm.

Full size image

The PS set output by Algorithm 2 is the distinguishing atrophic patches selected in the classification task \(CL_1\) vs. \(CL_2\). The DAPL algorithm achieves precise localization and screening of areas that significantly impact diagnosis, which helps to better extract features from neural images and perform high-precision disease diagnosis. We provide a data-driven approach to select different patches personalized for different diagnostic tasks. Furthermore, it is also convenient to adjust the number and size of the selected patches, showing our proposed method’s progressiveness.

Fig. 5
figure 5

The network structure of the intra-feature extraction branch DFB

Full size image
Table 1 The network architecture details of the single patch feature extraction network in DFB
Full size table

3.2.2 Dual-branch AD diagnostic model

We selected the M patches with the most discriminative ability through the above distinguishing atrophic patch localization algorithm. Next, we will construct a neural network to extract features from these patches. Existing patch-based AD diagnostic models often extract features from each patch in isolation and fuse the extracted features. Considering the mutual information between different patches, we propose a dual-branch patch feature extraction and disease diagnosis network. The network consists of two branches, the intra-feature extraction branch (DFB) and the inter-feature extraction branch (IFB). DFB is a typical network branch that extracts the internal features of each patch and fuses them. IFB integrates different patches into an input matrix at the voxel level through the defined Spatial Context Mixing matrix (SCM), and constructs a feature extraction network to extract features from the SCM, obtaining the interactive features between patches. Two branches perform feature-level fusion and connect classifiers to diagnose different diseases. The following will introduce these two branches separately.

(1) The intra-feature extraction branch DFB

In the DFB branch, we built a shallow convolutional neural network for each patch to extract features. The network structure is shown in Fig. 5. The network architecture details of the single a patch feature extraction network in DFB are shown in Table 1. Each subnetwork comprises six convolutional layers, two max-pooling layers, and one global average pooling (GAP) layer. We connected the batch normalization (BN) [19] layer behind each convolutional layer and added the ReLU [37] function. Adding the BN layer can accelerate our model’s learning speed, improve network learning stability, and avoid gradient disappearance. The final GAP layer of each subnetwork can significantly reduce the number of neurons, thereby reducing the number of parameters and avoiding overfitting. The features extracted from each patch are concatenated and implicitly represented through a fully connected (FC) layer, resulting in intra-patch features. M subnetworks are distributed in parallel, and during the training process, rapid training is achieved through parameter sharing.

Fig. 6
figure 6

A schematic diagram of constructing SCM

Full size image
Fig. 7
figure 7

The network structure of the inter-feature extraction branch IFB

Full size image

(2) The inter-feature extraction branch IFB

The DFB branch can effectively extract features within the selected patches, but lacks the extraction of mutual features between various patches. In order to better implement mutual feature extraction, we propose a voxel-wise network structure that extracts mutual features between patches by extracting features from the constructed SCM. Firstly, we give the definition of SCM.

Definition 8

(Spatial Context Mixing matrix) Let \(PS=\left\{ P_1,P_2,\ldots ,P_M\right\} \) be a selected patch set, and the size of each patch is \(l\times l\times l\). We flatten each patch into a vector of \(l^3\), and stack M expanded one-dimensional vectors into a matrix of size \(\sqrt{M}\times \sqrt{M}\times l^3\). This three-dimensional matrix is called the spatial context mixing matrix, abbreviated as SCM.

SCM can be regarded as screening the original MR images, selecting locations with significant differentiation significance for the disease, and reassembling them into an input matrix. The calculation process of SCM is illustrated in Fig. 6. The different patches are flattened and arranged one by one to form a new matrix, so that the feature representation of the potential Mutual information between patches can be well obtained when the matrix is feature extracted, which makes up the defect of the single feature extraction method for each patch.

Next, we perform feature extraction on SCM to obtain mutual features between patches. In the IFB branch, SCM serves as the input of the network. Due to the size of the input matrix, we constructed a deep-level feature extraction network based on CNN. The network contains eight convolutional layers, five max-pooling layers, and two fully connected layers. Similarly, the BN layer and ReLU function are connected after each convolutional layer. The network structure of the IFB branch is shown in Fig. 7. The network architecture details of the IFB are shown in Table 2. We can obtain more profound features correlated to the mutual information between patches through feature extraction and compression of SCM layer by layer.

Table 2 The network architecture details of the IFB
Full size table
Fig. 8
figure 8

Parameter selection experimental results. (a) The diagnostic accuracy of the DBDN model varies with different patch sizes, and (b) the diagnostic accuracy of the DBDN model varies when selecting different numbers of patches

Full size image

(3) Dual branch feature fusion

Our proposed dual-branch AD diagnostic network combines the DFB and IFB branches. Specifically, we first concatenate the features extracted from two branches, then design an artificial neural network composed of two fully connected layers, and finally obtain the classification results through softmax. The design of ANN can be used to learn the joint representation of more complex intra-patch and inter-patch features. By continuously optimizing the model through training, it is possible to achieve deep extraction of disease diagnostic features and provide accurate diagnosis of diseases.

4 Experiments and results

4.1 Experimental settings

We conducted experiments on a computer equipped with an NVIDIA GeForce RTX 2080Ti 11G GPU to implemented our proposed model. Our deep learning framework of choice was Pytorch, and we used SGD [45] as an optimizer. Our method was implemented in two diagnostic tasks: AD vs. CN task and sMCI vs. pMCI task. To evaluate the performance of our proposed model, we selected four indicators: accuracy (ACC), sensitivity (SEN), specificity (SPE), and area under the curve (AUC).

4.2 parameter selection

Our proposed DAPL algorithm offers flexibility by allowing for adjustable numbers and sizes of selected patches to adapt to various diagnostic tasks and data types. We conducted experiments to determine the optimal hyperparameters and selected a suitable range based on relevant research and our algorithm. The results, displayed in Fig. 8, show the impact of patch size and number on diagnostic performance. Figure 8(a) demonstrates that a patch size of \(25\times 25\times 25\) resulted in the highest diagnostic accuracy for two tasks, while Fig. 8(b) shows that selecting 36 patches achieved optimal performance due to the construction of the SCM matrix. Too many patches introduce unnecessary data, while too few patches result in incomplete information extraction. Therefore, we recommend a patch size of \(25\times 25\times 25\) and 36 patches for our method. Different hyperparameters can be set for various tasks, allowing for model reuse and good generalization ability.

Table 3 Ablation experiments in two diagnostic tasks
Full size table
Fig. 9
figure 9

Confusion matrices of three models DFB, IFB, and DBDN in two diagnostic tasks

Full size image

4.3 Performance of dual-branch AD diagnostic model

This section provides the experimental results of our proposed dual-branch AD diagnostic model. Firstly, we conducted ablation experiments on each branch in the model, and the results are displayed in Table 3 (Note: The bold value indicates the best result in that column.). The table demonstrates that the DBDN model combining the intra-feature extraction branch and inter-feature extraction branch has significantly improved diagnostic performance compared to DFB and IFB alone for diagnosis. Comparing IFB with DFB, we can find that the IFB method, which contains mutual information between patches, achieves better performance, which confirms the advantages of SCM and the necessity of IFB. Further, to more intuitively show the performance of different models in different diagnostic tasks, we give the confusion matrix of each model in two diagnostic tasks, as shown in Fig. 9. The confusion matrix indicates that the DBDN model integrating two branches can divide the test samples into the correct classification as far as possible. It has a lower error than using a single branch alone. The results of the ablation experiment verified that fusing the two branches can improve diagnostic performance on the one hand, and on the other hand, constructing SCM to extract mutual features between patches has better performance than fusing each patch after feature extraction in isolation.

We evaluated our proposed DBDN model against several patch-based approaches, and the results are presented in Table 4 (Note: The bold value indicates the best result in that column.). All methods in the table use MRI data from ADNI. As seen from the table, our accuracy has improved by 0.05 for AD diagnostic tasks compared with the best method in baselines. For MCI diagnostic tasks, our accuracy has improved by 0.109. The two diagnostic tasks are heavy. Almost all the evaluation indicators of our DBDN method are higher than the existing patch-based computer-aided diagnostic methods, indicating that our patch selection, dual-branch feature extraction, and diagnostic model are compelling.

Table 4 Performance comparison with other patch-based methods in two diagnostic tasks
Full size table
Fig. 10
figure 10

The loss function curves of two diagnostic tasks in training and validation

Full size image

To guarantee the success of our proposed DBDN model, we have contained a graph (Fig. 10) that displays the changes in training and validation loss curves for two tasks. Upon analyzing the chart, it is evident that the model started to converge after about thirty epochs and reached a plateau at around 50 epochs. This suggests that our training process was appropriately executed, without any overfitting or underfitting phenomenon, and the convergence process was accurate. As a result, the saved model is the most ideal diagnostic model.

We provide the ROC (Receiver Operating Characteristic) curves to further illustrate the performance of our model, shown as Fig. 11. The ROC curve is an essential tool for evaluating the effectiveness of classification models, especially in medical diagnostics. It displays the trade-off between the true positive rate and the false positive rate across various threshold settings. Our ROC curves demonstrate the high diagnostic accuracy of our DBDN model in the two diagnostic tasks, with the curves approaching the top-left corner, indicating excellent discrimination between the different classes. The high AUC values associated with the ROC curves underscore the model’s robustness and reliability in accurately diagnosing conditions based on neural imaging data.

Fig. 11
figure 11

ROC curves of our DBDN model in the two diagnostic tasks

Full size image

To validate the robustness and generalization ability of our model, we conducted 10-fold cross-validation. Specifically, the dataset was divided into ten equal parts. Each part was used as a validation set while the remaining nine parts were used as training sets. This process was repeated ten times to ensure that each data point was utilized for both training and validation. During the 10-fold cross-validation, we recorded ACC, SEN, SPE, and AUC values for each fold. These findings indicate that our model maintains stable performance across various data splits, showing no significant signs of overfitting. we plotted the distribution of these metrics across different folds, as shown in Fig. 12. The box plots illustrate that the performance metrics are closely clustered, with small standard deviations, which further confirms the robustness of the model. In addition, we recorded the training and inference times for the DBDN model. For each diagnostic task, training the model takes approximately 8 hours. Once the model is trained, the average inference time for predicting the condition of each patient is approximately 84 milliseconds.

Fig. 12
figure 12

The boxplots of the cross-validation results in the two diagnostic tasks

Full size image

4.4 Results of distinguishing atrophic patch localization

Here, we showcase the results of our proposed method for identifying atrophic patches. We divided the data into two diagnostic tasks: AD vs. CN and sMCI vs. pMCI. For each task, we selected 36 patches that contained the most atrophic tissues. To better show the locations of the patches identified by our DAPL algorithm, we visualized the top ten cerebral regions corresponding to the selected patches for both tasks. These visualizations are depicted in Figs. 13-14. Typically, the majority of atrophic areas were found in the Limbic system and temporal lobe. The Limbic system plays a crucial role in regulating mood, motivation, decision-making, and learning [14, 61]. If this system malfunctions, it can result in various psychiatric and neurological disorders. The temporal lobe is particularly important for encoding and retrieving memories [6], so damage to this region is a common symptom of early Alzheimer’s disease.

We analyzed the brain tissue information in 36 selected patches, as shown in Fig. 15. Our findings indicate that these patches are concentrated in various brain parts, including the limbic system [14], basal nuclei [59], temporal lobe [6], parietal lobe [14], frontal lobe [13], and occipital lobe [18]. The Hippocampus and surrounding Limbic system play a significant role in diagnosing cognitive diseases. Tissues in the temporal lobe also show good differentiation ability. This suggests that patients with cognitive impairment may experience atrophy in the Hippocampus, temporal lobe, and surrounding tissues. Furthermore, compared to patients with mild cognitive impairment (MCI), those with Alzheimer’s disease (AD) exhibit more extensive encephalatrophy coverage, with atrophic lesions in the basal nuclei, parietal lobe, frontal lobe, and occipital lobe.

Fig. 13
figure 13

The cerebral tissues of the selected patches by using DAPL algorithm in AD vs. CN diagnostic task

Full size image
Fig. 14
figure 14

The cerebral tissues of the selected patches by using DAPL algorithm in sMCI vs. pMCI diagnostic task

Full size image

5 Discussion

In this section, we initially compare our diagnostic model DBDN with several state-of-the-art previous AD diagnostic models. Following that, we quantitatively analyze the lesion area using the proposed DAPL algorithm and offer diagnostic recommendations for specific locations that require special attention.

5.1 Comparison with previous AD diagnostic model

Some computer-aided diagnostic models are already constructed for AD diagnosis in existing research. We have compiled the diagnostic results of state-of-the-art reported in the existing literature on AD and MCI diagnostic tasks, as shown in Table 5 (Note: The bold value indicates the best result in that column.). These methods include traditional feature engineering methods [20, 31, 60] and deep learning methods. All references use MRI neuroimages from the ADNI dataset. The table demonstrates that the method proposed in this paper performs more satisfactorily than most methods, whether in AD diagnostic tasks or MCI diagnostic tasks. Our model has achieved optimal performance owing to two factors. Our DAPL algorithm is developed to accurately identify disease-related locations for each diagnostic task, making it more precise. Additionally, our dual-branch feature extraction and diagnosis model effectively captures deep representations of features at various patch scales. It is worth noticing that although our method uses different amounts of data compared to other methods in the list, it can achieve optimal diagnostic performance regardless of whether it uses more data for training or models with fewer data. Therefore, it can be verified that our method is effective and advanced.

5.2 Qualitative analysis of pathological changes in patients

In Section 4.3, we used the DAPL algorithm to identify distinguishing patches and visualize their locations. Calculating the DI for different patches, we provide a list of potential lesion areas that should receive special attention in clinical practice, as shown in Table 6. The organizations listed in the table are named according to the AAL3V1 template [46], with the code column indicating the corresponding numbers in the template. The list is sorted based on the locations’ differentiation ability, with most being concentrated in the Limbic system, including the Hippocampus, Parahippocampal gyrus, and Amygdala. Other regions, such as the temporal lobe, basal nuclei, parietal lobe, frontal lobe, and occipital lobe, are also involved. For the two diagnostic tasks, the Hippocampus and surrounding areas are the tissues with the most substantial differentiation ability, consistent with the conclusion recorded in the literature [6]. For the temporal lobe, the Superior temporal gyrus has good discriminative ability in both diagnostic tasks. Compared to the Middle temporal gyrus and Inferior temporal gyrus, some frontal lobe areas exhibit more extraordinary discriminative ability in AD diagnostic tasks.

Table 5 Comparison with other state-of-the-art AD diagnostic models
Full size table
Table 6 Lists of cerebral regions of pathological changes that our algorithm recommends to focus on clinically
Full size table
Fig. 15
figure 15

The selected patches’ cerebral tissue counts statistics in two diagnostic tasks

Full size image

Meanwhile, the results show that when diagnosing AD, doctors should not only consider the Limbic system and temporal lobe, but also pay attention to the pathological changes in the patient’s occipital lobe. The table can also reflect the differences in the impact of different diseases on the brain. MCI patients often have lesions mainly concentrated in the limbic system and temporal lobe, affecting memory, cognition, thinking, and other functions. In contrast, AD significantly impacts the patient’s brain and a more comprehensive range of design.

In addition to facilitating computer-aided disease diagnosis, our results can provide clinical doctors with potential lesion areas to assist inexperienced doctors in identifying and treating lesions quickly. This impact can be profoundly positive, given the potential implications for early diagnosis and treatment for patients with neurodegenerative diseases.

5.3 Limitations and future Work

The proposed method, while demonstrating significant advancements in the automatic diagnosis of Alzheimer’s disease using neuroimaging data, is not without its limitations. One limitation is the method’s adaptability to varying input sizes of neuroimaging data. The current model is optimized for inputs of uniform dimensions. This lack of flexibility can potentially reduce the model’s effectiveness in diverse clinical settings. Another limitation concerns the separation of the patch selection and feature extraction processes. In our approach, patch selection is conducted independently of feature extraction. The quality and relevance of the selected patches directly impact the subsequent feature extraction and the overall accuracy of the disease diagnosis. If the patch selection algorithm does not perform well, it can hinder the model’s ability to extract meaningful features and make accurate predictions.

Looking ahead, our future work will address these limitations by developing a more integrated and adaptable approach. One primary direction is to design a unified model that combines patch selection and feature extraction into a single, cohesive process. This integrated approach aims to enhance the efficiency and accuracy of the diagnostic process by ensuring that the patches selected are inherently suitable for effective feature extraction, thereby improving the overall performance of the model. Additionally, we plan to develop a model capable of handling neuroimaging inputs of various scales. This model would be designed to process images acquired using different protocols, enhancing its adaptability and robustness in diverse clinical environments. Furthermore, we intend to explore the use of multimodal neuroimaging data to create a comprehensive diagnostic tool. Leveraging information from different imaging modalities can provide a more holistic view of patients’ condition, potentially leading to more accurate and insightful diagnosis.

6 Conclusion

In this study, a novel dual-branch Alzheimer’s disease diagnostic model based on distinguishing atrophic patch localization is developed. First, we propose a distinguishing atrophic patch localization algorithm by defining and calculating the distinguishing index and spatial contact ratio. When calculating DI, considering the difference in intensity between voxels with atrophic lesions and regular voxels, we propose a discontinuity voxel-based dynamic voxel wrapping algorithm to calculate the DI of a patch at a specified location. The DAPL algorithm can accurately select patches that significantly impact disease diagnosis, improving the efficiency of feature extraction in diagnostic networks and providing a reference for clinical doctors in locating patches. Through experiments, we selected 36 patches, which significantly reduced the computational complexity of the model and achieved efficient processing and analysis of image data. Then, we propose a dual-branch AD diagnostic model that includes an intra-feature extraction branch and an inter-feature extraction branch. The DFB branch obtains features within different patches and fuses them. The IFB branch extracts mutual information between patches through the defined spatial context mixing matrix and the constructed network structure. The fusion of the two parts improves the accuracy of diagnosis. Our method demonstrated optimal performance by comparing patch-based methods with various feature extraction strategies. At the same time, based on the analysis of the pathological site of the brain where the patch is located, we provide recommendations for diagnosing potential lesion areas for different diseases.

Structural MRI can display the structural lesions of the brain. In the future, we will consider more types of neuroimaging, such as functional MRI, Positron Emission Computed Tomography, and Diffusion Tensor Imaging, to explore more in-depth lesions related to brain function. At the same time, we are also considering constructing an imaging omics fusion diagnostic model that combines multiple neuroimages.