Recognizing and detecting COVID-19 in chest X-ray images using constrained multi-view spectral clustering

In this study, matrices are denoted in bold uppercase, while vectors are represented in bold lowercase. Let \(\textbf{X}^{(v)}\) represent the data matrix of view v, with \(v = 1, \ldots , V\). \(\textbf{X}^{(v)}\) is equivalent to \((\textbf{x}_1^{(v)}, \textbf{x}_2^{(v)}, \ldots , \textbf{x}_n^{(v)}) \in {\mathbb {R}}^{n \times d^{(v)}}\), where n is the total number of samples, and \(d_v\) is the dimensionality of the data in each view v. Our objective is to cluster the data into K clusters. Additionally, \(\textbf{x}_i^{(v)}\) signifies the i-th sample within the matrix \(\textbf{X}^{(v)}\). The trace of a matrix \(\textbf{M}\) is denoted by \(\textit{Tr}(\textbf{M})\), and the transpose is represented as \(\textbf{M}^T\). \(M_{ij}\) refers to the element in the i-th row and j-th column of the matrix \(\textbf{M}\). The Frobenius norm of this matrix is expressed as \(\Vert \textbf{M}\Vert _{2}\), and the \(l_2\)-norm of a vector \(\textbf{m}\) is given by \( \Vert \textbf{m}\Vert _{2}\). The primary matrices utilized in our work include:

The similarity matrix of each view denoted as \(\textbf{S}_v\), with the corresponding spectral projection matrix and Laplacian matrix represented by \(\textbf{P}_v\) and \(\textbf{L}_v\), respectively. The diagonal matrix is symbolized by \(\textbf{D}\), and the identity matrix is represented as \(\textbf{I}\). The cluster index matrix (non-embedding matrix) is denoted as \(\textbf{H}\). \({{\textbf {1}}}\) signifies a column vector in which all elements are set to one. The balance parameters employed in this article are denoted as \(\alpha \) and \(\lambda \).

The notations utilized in this paper are detailed in Table 1.

Nowadays, various multiview clustering methods have shown promising performance in different domains. In this paper, various multiview clustering methods are used to detect COVID-19 cases using chest X-ray datasets. In this section, the main categories of multiview clustering methods are explained. There are four main categories of multiview learning methods: (1) spectral clustering, (2) graph-based methods, (3) matrix factorization, and (4) subspace clustering methods

The goal of spectral clustering methods [13, 14] is to project data into a space where they are linearly separable. First, a similarity matrix is created between the data points for each view. Then, a unified similarity matrix is created by merging the different similarity matrices. Then, a spectral projection matrix corresponding to this similarity matrix is computed, whose rows lead to the final clustering results using the k-means method. The co-training method, as proposed in [3], is a widely recognized technique for multi-view spectral clustering. In this method, the same set of instances is clustered across multiple views. It achieves this by using the clustering result from the first view to adjust the affinity matrix of the second view, making it more similar to the first view. Co-regularized spectral clustering [4] is another prominent method that adaptively combines multiple similarity matrices from various views to achieve more accurate clustering results.

There are other multi-view clustering methods that take into account the importance of each view by assigning weights to each view, as outlined in references such as [2, 15]. While these approaches offer a practical solution by providing a common framework to combine different graphs, they do so at the expense of introducing extra weighting factors for each view. These weighting parameters can increase the computational complexity of the proposed methods.

To address this limitation, several methods have incorporated automatic weight learning, eliminating the need for manually set hyperparameters. Notable examples include [16‐20], where the weight for each view is calculated automatically.

Recently, spectral clustering has gained prominence in clustering with multiple views [21‐23]. Spectral-based methods create a similarity matrix across all views and generate the spectral projection matrix with K connected components from V graphs. Notably, the authors of [24] introduced the “Adaptively Weighted Procrustes” (AWP) method, a spectral-based clustering variant that utilizes spectral rotation to learn the cluster indicator matrix. Compared to previous graph-based approaches, AWP is characterized by lower computational complexity and higher precision. Furthermore, there is the Multi-View Subspace-based Clustering (MVSC) algorithm introduced by the authors in [5‐7]. These methods aim to generate the most coherent representation matrix of the data in a low-dimensional space.

In the work presented in [25], the authors introduced the Multi-View Learning with Adaptive Neighbors (MLAN) algorithm, which can simultaneously learn the graph structure and perform the clustering step. However, these methods have a significant drawback in terms of computational cost due to matrix inversion and eigenvalue decomposition.

Another approach presented in [26] employs two weighting systems: one for each view and the other for the features within each view. This method selects both the best view and the most informative features for each view. The rationale behind this is that feature selection can enhance the quality of clustering. This algorithm is particularly useful when dealing with high-dimensional features, as it performs feature selection and multi-view clustering concurrently.

The approach detailed in [27] can concurrently handle three tasks: (1) learning the similarity matrix, (2) creating the unified graph matrix, and (3) making the final clustering assignment. It utilizes an innovative multi-view fusion technique that automatically assigns weights to the graphs from each view to construct the unified graph matrix. Additionally, this method enforces a rank constraint on the Laplacian matrix to guarantee the presence of exactly K clusters. In addition, the researchers in [22] introduced a comprehensive framework for a multi-view spectral clustering approach that provides various learned graphs from each view, the unified fused graph, and the spectral clustering matrix simultaneously. Furthermore, [16] presents two automatic weighted clustering algorithms specifically designed for multiple views.

To reduce the computation time of graph-based methods, matrix factorization methods [12, 28] have become popular approaches for multi-view clustering. The basic idea of these methods is to convert the initial matrix into a product of two smaller matrices, which allows them to handle large datasets. Moreover, in [28], the authors present a method for multi-view clustering by consensus graph learning and non-negative embedding (MVCGE). This method benefits from kernelized graph learning methods and matrix factorization methods. The kernel is used to project the data into a space where it can be linearly separated. Thus, it can represent the similarity between the data points and account for the nonlinearity of the data. Moreover, the nonnegative matrix factorization matrix calculated by this method is introduced to directly obtain the final clustering result. This method is also able to automatically compute robust similarity and spectral projection matrices as well as the weights of each view. The authors of [29] develop a new method called one-step multi-view spectral clustering with cluster-label correlation graph. This method represents a significant innovation over existing multiview clustering methods. In addition to maintaining the advantages of methods for factorizing nonnegative matrices, this approach introduces an innovation inspired by semi-supervised learning. An additional graph is created to represent the similarity of the predicted labels. This graph is called the cluster label graph and is used in addition to the graphs associated with the data space.

A recent method called Dual Shared-Specific Multi-view Subspace Clustering (DSS-MSC) was introduced in [30]. DSS-MSC employs a dual learning model to simultaneously explore the characteristics of each view in a low-dimensional space. This approach aims to leverage the valuable and precise information from each view, while also considering the relationships among the shared information across different views. Additionally, the authors of [31] propose an innovative approach for learning a unified consistent graph. They achieve this by jointly computing the self-expressive coefficients and affinity matrix derived from different kernels. This unified graph is then used for the final clustering process. Two comprehensive surveys on the subject of multiview clustering can be found in [32, 33]. These surveys offer an extensive overview of multiview clustering methods, encompassing both generative and discriminative approaches.

The “Nonnegative Embedding and Spectral Embedding method” (NESE) is presented in [12]. NESE takes a distinctive approach by concurrently estimating the nonnegative embedding and spectral embedding matrices. This method aims to produce clustering results directly, obviating the necessity for supplementary clustering steps or additional parameters. The authors in [12] introduce a novel objective function for computing a unified nonnegative embedding matrix \(\textbf{H}\). This objective function draws inspiration from symmetric nonnegative matrix factorization and the relaxed continuous Ncut. The primary objective function of NESE is:In this method, \(\textbf{S}_v\) represents the similarity matrix for the respective view v, \(\textbf{P}_v\) denotes the spectral projection matrix, and \(\textbf{H}\) serves as the unified nonnegative embedding matrix used for clustering assignments (where each row corresponds to a sample). Notably, this approach removes the requirement for supplementary parameters or additional clustering steps, like k-means, which can be notably influenced by the selection of an initial configuration. The authors employ an iterative optimization technique to compute the results of their approach, which encompass the spectral projection matrices and the unified nonnegative embedding matrix. This iterative process ensures the accuracy of the method’s results.

In this section, we will outline our approach to optimizing the objective function defined in (7). To update the matrices \(\textbf{H}\) and \(\textbf{P}_v\), we employ an alternating minimization approach. This approach entails updating one of these two matrices while keeping the other fixed, and this process is reiterated until convergence is achieved.

Initially, we set the parameters \(\lambda \) and \(\alpha \) to zero and subsequently solve the resultant minimization problem, which yields the results akin to the NESE method. Consequently, the matrix \(\textbf{H}\) estimated by NESE acts as the initial matrix for our optimization. We also implement the same methodologies described in [35] for computing the matrices \(\textbf{S}_v\) for each view v and initializing \(\textbf{P}_v\).

Update \(\textbf{P}_v\): Fixing \(\textbf{H}\), \(w_v\), and \(\delta _v\), the objective function of OCNE will be equivalent to:Given that \(\textbf{P}_v^T, \textbf{P}_v = \textbf{I}\), this problem is the well-known orthogonal Procrustes problem, and its solution can be obtained using the singular value decomposition of \(\textbf{S}_v^T ,\textbf{H}\). Let \(\textbf{O}\Sigma \textbf{Q}^T = \text {SVD}(\textbf{S}_v^T ,\textbf{H})\). The solution to equation (8) is then given by:Update H :

If we fix \(\textbf{P}_v\), \(w_v\), and \(\delta _v\), we calculate the derivative of the function in (7) w.r.t. \(\textbf{H}\):Knowing that any real matrix \(\textbf{T}\) can be written as the difference of two nonnegative matrices, i.e., \(\textbf{T}= \textbf{T}^{+} - \textbf{T}^{-}\) where \(\textbf{T}^{+} = \frac{1}{2} \left( | \textbf{T}| + \textbf{T}\right) \) and \(\textbf{T}^{-} = \frac{1}{2} \left( | \textbf{T}| - \textbf{T}\right) \). Suppose that \(\textbf{N}_v=\,\textbf{S}_v \textbf{P}_v\,=\,\textbf{N}_v^+ -\, \textbf{N}_v^-\,,\;\)

and \(\textbf{L}_v\,=\,\textbf{L}_v^+ -\, \textbf{L}_v^-\,\).

After some algebraic manipulations, the gradient matrix will be equivalent to:

\( \frac{\partial f}{\partial \textbf{H}} = 2\, (\,\varvec{\Delta ^-} \, -\, \varvec{\Delta ^+})\) where:

\(\varvec{\Delta ^-} \,= \,\sum _{v=1}^V \,\delta _v\,\textbf{H}\,+ \,\sum _{v=1}^V \,\delta _v\,\textbf{N}_v^-\,+\,\lambda \,\sum _{v=1}^V \,w_v \,\textbf{L}_v^+\,\textbf{H}\,+ \,2 \, \alpha \,\textbf{H}\, \textbf{H}^T \, \textbf{H}\, \).

\(\varvec{\Delta ^+}\,= \,\sum _{v=1}^V \,\delta _v\,\textbf{N}_v^+\,+\,\lambda \,\sum _{v=1}^V \,w_v \,\textbf{L}_v^-\,\textbf{H}\,+ \,2 \, \alpha \,\textbf{H}\, \).

According to the nonnegative embedding matrix \(\textbf{H}\), it is updated by using the gradient descent algorithm. A step is given by:The learning parameter of the above equation \( \mu _{ij}\) is set to \(\frac{1}{2\,\Delta _{ij}^-}\,H_{ij}\). Therefore, the matrix \(\textbf{H}\) can be updated as follows:Update \(w_v\) and \(\delta _v\): The weights are updated using Eqs. (6) and (5), respectively, after all the mentioned matrices have been updated.

A summarized procedure of our OCNE method can be found in Table 2.

Five image datasets were utilized to assess the effectiveness of our approach: MSRCV1, Caltech101-7, MNIST-10000, NUS, and COVIDx (Table 3). The MNIST-10000 and COVIDx datasets are relatively large for graph-based multiview clustering approaches. The COVIDx dataset comprises 13,892 chest X-ray images categorized into three classes: COVID-19, normal, and pneumonia. Although this dataset is commonly employed for supervised classification, we used it to evaluate the proposed unsupervised method. For each X-ray image, three different deep CNNs provided three image descriptors: ResNet50, ResNet101 [41], and DenseNet169 [42], trained on the ImageNet dataset. The dimensions of these descriptors are 2048, 2048, and 1664, respectively.

Our method was compared with several other approaches, including:The evaluation compared the performance of these methods on the datasets mentioned earlier.

With respect to the dataset NUS, we compare the OCNE method with the following competing methods:For all these methods, we directly reproduce the best experimental results from the corresponding published paper [12]. The optimization procedure in our approach involves two hyperparameters: \(\alpha \) and \(\lambda \). We set \(\alpha \) to \(10^6\) to enforce the orthogonality constraint. The parameter \(\lambda \) is varied in the range from 10 to \(10^8\) in our experiments. To evaluate the performance of our method, we use four metrics: clustering accuracy (ACC), normalized mutual information (NMI), purity indicator, and adjusted rand index (ARI). These metrics are defined in [21]. Higher values of these metrics indicate better performance, meaning that the resulting clusters are more similar to the real clusters.

In this section, we provide a detailed evaluation of the experimental results. The best-performing results are indicated in bold, and for methods requiring an additional clustering step such as K-means, the standard deviation of the indicator parameters over multiple trials is shown in parentheses.

Our approach is compared to several state-of-the-art methods, including SC Fused, MVCSK, NESE, S-MVSC, and CI-GMVC. We assess the performance of our method not only on the COVIDx dataset but also on other datasets, including NUS, MSRCv1, Caltech101-7, and MNIST-10000. The results are presented in Tables 4, 5, and 6. Best results are shown in bold.

From these tables, it is evident that OCNE outperforms most of the competing methods across the five datasets. Furthermore, OCNE demonstrates superior performance on large datasets such as MNIST-10000 and COVIDx, highlighting its effectiveness on datasets of varying sizes.

It’s worth noting that this work represents a significant step in applying clustering algorithms to detect COVID-19 cases in lung images. While the clustering results on this dataset may not be perfect, this study lays the groundwork for the use of unsupervised and semi-supervised learning algorithms in such scenarios. This approach enables the utilization of datasets with missing labels, which is particularly valuable in the context of medical image analysis.

Regarding the convergence of OCNE, Fig. 4 displays the variation of the objective function concerning the number of iterations, focusing on the MSRCv1 dataset. The figure clearly demonstrates that our method converges rapidly, typically before reaching 10 iterations. This fast convergence is a valuable characteristic of OCNE, as it leads to efficient and effective clustering results.

In this section, we explore the sensitivity of the \(\lambda \) parameter. Figure 3 presents the values of the ACC and NMI indicators when varying the \(\lambda \) parameter from 10 to \(10^{+8}\) for the MSRCv1 dataset. The \(\alpha \) parameter was fixed at \(10^{+6}\).

From this figure, it is evident that OCNE achieves its best performance when the value of the \(\lambda \) parameter is set to \(10^{+5}\). This result highlights the importance of parameter tuning for optimizing the performance of the OCNE method.

In this section, we delve into a comprehensive analysis of the performance of our approach across various scenarios using different-sized subsets of the COVIDx dataset, aiming to gain deeper insights into its adaptability and robustness. The exploration of these scenarios serves the dual purpose of assessing the scalability of our method and shedding light on its effectiveness under different data distributions. This is done by considering different sizes of this dataset and testing them with the SC, NESE and OCNE approaches.

First, a small sample of 2468 instances is extracted from the large dataset, consisting of 1000 instances of the Pneumonia class, 468 instances of the COVID-19 class, and 1000 instances of the Normal class. This sample is referred to as COVIDx-2468. This sample, while relatively limited in size, provides an initial glimpse into our method’s capabilities in scenarios where data resources may be constrained. It challenges our approach to effectively distinguish COVID-19 cases from other conditions in a context where class imbalance is prevalent. The second sample contains 6468 instances, divided into 3000 instances of the Pneumonia class, 468 instances of the COVID-19 class, and 3000 instances of the Normal class. This sample is referred to as COVIDx-6468. This dataset size better aligns with real-world medical datasets and poses a different set of challenges related to class balance and diversity in the data. The final sample is the large COVIDx dataset, which consists of 13,892 instances, as described previously. This extensive dataset encompasses a wide range of cases, providing a comprehensive evaluation of our approach’s performance across a diverse set of instances.

First, the famous “spectral clustering” method SC [38] is tested on these different samples, adapted for multi-view analysis. To assess the individual contributions of each view, we apply SC separately to each view, by considering the affinity matrix of the corresponding view, which can be obtained using the same method as in [12]. We refer to this method as SC1, SC2, and SC3 corresponding to \(View_1\), \(View_2\), and \(View_3\), respectively. Also, SC is applied to all views together by considering the affinity matrix as the average of all affinity matrices for all views. Table 7 summarizes the corresponding experimental results. By dissecting the results across different views and considering their collective impact, we gain an understanding of the strengths and limitations of our approach. This analysis serves to underscore the significance of multi-view clustering techniques and their potential for enhancing the diagnostic capabilities of our methodology.

Once we have a particular predicted clustering, we look for the best mapping between the three clusters obtained and the three classes. We can then compute the confusion matrices from which we can derive three evaluation metrics to measure the quality of the clustering results: Precision (indicates the percentage of all items predicted as positive that were actually positive), Recall (indicates the percentage of all true positive samples that the model was able to detect), and Selectivity Indicator (indicates the percentage of all true negative samples that the model was able to detect). All of these metrics have been widely used in previous studies, and their definitions can be found in [39]. In Table 7, we report these indicators for the three different classes: (1) pneumonia, (2) COVID-19, and (3) normal, which are referred to as class 1, 2, and 3, respectively. These evaluation metrics offer a comprehensive assessment of the clustering quality for each class. Importantly, a higher value for each metric corresponds to superior clustering results, emphasizing the significance of achieving higher precision, recall, and selectivity in our methodology. By reporting these metrics for each class in Table 7, we provide a detailed performance analysis that underscores the diagnostic potential of our approach. The focus on these well-established metrics ensures the robustness and reliability of our evaluation methodology, aligning with best practices in medical diagnostic research. For each evaluation metric, the larger the values, the better the results.

Therefore, NESE and OCNE are also evaluated with different cases. First, these methods are applied to two views each (\(View_1\) & \(View_2\), denoted \(v_1\) & \(v_2\), \(View_1\) & \(View_3\), denoted \(v_1\) & \(v_3\), and \(View_2\) & \(View_3\), denoted \(v_2\) & \(v_3\)). Then all views are considered. The experimental results are shown in Table 8.

From Table 7 which presents the results of the classical spectral clustering method, we can clearly see for the COVIDx-2468 subset that View 2 is the most informative for detecting COVID-19 cases, as indicated by Class 2 in the table (see Precision and Recall indicators). However, for the COVIDx-6468 and COVIDx datasets, view 3 is the most informative view. The difference between the clustering results of the different classes can be interpreted as follows. Our dataset contains “468” COVID-19 samples. This number is small compared with the other two classes (pneumonia and normal) for all subsets considered. Therefore, our data are imbalanced, which may reduce the efficiency of clustering methods for COVID-19 case detection. This highlights the importance of addressing class imbalance in future research endeavors and underscores the need for more advanced techniques in handling such scenarios. Because COVID-19 disease has similar symptoms to pneumonia, the algorithm may be confused when clustering images corresponding to these two different classes because they may have similar and overlapping patterns. Table 7 also shows that the result of Multiview Spectral Clustering (SC) for all subsets is generally lower than the result obtained by the best single view. This result can be interpreted by the fact that there is one view that gives significantly worse results than the other two views, which distorts the overall result of the multiview method SC. Furthermore, the graph fusion was performed with a simple averaging of the view graphs. It is worth noting that the graph fusion in our methodology offers a glimpse into the potential for further refinement. In summary, the detailed analysis of our experimental results in Table 7 unravels the intricate dynamics of multi-view clustering in the context of COVID-19 diagnosis. These findings not only enhance our understanding of clustering performance but also pave the way for future research focused on refining multi-view methodologies for medical diagnostics. Finally, we can observe that the clustering performance of the methods NESE and OCNE decreases as the imbalance if the ground-truth classes increases.

Regarding the results obtained by applying the method NESE using to the three subsets of the COVIDx dataset, Table 8 shows that the combination of all views did not provide the best clustering results for the three subsets. For the NESE method, the clustering performance was relatively good for both the combination of \(View_1\) & \(View_2\) and the combination of \(View_2\) & \(View_3\).

From Table 8, it can be seen that the proposed OCNE method gives the best clustering results when we consider \(View_2\) & \(View_3\) for all three subsets. It can also be observed that the fusion of all views in both NESE and our proposed OCNE method helps to improve the recall indicator associated with COVID-19 (class 2). The recall associated with OCNE was also higher than that obtained with NESE. We emphasize that a high recall indicator is required for this type of problem. In conclusion, the performance of both clustering methods NESE and OCNE decreases with increasing imbalance of ground-truth classes.

The results of our experiments have promising practical implications for the field of medical diagnostics, especially for the detection and diagnosis of COVID-19 cases. Although our method is currently still in the research phase, its potential applications in clinical practice are remarkable. If further developed and rigorously validated, our approach could become a valuable complementary tool for medical professionals. In settings where access to extensive labeled COVID-19 data is difficult, such as in resource-limited regions, the ability of our method to work with unlabeled data demonstrates its resource efficiency. Moreover, the transparent and interpretable results of our approach are indispensable in the medical field, as they allow physicians to make informed decisions based on the clustering results. However, we recognize that our work is a step toward practical implementation. Future clinical trials and collaboration with healthcare institutions will be instrumental in validating the effectiveness and safety of our method in practice and ultimately establishing it in the healthcare system.