Normalized Mutual Information Research Articles

Spectral ensemble clustering with doubly stochastic co-association matrix

Although co-association (CA) matrix-based spectral ensemble clustering (SEC) has achieved significant success, it faces two persistent challenges: (1) CA matrix-based SEC may result in imbalanced cluster outcomes, and (2) SEC encounters difficulties due to the high time and space complexity resulting from the quadratic scaling of the CA matrix with the number of data points. To tackle these issues, we propose a doubly stochastic co-association (DSCA) matrix construction method. We have theoretically demonstrated that DSCA matrix-based SEC can achieve a more balanced cluster outcome compared to CA matrix-based SEC. Leveraging the DSCA matrix, we introduce two novel methods: Fast SEC (FSEC) and Approximate SEC (ASEC). FSEC efficiently solves the normalized cut on the DSCA matrix as it is symmetric and doubly stochastic. In ASEC, we employ random samples rather than the entire big dataset for ensemble clustering. After obtaining the ensemble clustering result of benchmark random sample, the approximate ensemble clustering result for all data points is derived using our newly proposed probability nearest neighbors (PNN) algorithm. Experimental results on both real-world and synthetic datasets are evaluated in terms of accuracy (ACC), normalized mutual information (NMI), purity, F1-score, and adjusted rand index (ARI) and confirm the scalability and effectiveness of the proposed FSEC and ASEC.

Optimizing Product Matching in E-Commerce with DOC2VEC: Leveraging Hierarchical Clustering Parameters Based on Product Titles

Information technology is pivotal in increasing efficiency and effectiveness in online retail, particularly in product matching. This research delves into the challenges associated with product matching in the e-commerce sector, addressing issues related to the diversity and ambiguity of product titles and the fast-paced introduction of new products to the market. As a solution, we implement a neural network-based approach. The main contribution of this research is the implementation and validation of the Doc2Vec method in the context of product matching for e-commerce products. Additionally, this study successfully identifies the optimal parameter combinations for Hierarchical Clustering, which has been tested and validated on 4,000 product title data points. The data for learning and evaluation comes from an online retail platform and includes 34,000 product names from various sectors. The research compares two Doc2Vec architectures for feature extraction from product titles and then integrates them with a Hierarchical Clustering approach to group similar products. The results indicate that the Doc2Vec model with the DBOW (Distributed Bag of Words) architecture yields a better average NMI (Normalized Mutual Information) Score than the DM (Distributed Memory) architecture.

Muscle Network Connectivity Study in Diabetic Peripheral Neuropathy Patients.

Diabetic peripheral neuropathy (DPN) is a prevalent complication of chronic diabetes mellitus and has a significant impact on quality of life. DPN typically manifests itself as a symmetrical, length-dependent sensorimotor polyneuropathy with severe effects on gait. Surface electromyography (sEMG) is a valuable low-cost tool for assessing muscle activation patterns and precise identification of abnormalities. For the present study, we used information theory methods, such as cross-correlation (CC), normalized mutual information (NMI), conditional granger causality (CG-Causality), and transfer entropy (TE), to evaluate muscle network connectivity in three population groups: 33 controls (healthy volunteers, CT), 10 diabetic patients with a low risk of DPN (LW), and 17 moderate/high risk patients (MH). The results obtained indicated significant alterations in the intermuscular coupling mechanisms due to diabetes and DPN, with the TE group showing the best performance in detecting differences. The data revealed a significant increase in information transfer and muscle connectivity in the LW group over the CT group, while the MH group obtained significantly lower values for these metrics than the other two groups. These findings highlight the sEMG coupling metrics' potential to reveal neuromuscular mechanisms that could aid the development of targeted rehabilitation strategies and help monitor DPN patients.

Hybrid Filtering and Probabilistic Techniques for Privacy-Preserving Community Detection in OSNs

Online Social Networks (OSNs) face the major challenge of protecting participant's privacy, due to the high dimensionality and volume of the data. In real-time social networks, where hundreds of personal details of people are shared every day, there remains a significant threat to privacy. Privacy preservation is challenging for a community detection problem due to the high computational complexity and memory requirements, especially in larger real-world OSN graphs. Although weighted nodes provide better results, as they allow capturing the frequencies of the values, the privacy preservation of sensitive attributes such as specific profiles becomes harder compared to these models. This problem can result in queries and subsequent learnings from social network profiles of specific individuals, which may be of personal, political or otherwise concern. Online social networks (OSNs) grapple with significant privacy challenges due to the extensive dimensions and vast quantities of data involved. This research fills the void in current privacy-preserving community detection methodologies, which face problems in computational complexity and memory usage in large-scale OSNs. The proposed framework seeks to bolster privacy preservation through a comprehensive multi-step process. Data filtering uses a blended data filter system to remove outliers and irrelevant data, thus enhancing the quality of the input data. Density-Oriented Clustering phase employs a density-oriented clustering model to identify communities, with each cluster representing a distinct community. Privacy Preservation component introduces a new privacy preservation technique for sensitive OSN attributes, surpassing existing k-anonymization methods. The developed density-based social network community detection model and its novel privacy-preserving scheme are evaluated using the datasets Yelp, Football, Zachary and Dolphin from the SNAP dataset. Experimental results on these datasets embed a comprehensive evaluation based on the order of each node and the graph-based networks, where each node is laden with weights as proximity values, indicating the semantic proximity between communities and individuals. The proposed framework employs the normalized mutual information (NMI), modularity (Q), Rand index and runtime measurements to demonstrate widespread advantages in the multi-dimensional functional space, including greater accuracy, cluster compatibility, and computational tractability over the existing prominent traditional models for OSNs.

Integrating topology and content equally in non-negative matrix factorization for community detection

Non-negative matrix factorization (NMF) is widely used in analyzing the community structure of complex networks. However, most of the current related efforts only focus on the network structural topology, while neglecting the node content. Although there have been efforts to integrate topology and content, inefficient strategies balancing their contributions can also lead to unsatisfactory results. In this work, we treat topology and content equally and propose two novel NMF-based methods to finely balance topology and content to improve the performance of community detection. We first propose a double encoder–decoder-based NMF (DEDNMF) method that employs two NMF-based encoder–decoders to integrate topology and content adaptively. Experimental results show that DEDNMF outperforms the baseline methods with average improvements of 50.69% and 46.93% in terms of normalized mutual information (NMI) and adjusted rand index (ARI) on all experimental networks, respectively. Moreover, we propose a double-decoder-based NMF (DDNMF) method that uses only two decoders for both topology and content, imposing sparsity constraints on NMF. Experimental results show that DDNMF outperforms the baseline methods with average improvements of 72.11% and 77.00% in terms of NMI and ARI on all experimental networks. Experimental results also show that the proposed two methods have good convergence and robustness.

Explicit unsupervised feature selection based on structured graph and locally linear embedding

Numerous redundant and irrelevant features are usually contained in high-dimensional data whose presence, if ignored, can bring detrimental effects on the performance of data processing tasks. Unsupervised feature selection (UFS) as a trending topic in a great deal of domains has been greatly concerned, which can effectively filter out redundant and irrelevant features in the unlabeled data. The majority of existing UFS methods mainly focus on building the parameterized model in the original feature space or some certain low-dimensional feature subspace, which cannot fully capture the information of the target space. In this work, we propose a novel UFS method that unifies the explicit feature selective matrix and structured graph into a learning framework. An explicit selection matrix, whose scale, bound of the elements and structure are considered, is tailored for UFS. Furthermore, we take into account the local consistency so that the low dimensional manifold structures can be preserved to a better degree. Instead of using the pre-defined similarity matrix to construct graph, we make use of the way of learning to obtain the adaptive graph. Besides, for the purpose of exploiting the cluster structure of data, we impose a rank constraint on the graph. Notably, both the explicit selection matrix and intrinsic structure are learned in the target feature subspace, and thus the proposed methodology is more straightforward and effective. We present an effective and efficient algorithm for the resulting problem based on alternative optimization strategy, and establish its convergence and computational complexity analysis. The experimental results show that our proposed approach has improved by over 5.28% and 5.79% on average in terms of clustering accuracy (ACC) and Normalized Mutual Information (NMI) compared with comparison methods, which demonstrates its superiority.

ST-CellSeg: Cell segmentation for imaging-based spatial transcriptomics using multi-scale manifold learning.

Spatial transcriptomics has gained popularity over the past decade due to its ability to evaluate transcriptome data while preserving spatial information. Cell segmentation is a crucial step in spatial transcriptomic analysis, as it enables the avoidance of unpredictable tissue disentanglement steps. Although high-quality cell segmentation algorithms can aid in the extraction of valuable data, traditional methods are frequently non-spatial, do not account for spatial information efficiently, and perform poorly when confronted with the problem of spatial transcriptome cell segmentation with varying shapes. In this study, we propose ST-CellSeg, an image-based machine learning method for spatial transcriptomics that uses manifold for cell segmentation and is novel in its consideration of multi-scale information. We first construct a fully connected graph which acts as a spatial transcriptomic manifold. Using multi-scale data, we then determine the low-dimensional spatial probability distribution representation for cell segmentation. Using the adjusted Rand index (ARI), normalized mutual information (NMI), and Silhouette coefficient (SC) as model performance measures, the proposed algorithm significantly outperforms baseline models in selected datasets and is efficient in computational complexity.

Projected fuzzy [formula omitted]-means clustering algorithm with instance penalty

At present, high-dimensional data clustering has become a vital research field in machine learning. Traditional clustering algorithms cannot perform well on high-dimensional data, where the clustering task is usually divided into two stages: dimensionality reduction first and clustering later. In general, the existing high-dimensional clustering methods usually have the following shortcomings: (1) the two-stage strategy splits the connection between clustering and dimensionality reduction; (2) these algorithms do not consider the impact of anomalous instances in high-dimensional data on clustering performance. Therefore, to address these problems, a projected fuzzy c-means clustering algorithm with instance penalty (PCIP) is proposed. Firstly, we construct an instance penalty matrix and assign an instance penalty coefficient to each sample. Secondly, a model for clustering high-dimensional data is constructed by integrating fuzzy c-means clustering (FCM) and principal component analysis (PCA). The proposed model can perform dimensionality reduction and clustering simultaneously. In addition, the time complexity of the proposed algorithm is linearly related to the number of samples n, which can efficiently deal with large data sets. The proposed PCIP algorithm is verified by experiments using clustering accuracy and normalized mutual information (NMI) as evaluation metrics. The experimental results on 10 image datasets show that the average accuracy and average NMI of the PCIP algorithm are improved by 0.0375 and 0.0275, respectively, compared to the second-ranked algorithm.

Similarity-Based Three-Way Clustering by Using Dimensionality Reduction

Three-way clustering uses core region and fringe region to describe a cluster, which divide the dataset into three parts. The division helps identify the central core and outer sparse regions of a cluster. One of the main challenges in three-way clustering is the meaningful construction of the two sets. Aimed at handling high-dimensional data and improving the stability of clustering, this paper proposes a novel three-way clustering method. The proposed method uses dimensionality reduction techniques to reduce data dimensions and eliminate noise. Based on the reduced dataset, random sampling and feature extraction are performed multiple times to introduce randomness and diversity, enhancing the algorithm’s robustness. Ensemble strategies are applied on these subsets, and the k-means algorithm is utilized to obtain multiple clustering results. Based on these results, we obtain co-association frequency between different samples and fused clustering result using the single-linkage method of hierarchical clustering. In order to describe the core region and fringe region of each cluster, the similar class of each sample is defined by co-association frequency. The lower and upper approximations of each cluster are obtained based on similar class. The samples in the lower approximation of each cluster belong to the core region of the cluster. The differences between lower and upper approximations of each cluster are defined as fringe region. Therefore, a three-way explanation of each cluster is naturally formed. By employing various UC Irvine Machine Learning Repository (UCI) datasets and comparing different clustering metrics such as Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and Accuracy (ACC), the experimental results show that the proposed strategy is effective in improving the structure of clustering results.

Analyzing the Quality of Distorted Images by the Normalized Mutual Information Measure

This research explores how different types of distorting algorithms impact the Full-Reference image quality assessment, particularly when subjective quality evaluations are incorporated. We draw upon the TID2013 database, which contains 3000 images distorted by 24 distinct algorithms, in conjunction with Mean Opinion Scores (MOS) for quality ratings. We compare the results of Normalized Mutual Information (NMI) for image quality score with W2, based on Weibull distribution, the common PSNR similarity measure and MOS. We advocate for integrating of NMI into the repertoire of image quality assessment metrics.

Assessing the Association Between Trade and Regional Trade Agreements: A Network Approach

Over the years, the global economy has become increasingly integrated due to globalisation, resulting in an increase in international trade flows and a complex network of trade links. The establishment of regional trade agreements (RTAs) has created an intricate network of connections and overlapping agreements among multiple members. The study traces the evolution of trade and RTA networks using tools of network analysis for the periods 1995-2021. We find that the importance of countries in the trade network has changed, with developing countries replacing the previously dominant players. China, in particular, has emerged as a key player during this period. We find that connectivity in the trade network is n ot s trongly associated w ith t he RTA network. We a lso identify t hree m ajor c ommunities w ithin t he trade network, with the United States, Germany, and Japan (later China) as the core countries in these communities. Furthermore, we compare the community structures of the trade network and the RTA network using Normalised Mutual Information. Our analysis suggests that the community structure of trade has limited overlap with the community structure of RTAs. Furthermore, correlations based on the quadratic assignment procedure indicate a weak association, if any, between the two networks over the years.

Lung cancer clustering by identification of similarities and discrepancies of DNA copy numbers using maximal information coefficient.

Lung cancer is the second most diagnosed cancer and the first cause of cancer related death for men and women in the United States. Early detection is essential as patient survival is not optimal and recurrence rate is high. Copy number (CN) changes in cancer populations have been broadly investigated to identify CN gains and deletions associated with the cancer. In this research, the similarities between cancer and paired peripheral blood samples are identified using maximal information coefficient (MIC) and the spatial locations with substantially high MIC scores in each chromosome are used for clustering analysis. The results showed that a sizable reduction of feature set can be obtained using only a subset of locations with high MIC values. The clustering performance was evaluated using both true rate and normalized mutual information (NMI). Clustering results using the reduced feature set outperformed the performance of clustering using entire feature set in several chromosomes that are highly associated with lung cancer with several identified oncogenes.

Robust deep image clustering using convolutional autoencoder with separable discrete Krawtchouk and Hahn orthogonal moments

By cooperatively learning features and assigning clusters, deep clustering is superior to conventional clustering algorithms. Numerous deep clustering algorithms have been developed for a variety of application levels; however, the majority are still incapable of learning robust noise-resistant latent features, which limits the clustering performance. To address this open research challenge, we introduce, for the first time, a new approach called: Robust Deep Embedded Image Clustering algorithm with Separable Krawtchouk and Hahn Moments (RDEICSKHM). Our approach leverages the advantages of Krawtchouk and Hahn moments, such as local feature extraction, discrete orthogonality, and noise tolerance, to obtain a meaningful and robust image representation. Moreover, we employ LayerNormalization to further improve the latent space quality and facilitate the clustering process. We evaluate our approach on four image datasets: MNIST, MNIST-test, USPS, and Fashion-MNIST. We compare our method with several deep clustering methods based on two metrics: clustering accuracy (ACC) and normalized mutual information (NMI). The experimental results show that our method achieves superior or competitive performance on all datasets, demonstrating its effectiveness and robustness for deep image clustering.

Hybrid topic modeling method based on dirichlet multinomial mixture and fuzzy match algorithm for short text clustering

Topic modeling methods proved to be effective for inferring latent topics from short texts. Dealing with short texts is challenging yet helpful for many real-world applications, due to the sparse terms in the text and the high dimensionality representation. Most of the topic modeling methods require the number of topics to be defined earlier. Similarly, methods based on Dirichlet Multinomial Mixture (DMM) involve the maximum possible number of topics before execution which is hard to determine due to topic uncertainty, and many noises exist in the dataset. Hence, a new approach called the Topic Clustering algorithm based on Levenshtein Distance (TCLD) is introduced in this paper, TCLD combines DMM models and the Fuzzy matching algorithm to address two key challenges in topic modeling: (a) The outlier problem in topic modeling methods. (b) The problem of determining the optimal number of topics. TCLD uses the initial clustered topics generated by DMM models and then evaluates the semantic relationships between documents using Levenshtein Distance. Subsequently, it determines whether to keep the document in the same cluster, relocate it to another cluster, or mark it as an outlier. The results demonstrate the efficiency of the proposed approach across six English benchmark datasets, in comparison to seven topic modeling approaches, with 83% improvement in purity and 67% enhancement in Normalized Mutual Information (NMI) across all datasets. The proposed method was also applied to a collected Arabic tweet and the results showed that only 12% of the Arabic short texts were incorrectly clustered, according to human inspection.

Proof of biased behavior of Normalized Mutual Information

The Normalized Mutual Information (NMI) metric is widely utilized in the evaluation of clustering and community detection algorithms. This study explores the performance of NMI, specifically examining its performance in relation to the quantity of communities, and uncovers a significant drawback associated with the metric's behavior as the number of communities increases. Our findings reveal a pronounced bias in the NMI as the number of communities escalates. While previous studies have noted this biased behavior, they have not provided a formal proof and have not addressed the causation of this problem, leaving a gap in the existing literature. In this study, we fill this gap by employing a mathematical approach to formally demonstrate why NMI exhibits biased behavior, thereby establishing its unsuitability as a metric for evaluating clustering and community detection algorithms. Crucially, our study exposes the vulnerability of entropy-based metrics that employ logarithmic functions to similar bias.

SEC: More Accurate Clustering Algorithm via Structural Entropy

As one of the most popular machine learning tools in the field of unsupervised learning, clustering has been widely used in various practical applications. While numerous methods have been proposed for clustering, a commonly encountered issue is that the existing clustering methods rely heavily on local neighborhood information during the optimization process, which leads to suboptimal performance on real-world datasets. Besides, most existing clustering methods use Euclidean distances or densities to measure the similarity between data points. This could constrain the effectiveness of the algorithms for handling datasets with irregular patterns. Thus, a key challenge is how to effectively capture the global structural information in clustering instances to improve the clustering quality. In this paper, we propose a new clustering algorithm, called SEC. This algorithm uses the global structural information extracted from an encoding tree to guide the clustering optimization process. Based on the relation between data points in the instance, a sparse graph of the clustering instance can be constructed. By leveraging the sparse graph constructed, we propose an iterative encoding tree method, where hierarchical abstractions of the encoding tree are iteratively extracted as new clustering features to obtain better clustering results. To avoid the influence of easily misclustered data points located on the boundaries of the clustering partitions, which we call "fringe points", we propose an iterative pre-deletion and reassignment technique such that the algorithm can delete and reassign the "fringe points" to obtain more resilient and precise clustering results. Empirical experiments on both synthetic and real-world datasets demonstrate that our proposed algorithm outperforms state-of-the-art clustering methods and achieves better clustering performances. On average, the clustering accuracy (ACC) is increased by 1.7% and the normalized mutual information (NMI) by 7.9% compared with the current state-of-the-art (SOTA) algorithm on synthetic datasets. On real-world datasets, our method outperforms other clustering methods with an average increase of 12.3% in ACC and 5.2% in NMI, respectively.

Effective community detection with topic modeling in article recommender systems using LS-SLM and PCC-LDA

This paper introduces an innovative approach, the LS-SLM (Local Search with Smart Local Moving) technique, for enhancing the efficiency of article recommendation systems based on community detection and topic modeling. The methodology undergoes rigorous evaluation using a comprehensive dataset extracted from the “dblp. v12.json” citation network. Experimental results presented herein provide a clear depiction of the superior performance of the LS-SLM technique when compared to established algorithms, namely the Louvain Algorithm (LA), Stochastic Block Model (SBM), Fast Greedy Algorithm (FGA), and Smart Local Moving (SLM). The evaluation metrics include accuracy, precision, specificity, recall, F-Score, modularity, Normalized Mutual Information (NMI), betweenness centrality (BTC), and community detection time. Notably, the LS-SLM technique outperforms existing solutions across all metrics. For instance, the proposed methodology achieves an accuracy of 96.32%, surpassing LA by 16% and demonstrating a 10.6% improvement over SBM. Precision, a critical measure of relevance, stands at 96.32%, showcasing a significant advancement over GCR-GAN (61.7%) and CR-HBNE (45.9%). Additionally, sensitivity analysis reveals that the LS-SLM technique achieves the highest sensitivity value of 96.5487%, outperforming LA by 14.2%. The LS-SLM also demonstrates superior specificity and recall, with values of 96.5478% and 96.5487%, respectively. The modularity performance is exceptional, with LS-SLM obtaining 95.6119%, significantly outpacing SLM, FGA, SBM, and LA. Furthermore, the LS-SLM technique excels in community detection time, completing the process in 38,652 ms, showcasing efficiency gains over existing techniques. The BTC analysis indicates that LS-SLM achieves a value of 94.6650%, demonstrating its proficiency in controlling information flow within the network.

Increasing variance of rich-club nodes distribution in early onset depression according to dynamic network.

Early onset depression (EOD) and late onset depression (LOD) are thought to have different pathogeneses, but lack of pathological evidence. In the current study we describe the dynamic rich-club properties of patients with EOD and LOD to address this question indirectly. We recruited 82 patients with late life depression (EOD 40, LOD 42) and 90 healthy controls. Memory, executive function and processing speed were measured, and resting-stage functional MRI was performed with all participants. We constructed a dynamic functional connectivity network and carried out rich-club and modularity analyses. Normalized mutual information (NMI) was applied to describe the variance in rich-club nodes distribution and partitioning. The NMI coefficient of rich club nodes distribution among the three groups was the lowest in the EOD patients (F = 4.298; P = 0.0151, FDR = 0.0231), which was positively correlated with rich-club connectivity (R = 0.886, P < 0.001) and negatively correlated with memory (R = -0.347, P = 0.038) in the EOD group. In the LOD patients, non-rich-club connectivity was positively correlated with memory (R = 0.353, P = 0.030 and R = 0.420, P = 0.009). Furthermore, local connectivity was positively correlated with processing speed in the LOD patients (R = 0.374, P = 0.021). The modular partition was different between the EOD patients and the HCs (P = 0.0013 < 0.05/3). The temporal instability of rich-club nodes was found in the EOD patients, but not the LOD patients, supporting the hypothesis that EOD and LOD result from different pathogenesis, and showing that the instability of the rich-club nodes across time might disrupt rich-club connectivity.

Discrete-Time Quantum Walks Community Detection in Multi-Domain Networks

Abstract The problem of detecting communities in real-world networks has been extensively studied in the past, but most of the existing approaches work on single-domain networks, i.e. they consider only one type of relationship between nodes. Single-domain networks may contain noisy edges and they may lack some important information. Thus, some authors have proposed to consider the multiple relationships that connect the nodes of a network, thus obtaining multi-domain networks. However, most community detection approaches are limited to multi-layer networks, i.e. networks generated from the superposition of several single-domain networks (called layers) that are regarded as independent of each other. In addition to being computationally expensive, multi-layer approaches might yield inaccurate results because they ignore potential dependencies between layers. This paper proposes a multi-domain discrete-time quantum walks (MDQW) model for multi-domain networks. First, the walking space of network nodes in multi-domain network is constructed. Second, the quantum permutation circuit of the coin state is designed based on the coded particle state. Then, using different coin states, the shift operator performs several quantum walks on the particles. Finally, the corresponding update rule is selected to move the node according to the measurement result of the quantum state. With continuous update iteration, the shift operator automatically optimizes the discovered community structure. We experimentally compared our MDQW method with four state-of-the-art competitors on five real datasets. We used the normalized mutual information (NMI) to compare clustering quality, and we report an increase in NMI of up to 3.51 of our MDQW method in comparison with the second-best performing competitor. The MDQW method is much faster than its competitors, allowing us to conclude that MDQW is a useful tool in the analysis of large real-life multi-domain networks. Finally, we illustrate the usefulness of our approach on two real-world case studies.

The correlations between galaxy properties in different environments of the cosmic web

Abstract We study the correlations between (u-r) colour, stellar mass, specific star formation rate (sSFR) and metallicity of galaxies in different geometric environments of the cosmic web using a volume limited sample from the SDSS. The geometric environment at the location of each galaxy is determined using the eigenvalues of the tidal tensor in three dimensions. We use the Pearson correlation coefficient (PCC) and the normalized mutual information (NMI) to quantify the correlations between these galaxy properties in sheets, filaments and clusters after matching the stellar mass distributions of the galaxies in these environments. A two-tailed t-test assesses the statistical significance of the observed differences between these relations in different geometric environments. The null hypothesis can be rejected at &gt;99.99% significance level in most of the cases, suggesting that the scaling relations between the observable galaxy properties are susceptible to the geometric environments of the cosmic web.