Jensen-Shannon Divergence Research Articles

Quantifying magic resource via quantum Jensen-Shannon divergence

Abstract Magic is a precious resource necessary for achieving universal fault-tolerant quantum computation. Therefore, it is of vital importance to study the detection and quantification of the magic resource encompassed in quantum states and quantum gates both theoretically and experimentally. In this work, we adopt the quantum Jensen-Shannon divergence to quantify the magic resource of quantum states and quantum gates. On the one hand, we determine the magic resource of a pure state as the minimal and average distance between this state and the set of pure stabilizer states via the quantum Jensen-Shannon divergence, respectively, and extend them to the general mixed states through the method of convex roof construction. We investigate the basic properties of these two magic quantifiers and utilize them to evaluate the magic resource for some typical qubit and qutrit states. By comparing the magic quantifier via the quantum Jensen-Shannon divergence with the min-relative entropy of magic and the stabilizer α-Rényi entropies, we find that the min-relative entropy of magic provides both an upper bound and a lower bound for the magic quantifier via the quantum Jensen-Shannon divergence, and the stabilizer α-Rényi entropies provide a series of lower bounds for the magic quantifier via the quantum Jensen-Shannon divergence. On the other hand, based on the magic quantifier via the quantum Jensen-Shannon divergence for quantum states, we further propose two quantifiers for the magic-resource-generating power of quantum gates and demonstrate that the T-gate is the optimal diagonal unitary gate in creating magic resource for both qubit and qutrit systems in the sense of Clifford equivalence.

Automatic classification of HEp-2 specimens by explainable deep learning and Jensen-Shannon reliability index

The Anti-Nuclear Antibodies (ANA) test using Human Epithelial type 2 (HEp-2) cells in the Indirect Immuno-Fluorescence (IIF) assay protocol is considered the gold standard for detecting Connective Tissue Diseases. Computer-assisted systems for HEp-2 image analysis represent a growing field that harnesses the potential offered by novel machine learning techniques to address the classification of HEp-2 images and ANA patterns.In this study, we introduce an innovative platform based on transfer learning with pre-trained deep learning models. This platform combines the power of unsupervised deep description of HEp-2 images, a novel feature selection approach designed for unbalanced datasets, and independent testing using two distinct datasets from different hospitals to tackle cross-hardware compatibility issues. To enhance the trustworthiness of our method, we also present a modified version of gradient-weighted class activation mapping for regional explainability and introduce a new sample quality index based on the Jensen-Shannon divergence to enhance method reliability and quantify sample heterogeneity.The results we provide demonstrate exceptionally high performance in intensity and ANA pattern recognition when compared to state-of-the-art approaches. Our method's ability to eliminate the need for cell segmentation in favor of statistical analysis of the sample makes it applicable, robust, and versatile. Our future work will focus on addressing the challenge of mitotic spindle recognition by expanding our proposed approach to cover mixed patterns.

A New Natural Language Processing-Inspired Methodology (Detection, Initial Characterization, and Semantic Characterization) to Investigate Temporal Shifts (Drifts) in Health Care Data: Quantitative Study.

Proper analysis and interpretation of health care data can significantly improve patient outcomes by enhancing services and revealing the impacts of new technologies and treatments. Understanding the substantial impact of temporal shifts in these data is crucial. For example, COVID-19 vaccination initially lowered the mean age of at-risk patients and later changed the characteristics of those who died. This highlights the importance of understanding these shifts for assessing factors that affect patient outcomes. This study aims to propose detection, initial characterization, and semantic characterization (DIS), a new methodology for analyzingchanges in health outcomes and variables over time while discovering contextual changes for outcomes in large volumes of data. The DIS methodology involves 3 steps: detection, initial characterization, and semantic characterization. Detection uses metrics such as Jensen-Shannon divergence to identify significant data drifts. Initial characterization offers a global analysis of changes in data distribution and predictive feature significance over time. Semantic characterization uses natural language processing-inspired techniques to understand the local context of these changes, helping identify factors driving changes in patient outcomes. By integrating the outcomes from these 3 steps, our results can identify specific factors (eg, interventions and modifications in health care practices) that drive changes in patient outcomes. DIS was applied to the Brazilian COVID-19 Registry and the Medical Information Mart for Intensive Care, version IV (MIMIC-IV) data sets. Our approach allowed us to (1) identify drifts effectively, especially using metrics such as the Jensen-Shannon divergence, and (2) uncover reasons for the decline in overall mortality in both the COVID-19 and MIMIC-IV data sets, as well as changes in the cooccurrence between different diseases and this particular outcome. Factors such as vaccination during the COVID-19 pandemic and reduced iatrogenic events and cancer-related deaths in MIMIC-IV were highlighted. The methodology also pinpointed shifts in patient demographics and disease patterns, providing insights into the evolving health care landscape during the study period. We developed a novel methodology combining machine learningand natural language processing techniques to detect, characterize, and understand temporal shifts in health care data. This understanding can enhance predictive algorithms, improve patient outcomes, and optimize health care resource allocation, ultimatelyimproving the effectiveness of machine learning predictive algorithms applied to health care data. Our methodology can be applied to a variety of scenarios beyond those discussed in this paper.

Enhancing long-term water quality modeling by addressing base demand, demand patterns, and temperature uncertainty using unsupervised machine learning techniques

Water quality modelling in Water Distribution systems (WDS) is frequently affected by uncertainties in input variables such as base demand and decay constants. When utilizing simulation tools like EPANET, which necessitate exact numerical inputs, these uncertainties can result in inaccurate simulations. This study proposes a novel framework that leverages unsupervised machine learning, specifically a Gaussian Mixture Model (GMMs), to represent and integrate these uncertainties in the simulation. By classifying historical water demand into fuzzy clusters, the framework allows for certain linguistic inputs (e.g., "high" or "low" demand) to be used in water quality simulations. The framework also incorporates representative hourly demand patterns and temperature-dependent chlorine decay constants based on historical data correlations. Validations were conducted on the Anytown network using WNTR-EPANET, comparing simulated chlorine residuals with Validation data from 181 steady-state simulations. The simulation through the framework achieved a Jensen-Shannon Divergence (JSD) of <0.008 across all demand clusters, indicating high similarity between predicted and actual probability distributions . In comparison to other simulation scenarios tested, which exhibited increased variability (JSD > 0.18), the proposed framework demonstrated improved accuracy in representing chlorine residual distributions. The methodology is adaptable to other systems, if similar historical datasets containing key variables, such as flow rates and temperature, are provided. While the framework offers a more flexible and accurate approach to handling uncertainties in WDS, its effectiveness is contingent upon the availability of robust historical demand and temperature data for decay constant calibration.

A Basket Trial Design Based on Power Priors

In basket trials a treatment is investigated in several subgroups. They are primarily used in oncology in early clinical phases as single-arm trials with a binary endpoint. For their analysis primarily Bayesian methods have been suggested, as they allow partial sharing of information based on the observed similarity between subgroups. Fujikawa et al. suggested an approach using empirical Bayes methods that allows flexible sharing based on easily interpretable weights derived from the Jensen-Shannon divergence between the subgroup-wise posterior distributions. We show that this design is closely related to the method of power priors and investigate several modifications of Fujikawa’s design using methods from the power prior literature. While in Fujikawa’s design, the amount of information that is shared between two baskets is only determined by their pairwise similarity, we also discuss extensions where the outcomes of all baskets are considered in the computation of the sharing weights. The results of our comparison study show that the power prior design has comparable performance to fully Bayesian designs in a range of different scenarios. At the same time, the power prior design is computationally cheap and even allows analytical computation of operating characteristics in some settings.

Should the multi-layer transportation network structure be reduced?

ABSTRACT The increasing diversity of transportation modes and the rapid expansion of transportation networks present significant challenges for modeling multi-layer comprehensive transportation networks. It is crucial to determine whether aggregating certain layers is a viable option for balancing complexity reduction and information preservation. This decision defines the layered structures and informs subsequent analyses of these networks. Two-dimensional factors, namely topological structures and transportation attributes, are considered to enhance understanding of the similarities among network layers. The relative entropy and the Gini index are employed as metrics to assess information gain or loss resulting from layer aggregation or segregation, guiding decisions on network reduction. Furthermore, an integrated similarity measure, based on the quantum Jensen-Shannon divergence and the Gower distance, is utilized to identify the optimal aggregation sequences. Two real-world transportation networks serve as case studies. Results demonstrate that these transportation networks are more effectively maintained with layer-separated structures, preserving maximum information.

Tensor multi-view clustering method for natural image segmentation

Image segmentation methods based on clustering have become a advancement in the field of computer vision, and the performance of these methods mainly depends on the performance of the clustering algorithm. However, these methods have some shortcomings. First, a single feature cannot accurately obtain the complete information of the superpixels. Second, the commonly used Euclidean distance cannot extract the nonlinear manifold structure between superpixels. Third, the high-dimensional information between superpixels is not considered. Therefore, this article proposes a Tensor Multi-view Clustering Method for Natural Image Segmentation (TMCNIS), in which firstly the multiclass features of the superpixels are extracted to obtain complete information. Secondly, the Jensen–Shannon divergence is used to delineate the relationship between superpixels to obtain a more complete nonlinear structure. Thirdly, an adaptively weighted tensor Schatten-p norm is proposed to better approximate the target rank of the tensor. It also automatically shrinks the singular values appropriately to construct a finer tensor, thus fully capturing the spatial structure in the tensor. Experimental results on four clustering datasets and two image segmentation datasets demonstrate the superiority of TMCNIS compared to existing methods.

Assessing clustering methods using Shannon's entropy

Unsupervised clustering techniques are crucial for effectively partitioning datasets into meaningful subgroups. In this paper, we introduce a novel clustering fuzziness metric based on Shannon's entropy, which quantifies the level of uncertainty in clustering assignments. This metric is employed to develop a statistical test for evaluating clustering algorithms and to propose parametric corrections to enhance their performance. We provide empirical evidence showing that our approach significantly improves clustering quality compared to well-known algorithms such as Fuzzy K-Means (FKM) and Fanny. For instance, applying our correction method led to a notable decrease in Jensen-Shannon Divergence (JSD) values, indicating enhanced clustering accuracy. Our methodology is demonstrated through comprehensive experiments on both simulated and real-world datasets. Additionally, our approach proves effective in determining the optimal number of clusters, showing superiority over traditional methods. These results highlight the practical benefits of our metric and correction techniques for improving clustering performance.

Multi-Regional Pelvic Floor Muscle Function Diagnosis System Based on Inflatable Stretchable Electrode Array.

Effective prevention and treatment of pelvic floor dysfunction (PFD) necessitates the identification of lesions within the complex pelvic floor muscle (PFM) groups associated with various symptoms. Here, we developed a multi-region pelvic floor muscle functional diagnosis system (MPDS) based on an inflatable stretchable electrode array, which aids in accurately locating areas related to PFD. Clinical diagnostic experiments were conducted on 56 patients with postpartum stress urinary incontinence (PSUI) and 73 postpartum asymptomatic controls. MPDS collects pelvic floor electromyography from all participants. By assessing EMG parameters such as activation time differences (ATD) and using Jensen-Shannon (JS) divergence to verify, with the aim of locating target muscle groups with functional abnormalities. Clinical test results showed that by observing the AT sequence of the PSUI group and the control group, muscle groups with functional abnormalities in the Pubococcygeus muscle (PC) and Puborectalis muscle (PR) regions could be preliminarily diagnosed. In the assessment of regional muscle contribution values based on JS divergence, it was verified that the contribution values of rapid contraction in the PC and PR regions of the PSUI group were relatively lower compared to those of the control group, which correlated with urinary control dysfunction. These experiments demonstrate that the MPDS helps in accurately locating target muscle groups with functional abnormalities, showcasing its potential in precise assessment of complex muscle groups such as PFM, which may improve diagnostic precision and reliability.

LDAPrototype: a model selection algorithm to improve reliability of latent Dirichlet allocation

Latent Dirichlet allocation (LDA) is a popular method for analyzing large text corpora, but it suffers from instability due to its reliance on random initialization. This results in different outcomes for replicated runs, hindering reproducibility. To address this, we introduce LDAPrototype, a new approach for selecting the most representative LDA run from multiple replications on the same dataset. LDAPrototype enhances the reliability of LDA conclusions by ensuring greater similarity between replications compared to traditional LDA runs or models chosen based on perplexity or NPMI. A key feature of LDAPrototype is its use of a novel model similarity measure called S-CLOP (Similarity of multiple sets by Clustering with LOcal Pruning). It is based on topic similarities, for which we compare the usage of measures like the thresholded Jaccard coefficient, cosine similarity, Jensen-Shannon divergence, and rank-biased overlap. The effectiveness of LDAPrototype is demonstrated through its application to six real datasets, including newspaper articles and tweets. The results show improved reproducibility and reliability in topic modeling outcomes. LDAPrototype’s approach is noteworthy for its practical applicability, comprehensibility, ease of implementation, and computational efficiency. Furthermore, the algorithm’s concept can be generalized to other topic modeling procedures that characterize topics through word distributions, making it a versatile tool in text data analysis.

Steered quantum coherence and entropic uncertainty relation in the cluster Ising model

Abstract In this paper, we investigate the cluster Ising model (CIM) via steered quantum coherence (SQC) and entropic uncertainty relation (EUR). We present the behavior of SQC quantified by the L1 norm, relative entropy and quantum Jensen–Shannon divergence. We also demonstrate the properties of EUR in the CIM. In addition, we provide a comparative analysis of these measures and present detailed numerical results.

Enhancing fruit and vegetable detection in unconstrained environment with a novel dataset

Automating the detection of fruits and vegetables using computer vision is essential for modernizing agriculture, improving efficiency, ensuring food quality, and contributing to sustainable and technologically advanced farming practices. This paper presents an end-to-end pipeline for detecting and localizing fruits and vegetables in real-world scenarios. To achieve this, a dataset named FRUVEG67 was curated that includes images of 67 classes of fruits and vegetables captured in unconstrained scenarios, with only a few manually annotated samples per class. A semi-supervised data annotation algorithm (SSDA) was developed that generates bounding boxes for objects to label the remaining nonannotated images. For detection, Fruit and Vegetable Detection Network (FVDNet) was proposed, an ensemble version of YOLOv8n featuring three distinct grid configurations. In addition, an averaging approach for the prediction of the bounding box and a voting mechanism for the prediction of the classes was implemented. Finally Jensen–Shannon Divergence (JSD) in conjunction with focal loss was integrated as the overall loss function for better detection of smaller objects. Experimental results highlight the superiority of FVDNet compared to recent versions of YOLO, showcasing remarkable improvements in detection and localization performance. An impressive mean average precision (mAP) score of 0.82 across all classes was achieved. Furthermore, the efficacy of FVDNet on open-category refrigerator images were evaluated, where it demonstrates promising results.

Metabolic network connectivity disturbances in Parkinson's disease: a novel imaging biomarker.

The diagnosis of Parkinson's Disease (PD) presents ongoing challenges. Advances in imaging techniques like 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) have highlighted metabolic alterations in PD, yet the dynamic network interactions within the metabolic connectome remain elusive. To this end, we examined a dataset comprising 49 PD patients and 49 healthy controls. By employing a personalized metabolic connectome approach, we assessed both within- and between-network connectivities using Standard Uptake Value (SUV) and Jensen-Shannon Divergence Similarity Estimation (JSSE). A random forest algorithm was utilized to pinpoint key neuroimaging features differentiating PD from healthy states. Specifically, the results revealed heightened internetwork connectivity in PD, specifically within the somatomotor (SMN) and frontoparietal (FPN) networks, persisting after multiple comparison corrections (P < 0.05, Bonferroni adjusted for 10% and 20% sparsity). This altered connectivity effectively distinguished PD patients from healthy individuals. Notably, this study utilizes 18F-FDG PET imaging to map individual metabolic networks, revealing enhanced connectivity in the SMN and FPN among PD patients. This enhanced connectivity may serve as a promising imaging biomarker, offering a valuable asset for early PD detection.

Automatic Identification of Three-Body Scatter Spike Based on Jensen–Shannon Divergence and Support Vector Machine

Abstract The three-body scatter spike (TBSS), an echo artifact in radar imagery, manifests as a weak, linear echo extending radially from a core of high reflectivity. Adequately, though not indispensably, indicating the presence of large hail in convective storms, the automatic identification of the TBSS proves advantageous in significantly improving the effectiveness of hailstorm detection. This study introduces an algorithm that synergizes Jensen–Shannon divergence (JSD) and support vector machine (SVM) for rapid TBSS detection in two decades’ worth of single-polarization radar data across China. The algorithm, tested on data from 50 S-band China Next Generation Weather Radar (CINRAD) in central and eastern China, utilized reflectivity factor images for sample extraction. An application in Chenzhou, China, demonstrates the algorithm’s efficacy in improving hailstorm detection resolution. Significance Statement In recent years, China’s hail recordkeeping, primarily based on manual observations at national surface meteorological stations, has suffered from limited spatial and temporal detail. However, the advent of the China Next Generation Weather Radar (CINRAD) network offers a new avenue for hailstorm detection. TBSS, a secondary but significant indicator for large hail in S-band radar, presents an opportunity for enhanced hail warning capabilities. By automating TBSS detection in radar archives spanning two decades, this research significantly enhances the resolution of hailstorm climatology, contributing to more effective hail disaster mitigation and management.

Efficient Jamming Policy Generation Method Based on Multi-Timescale Ensemble Q-Learning

With the advancement of radar technology toward multifunctionality and cognitive capabilities, traditional radar countermeasures are no longer sufficient to meet the demands of countering the advanced multifunctional radar (MFR) systems. Rapid and accurate generation of the optimal jamming strategy is one of the key technologies for efficiently completing radar countermeasures. To enhance the efficiency and accuracy of jamming policy generation, an efficient jamming policy generation method based on multi-timescale ensemble Q-learning (MTEQL) is proposed in this paper. First, the task of generating jamming strategies is framed as a Markov decision process (MDP) by constructing a countermeasure scenario between the jammer and radar, while analyzing the principle radar operation mode transitions. Then, multiple structure-dependent Markov environments are created based on the real-world adversarial interactions between jammers and radars. Q-learning algorithms are executed concurrently in these environments, and their results are merged through an adaptive weighting mechanism that utilizes the Jensen–Shannon divergence (JSD). Ultimately, a low-complexity and near-optimal jamming policy is derived. Simulation results indicate that the proposed method has superior jamming policy generation performance compared with the Q-learning algorithm, in terms of the short jamming decision-making time and low average strategy error rate.

GUIDE: GAN-based UAV IDS Enhancement

With the development of information technology, many devices are connected and automated by networks. Unmanned Areal Vehicles (UAVs), commonly known as drones, are one of the most popular devices that can perform various tasks. However, the risk of cyberattacks on UAVs is increasing as UAV utilization grows. These cyberattacks can cause serious safety problems, such as crashes. Therefore, it is essential to detect these attacks and take countermeasures. As a countermeasure, intrusion detection system (IDS) is widely adopted. To implement IDS for UAVs, it should be lightweight and be able to detect unknown attacks as a requirement. We propose GAN-based UAV IDS Enhancement (GUIDE) to meet the requirements. The GUIDE employs a generative adversarial network (GAN) for integer-valued sequence data augmentation to enhance an IDS’s performance on known and unknown attacks. We used five GANs: SeqGAN, MaskGAN, RankGAN, StepGAN, and LeakGAN; we used four non-learning augmentation methods for the comparative experiment: oversampling, undersampling, noise addition, and random generation. The experimental results demonstrated that the synthetic data generated by GANs improved the detection of known attacks (up to 37 percentage points) and unknown attacks (up to 30 percentage points) while maintaining stable IDS performance. We also analyzed the synthetic data by employing Jensen–Shannon divergence, synthetic ranking agreement, and visualization; we confirmed that the synthetic data contained the characteristics of real data and could be used for training the IDS.

New Improvements of the Jensen–Mercer Inequality for Strongly Convex Functions with Applications

In this paper, we use the generalized version of convex functions, known as strongly convex functions, to derive improvements to the Jensen–Mercer inequality. We achieve these improvements through the newly discovered characterizations of strongly convex functions, along with some previously known results about strongly convex functions. We are also focused on important applications of the derived results in information theory, deducing estimates for χ-divergence, Kullback–Leibler divergence, Hellinger distance, Bhattacharya distance, Jeffreys distance, and Jensen–Shannon divergence. Additionally, we prove some applications to Mercer-type power means at the end.

Incoherence: A Generalized Measure of Complexity to Quantify Ensemble Divergence in Multi-Trial Experiments and Simulations.

Complex systems pose significant challenges to traditional scientific and statistical methods due to their inherent unpredictability and resistance to simplification. Accurately detecting complex behavior and the uncertainty which comes with it is therefore essential. Using the context of previous studies, we introduce a new information-theoretic measure, termed "incoherence". By using an adapted Jensen-Shannon Divergence across an ensemble of outcomes, we quantify the aleatoric uncertainty of the system. First we compared this measure to established statistical tests using both continuous and discrete data. Before demonstrating how incoherence can be applied to identify key characteristics of complex systems, including sensitivity to initial conditions, criticality, and response to perturbations.

Complex Function-Based Fault Detection: A New Method to Measure the Complexity of Nonlinear Time Series

In this paper, we propose a new complexity computation based on the complex function. This measure exploits the dispersion Lempel–Ziv complexity (DLZC) and the dispersion statistical complexity measure based on Jensen–Shannon divergence (DCJS) of the analytic signal and constructs the binary complex function, called complex number complexity (CNC). The statistical measure depicts the complex system from different angle and mines more potential information. Through simulation experiments, we prove that the proposed method is able to detect data fluctuations more sensitively and accurately. For an application, the CNC is applied to fault detection which has different fault diameters and rotational speeds. The results deliver that the CNC measure can well represent the complexity of faulty bearings and has significant differences in any two fault types. The technique is effective to characterize different fault.

A hyper-distance-based method for hypernetwork comparison.

Hypernetwork is a useful way to depict multiple connections between nodes, making it an ideal tool for representing complex relationships in network science. In recent years, there has been a marked increase in studies on hypernetworks; however, the comparison of the difference between two hypernetworks has received less attention. This paper proposes a hyper-distance (HD)-based method for comparing hypernetworks. The method is based on higher-order information, i.e, the higher-order distance between nodes and Jensen-Shannon divergence. Experiments carried out on synthetic hypernetworks have shown that HD is capable of distinguishing between hypernetworks generated with different parameters, and it is successful in the classification of hypernetworks. Furthermore, HD outperforms current state-of-the-art baselines to distinguish empirical hypernetworks when hyperedges are randomly perturbed.