Jensen-Shannon Divergence Research Articles

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Revealing the critical state and identifying individualized dynamic network biomarker for type 2 diabetes through advanced analysis methods on individual basis

Complex diseases may not always progress in a gradual manner. In the early stages of complex diseases, obvious symptoms are usually not observable, but there is a commonality: there is a brief state of predisease between the progression from normal state to disease state, which usually includes three stages: normal state, critical state of predisease, and disease state. Identifying this critical state, especially with a single sample from an individual, remains a difficult task. In this study, we applied three methods, i.e., single-sample Jensen–Shannon Divergence (sJSD), network information gain (NIG), and temporal network flow entropy (TNFE) method, to a simulated dataset and type 2 diabetes (GSE13268 and GSE13269). Three different methods were utilized to create indexes, including the Inconsistency Index (ICI), NIG, and TNFE, to measure the overall disruption caused by individual samples compared to a set of reference samples. Changes in these indexes were used to identify critical states during the progression of the disease. Results from the numerical simulations show the effectiveness of the three methods. All the methods can detect two critical states based on a single sample, which are respectively at 8 weeks and 16 weeks for GSE13268 and at 4 weeks and 16 weeks for GSE13269, indicating the critical states before deterioration can be detected and the dynamic network biomarkers (DNBs) can be identified successfully. But there are differences in the sensitivity of predictive indicators based on the three methods. The identified dynamic network biomarkers are also significantly different. In addition, the computational principles of the three methods are compared. The proposed three methods can effectively detect the critical state and identify the DNB, solely based on a single sample. The three methods are data-driven and model-free on an individual basis. sJSD method is more sensitive to the critical state, while NIG and TNFE methods are more robust and effective. They can therefore not only help future studies of personalized disease diagnosis but also provide a better insight into clinical practice.

Quantifying and simulating the weather forecast uncertainty for advanced building control

Weather forecast uncertainty is unavoidable despite technological advancements. Accurately quantifying and modelling this uncertainty is essential for developing and comparing advanced building controllers. In this study, we present a structured approach using a first-order autoregressive model (AR(1)) to model uncertainty in ambient temperature and global solar irradiation (GHI) forecasts. We analyzed weather data from four cities and employed Jensen–Shannon divergence (JSD) to evaluate the similarity between synthetic and actual forecast errors. The average JSD values for temperature are 0.027 (Berkeley), 0.021 (Leuven), 0.018 (Berlin), and 0.008 (Oslo), and for GHI, the average JSD values are 0.016 (Berkeley), 0.058 (Leuven), and 0.013 (Berlin). The low JSD values indicate a high similarity between the synthetic and real forecast error distributions. Our approach successfully generates synthetic weather forecasts that mirror the statistical properties of actual forecasts. The implementation of our method for uncertain forecast generation is being added to the BOPTEST framework.

Measuring and utilizing temporal network dissimilarity

Quantifying the structural and functional differences of temporal networks remains a fundamental and challenging problem in the era of big data. Traditional network comparison methods, originally developed for static networks, often fall short in capturing the intricate interplay between structural configurations and dynamic temporal patterns inherent in complex systems. This work proposes a temporal dissimilarity measure for temporal network comparison based on the first arrival distance distribution and spectral entropy based Jensen-Shannon divergence. Experimental results on both synthetic and empirical temporal networks show that the proposed measure could discriminate diverse temporal networks with different structures by capturing various topological and temporal properties. Moreover, the proposed measure can discern the functional distinctions and is found effective applications in temporal network classification and spreadability discrimination.

A novel method for functional brain networks based on static cerebral blood flow.

Cerebral blood flow (CBF) offers a quantitative and reliable measurement for brain activity and is increasingly used to study functional networks. However, current methods evaluate inter-regional relations mainly based on CBF temporal dynamics, which suffers from low signal-to-noise ratio and poor temporal resolution. Here we proposed a method to construct functional brain networks by estimating shape similarity (index by Jensen-Shannon divergence) in probability distributions of regional static CBF measured by arterial spin labeling perfusion imaging over a scanning period. Based on CBF data of 30 healthy participants from 10 visits, we found that the CBF networks exhibited non-trivial topological features (e.g., small-world organization, modular architecture, and hubs) and showed low-to-fair test-retest reliability and high between-subject consistency. We further found that interregional CBF similarities were depended on anatomical distance and differed between high- and lower-order subnetworks. Moreover, interregional CBF similarities within high-order subnetworks showed significantly lower reliability than those within low-order subnetworks. Finally, we showed that nodal degree of the CBF networks were related to regional sizes and CBF levels and spatially aligned with maps of the dopamine transporter and metabolic glutamate receptor 5 intensities, expression levels of genes primarily enriched in cholesterol-related pathways and endothelial cells, and meta-analytic activations related to memory, language, and executive function. Altogether, our proposed method provide a novel, relatively reliable, and neurobiologically meaningful means to study functional network organization of the human brain.

Transformer vibration state anomaly detection based on Jensen-Shannon divergence and ensemble learning

Transformer vibration state anomaly detection based on Jensen-Shannon divergence and ensemble learning

Quantifying magic resource via quantum Jensen–Shannon divergence

Abstract Magic is a precious resource necessary for achieving universal fault-tolerant quantum computation. Therefore, it is of vital importance to study the detection and quantification of the magic resource encompassed in quantum states and quantum gates both theoretically and experimentally. In this work, we adopt the quantum Jensen–Shannon divergence to quantify the magic resource of quantum states and quantum gates. On the one hand, we determine the magic resource of a pure state as the minimal and average distance between this state and the set of pure stabilizer states via the quantum Jensen–Shannon divergence, respectively, and extend them to the general mixed states through the method of convex roof construction. We investigate the basic properties of these two magic quantifiers and utilize them to evaluate the magic resource for some typical qubit and qutrit states. By comparing the magic quantifier via the quantum Jensen–Shannon divergence with the min-relative entropy of magic and the stabilizer α-Rényi entropies, we find that the min-relative entropy of magic provides both an upper bound and a lower bound for the magic quantifier via the quantum Jensen-Shannon divergence, and the stabilizer α-Rényi entropies provide a series of lower bounds for the magic quantifier via the quantum Jensen–Shannon divergence. On the other hand, based on the magic quantifier via the quantum Jensen–Shannon divergence for quantum states, we further propose two quantifiers for the magic-resource-generating power of quantum gates and demonstrate that the T-gate is the optimal diagonal unitary gate in creating magic resource for both qubit and qutrit systems in the sense of Clifford equivalence.

Oral Microbiome Signatures as Potential Biomarkers for Gastric Cancer Risk Assessment.

Gastric cancer (GC) is the fifth leading cause of cancer-related death worldwide. The oral microbiota was investigated for distinguishable characteristics between GC, premalignant gastric conditions (Pre-GC), and control participants. Mouthwash samples from GC, Pre-GC, and control participants at a tertiary care center were prospectively collected. Following DNA extraction and sequencing, analyses of oral microbiome biodiversity and composition were performed, and receiver operating characteristic curves were created to evaluate the discriminative power of oral microbiome signatures. Oral samples from 98 participants included 30 (30.6%) GC, 30 (30.6%) Pre-GC and 38 (38.8%) controls. Of these, 61 (62.2%) were female, 31 (31.6%) were Hispanic, and 18 (18.3%) were smokers. GC compared to controls demonstrated notable differences in beta diversity (Jensen-Shannon Divergence and Bray-Curtis Dissimilarity, p<0.02). 32 bacterial genera were found to be differentially abundant when comparing GC and controls, and 23 bacterial genera demonstrated differential abundance when comparing Pre-GC and controls (W-statistic >2). Minimal compositional differences between GC and Pre-GC were found, with only three differentially abundant bacterial genera (W-statistic >2). Models were constructed from the most significant bacterial signatures (W-statistic >5). These models discriminated between GC and control oral samples with an AUC of 0.880 (95% CI 0.808, 0.952) and between Pre-GC and control oral samples with an AUC of 0.943 (95% CI 0.887, 0.999). Oral rinses of GC and Pre-GC participants exhibited distinct but similar microbiome profiles, distinguishing them from controls. This compositional difference raises the possibility of utilizing these microbial signatures to predict GC risk.

Automatic classification of HEp-2 specimens by explainable deep learning and Jensen-Shannon reliability index

The Anti-Nuclear Antibodies (ANA) test using Human Epithelial type 2 (HEp-2) cells in the Indirect Immuno-Fluorescence (IIF) assay protocol is considered the gold standard for detecting Connective Tissue Diseases. Computer-assisted systems for HEp-2 image analysis represent a growing field that harnesses the potential offered by novel machine learning techniques to address the classification of HEp-2 images and ANA patterns.In this study, we introduce an innovative platform based on transfer learning with pre-trained deep learning models. This platform combines the power of unsupervised deep description of HEp-2 images, a novel feature selection approach designed for unbalanced datasets, and independent testing using two distinct datasets from different hospitals to tackle cross-hardware compatibility issues. To enhance the trustworthiness of our method, we also present a modified version of gradient-weighted class activation mapping for regional explainability and introduce a new sample quality index based on the Jensen-Shannon divergence to enhance method reliability and quantify sample heterogeneity.The results we provide demonstrate exceptionally high performance in intensity and ANA pattern recognition when compared to state-of-the-art approaches. Our method's ability to eliminate the need for cell segmentation in favor of statistical analysis of the sample makes it applicable, robust, and versatile. Our future work will focus on addressing the challenge of mitotic spindle recognition by expanding our proposed approach to cover mixed patterns.

A New Natural Language Processing-Inspired Methodology (Detection, Initial Characterization, and Semantic Characterization) to Investigate Temporal Shifts (Drifts) in Health Care Data: Quantitative Study.

Proper analysis and interpretation of health care data can significantly improve patient outcomes by enhancing services and revealing the impacts of new technologies and treatments. Understanding the substantial impact of temporal shifts in these data is crucial. For example, COVID-19 vaccination initially lowered the mean age of at-risk patients and later changed the characteristics of those who died. This highlights the importance of understanding these shifts for assessing factors that affect patient outcomes. This study aims to propose detection, initial characterization, and semantic characterization (DIS), a new methodology for analyzingchanges in health outcomes and variables over time while discovering contextual changes for outcomes in large volumes of data. The DIS methodology involves 3 steps: detection, initial characterization, and semantic characterization. Detection uses metrics such as Jensen-Shannon divergence to identify significant data drifts. Initial characterization offers a global analysis of changes in data distribution and predictive feature significance over time. Semantic characterization uses natural language processing-inspired techniques to understand the local context of these changes, helping identify factors driving changes in patient outcomes. By integrating the outcomes from these 3 steps, our results can identify specific factors (eg, interventions and modifications in health care practices) that drive changes in patient outcomes. DIS was applied to the Brazilian COVID-19 Registry and the Medical Information Mart for Intensive Care, version IV (MIMIC-IV) data sets. Our approach allowed us to (1) identify drifts effectively, especially using metrics such as the Jensen-Shannon divergence, and (2) uncover reasons for the decline in overall mortality in both the COVID-19 and MIMIC-IV data sets, as well as changes in the cooccurrence between different diseases and this particular outcome. Factors such as vaccination during the COVID-19 pandemic and reduced iatrogenic events and cancer-related deaths in MIMIC-IV were highlighted. The methodology also pinpointed shifts in patient demographics and disease patterns, providing insights into the evolving health care landscape during the study period. We developed a novel methodology combining machine learningand natural language processing techniques to detect, characterize, and understand temporal shifts in health care data. This understanding can enhance predictive algorithms, improve patient outcomes, and optimize health care resource allocation, ultimatelyimproving the effectiveness of machine learning predictive algorithms applied to health care data. Our methodology can be applied to a variety of scenarios beyond those discussed in this paper.

Enhancing long-term water quality modeling by addressing base demand, demand patterns, and temperature uncertainty using unsupervised machine learning techniques

Water quality modelling in Water Distribution systems (WDS) is frequently affected by uncertainties in input variables such as base demand and decay constants. When utilizing simulation tools like EPANET, which necessitate exact numerical inputs, these uncertainties can result in inaccurate simulations. This study proposes a novel framework that leverages unsupervised machine learning, specifically a Gaussian Mixture Model (GMMs), to represent and integrate these uncertainties in the simulation. By classifying historical water demand into fuzzy clusters, the framework allows for certain linguistic inputs (e.g., "high" or "low" demand) to be used in water quality simulations. The framework also incorporates representative hourly demand patterns and temperature-dependent chlorine decay constants based on historical data correlations. Validations were conducted on the Anytown network using WNTR-EPANET, comparing simulated chlorine residuals with Validation data from 181 steady-state simulations. The simulation through the framework achieved a Jensen-Shannon Divergence (JSD) of <0.008 across all demand clusters, indicating high similarity between predicted and actual probability distributions . In comparison to other simulation scenarios tested, which exhibited increased variability (JSD > 0.18), the proposed framework demonstrated improved accuracy in representing chlorine residual distributions. The methodology is adaptable to other systems, if similar historical datasets containing key variables, such as flow rates and temperature, are provided. While the framework offers a more flexible and accurate approach to handling uncertainties in WDS, its effectiveness is contingent upon the availability of robust historical demand and temperature data for decay constant calibration.

A Basket Trial Design Based on Power Priors

In basket trials a treatment is investigated in several subgroups. They are primarily used in oncology in early clinical phases as single-arm trials with a binary endpoint. For their analysis primarily Bayesian methods have been suggested, as they allow partial sharing of information based on the observed similarity between subgroups. Fujikawa et al. suggested an approach using empirical Bayes methods that allows flexible sharing based on easily interpretable weights derived from the Jensen-Shannon divergence between the subgroup-wise posterior distributions. We show that this design is closely related to the method of power priors and investigate several modifications of Fujikawa’s design using methods from the power prior literature. While in Fujikawa’s design, the amount of information that is shared between two baskets is only determined by their pairwise similarity, we also discuss extensions where the outcomes of all baskets are considered in the computation of the sharing weights. The results of our comparison study show that the power prior design has comparable performance to fully Bayesian designs in a range of different scenarios. At the same time, the power prior design is computationally cheap and even allows analytical computation of operating characteristics in some settings.

Should the multi-layer transportation network structure be reduced?

ABSTRACT The increasing diversity of transportation modes and the rapid expansion of transportation networks present significant challenges for modeling multi-layer comprehensive transportation networks. It is crucial to determine whether aggregating certain layers is a viable option for balancing complexity reduction and information preservation. This decision defines the layered structures and informs subsequent analyses of these networks. Two-dimensional factors, namely topological structures and transportation attributes, are considered to enhance understanding of the similarities among network layers. The relative entropy and the Gini index are employed as metrics to assess information gain or loss resulting from layer aggregation or segregation, guiding decisions on network reduction. Furthermore, an integrated similarity measure, based on the quantum Jensen-Shannon divergence and the Gower distance, is utilized to identify the optimal aggregation sequences. Two real-world transportation networks serve as case studies. Results demonstrate that these transportation networks are more effectively maintained with layer-separated structures, preserving maximum information.

Tensor multi-view clustering method for natural image segmentation

Image segmentation methods based on clustering have become a advancement in the field of computer vision, and the performance of these methods mainly depends on the performance of the clustering algorithm. However, these methods have some shortcomings. First, a single feature cannot accurately obtain the complete information of the superpixels. Second, the commonly used Euclidean distance cannot extract the nonlinear manifold structure between superpixels. Third, the high-dimensional information between superpixels is not considered. Therefore, this article proposes a Tensor Multi-view Clustering Method for Natural Image Segmentation (TMCNIS), in which firstly the multiclass features of the superpixels are extracted to obtain complete information. Secondly, the Jensen–Shannon divergence is used to delineate the relationship between superpixels to obtain a more complete nonlinear structure. Thirdly, an adaptively weighted tensor Schatten-p norm is proposed to better approximate the target rank of the tensor. It also automatically shrinks the singular values appropriately to construct a finer tensor, thus fully capturing the spatial structure in the tensor. Experimental results on four clustering datasets and two image segmentation datasets demonstrate the superiority of TMCNIS compared to existing methods.

Assessing clustering methods using Shannon's entropy

Unsupervised clustering techniques are crucial for effectively partitioning datasets into meaningful subgroups. In this paper, we introduce a novel clustering fuzziness metric based on Shannon's entropy, which quantifies the level of uncertainty in clustering assignments. This metric is employed to develop a statistical test for evaluating clustering algorithms and to propose parametric corrections to enhance their performance. We provide empirical evidence showing that our approach significantly improves clustering quality compared to well-known algorithms such as Fuzzy K-Means (FKM) and Fanny. For instance, applying our correction method led to a notable decrease in Jensen-Shannon Divergence (JSD) values, indicating enhanced clustering accuracy. Our methodology is demonstrated through comprehensive experiments on both simulated and real-world datasets. Additionally, our approach proves effective in determining the optimal number of clusters, showing superiority over traditional methods. These results highlight the practical benefits of our metric and correction techniques for improving clustering performance.

Multi-Regional Pelvic Floor Muscle Function Diagnosis System Based on Inflatable Stretchable Electrode Array.

Effective prevention and treatment of pelvic floor dysfunction (PFD) necessitates the identification of lesions within the complex pelvic floor muscle (PFM) groups associated with various symptoms. Here, we developed a multi-region pelvic floor muscle functional diagnosis system (MPDS) based on an inflatable stretchable electrode array, which aids in accurately locating areas related to PFD. Clinical diagnostic experiments were conducted on 56 patients with postpartum stress urinary incontinence (PSUI) and 73 postpartum asymptomatic controls. MPDS collects pelvic floor electromyography from all participants. By assessing EMG parameters such as activation time differences (ATD) and using Jensen-Shannon (JS) divergence to verify, with the aim of locating target muscle groups with functional abnormalities. Clinical test results showed that by observing the AT sequence of the PSUI group and the control group, muscle groups with functional abnormalities in the Pubococcygeus muscle (PC) and Puborectalis muscle (PR) regions could be preliminarily diagnosed. In the assessment of regional muscle contribution values based on JS divergence, it was verified that the contribution values of rapid contraction in the PC and PR regions of the PSUI group were relatively lower compared to those of the control group, which correlated with urinary control dysfunction. These experiments demonstrate that the MPDS helps in accurately locating target muscle groups with functional abnormalities, showcasing its potential in precise assessment of complex muscle groups such as PFM, which may improve diagnostic precision and reliability.

LDAPrototype: a model selection algorithm to improve reliability of latent Dirichlet allocation.

Latent Dirichlet allocation (LDA) is a popular method for analyzing large text corpora, but it suffers from instability due to its reliance on random initialization. This results in different outcomes for replicated runs, hindering reproducibility. To address this, we introduce LDAPrototype, a new approach for selecting the most representative LDA run from multiple replications on the same dataset. LDAPrototype enhances the reliability of LDA conclusions by ensuring greater similarity between replications compared to traditional LDA runs or models chosen based on perplexity or NPMI. A key feature of LDAPrototype is its use of a novel model similarity measure called S-CLOP (Similarity of multiple sets by Clustering with LOcal Pruning). It is based on topic similarities, for which we compare the usage of measures like the thresholded Jaccard coefficient, cosine similarity, Jensen-Shannon divergence, and rank-biased overlap. The effectiveness of LDAPrototype is demonstrated through its application to six real datasets, including newspaper articles and tweets. The results show improved reproducibility and reliability in topic modeling outcomes. LDAPrototype's approach is noteworthy for its practical applicability, comprehensibility, ease of implementation, and computational efficiency. Furthermore, the algorithm's concept can be generalized to other topic modeling procedures that characterize topics through word distributions, making it a versatile tool in text data analysis.

Steered quantum coherence and entropic uncertainty relation in the cluster Ising model

Abstract In this paper, we investigate the cluster Ising model (CIM) via steered quantum coherence (SQC) and entropic uncertainty relation (EUR). We present the behavior of SQC quantified by the L1 norm, relative entropy and quantum Jensen–Shannon divergence. We also demonstrate the properties of EUR in the CIM. In addition, we provide a comparative analysis of these measures and present detailed numerical results.

Enhancing fruit and vegetable detection in unconstrained environment with a novel dataset

Automating the detection of fruits and vegetables using computer vision is essential for modernizing agriculture, improving efficiency, ensuring food quality, and contributing to sustainable and technologically advanced farming practices. This paper presents an end-to-end pipeline for detecting and localizing fruits and vegetables in real-world scenarios. To achieve this, a dataset named FRUVEG67 was curated that includes images of 67 classes of fruits and vegetables captured in unconstrained scenarios, with only a few manually annotated samples per class. A semi-supervised data annotation algorithm (SSDA) was developed that generates bounding boxes for objects to label the remaining nonannotated images. For detection, Fruit and Vegetable Detection Network (FVDNet) was proposed, an ensemble version of YOLOv8n featuring three distinct grid configurations. In addition, an averaging approach for the prediction of the bounding box and a voting mechanism for the prediction of the classes was implemented. Finally Jensen–Shannon Divergence (JSD) in conjunction with focal loss was integrated as the overall loss function for better detection of smaller objects. Experimental results highlight the superiority of FVDNet compared to recent versions of YOLO, showcasing remarkable improvements in detection and localization performance. An impressive mean average precision (mAP) score of 0.82 across all classes was achieved. Furthermore, the efficacy of FVDNet on open-category refrigerator images were evaluated, where it demonstrates promising results.

Metabolic network connectivity disturbances in Parkinson's disease: a novel imaging biomarker.

The diagnosis of Parkinson's Disease (PD) presents ongoing challenges. Advances in imaging techniques like 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) have highlighted metabolic alterations in PD, yet the dynamic network interactions within the metabolic connectome remain elusive. To this end, we examined a dataset comprising 49 PD patients and 49 healthy controls. By employing a personalized metabolic connectome approach, we assessed both within- and between-network connectivities using Standard Uptake Value (SUV) and Jensen-Shannon Divergence Similarity Estimation (JSSE). A random forest algorithm was utilized to pinpoint key neuroimaging features differentiating PD from healthy states. Specifically, the results revealed heightened internetwork connectivity in PD, specifically within the somatomotor (SMN) and frontoparietal (FPN) networks, persisting after multiple comparison corrections (P < 0.05, Bonferroni adjusted for 10% and 20% sparsity). This altered connectivity effectively distinguished PD patients from healthy individuals. Notably, this study utilizes 18F-FDG PET imaging to map individual metabolic networks, revealing enhanced connectivity in the SMN and FPN among PD patients. This enhanced connectivity may serve as a promising imaging biomarker, offering a valuable asset for early PD detection.