Kullback-Leibler Research Articles

Incipient fault detection and isolation with Cauchy–Schwarz divergence: A probabilistic approach

To monitor the dynamics and non-stationarity inherent in industrial processes, we propose a novel incipient fault detection and isolation scheme grounded in a probabilistic perspective, using the Cauchy–Schwarz (CS) divergence. Our innovation lies in the utilization of marginal CS divergence for incipient fault detection and the conditional CS divergence for fault isolation. This approach neither require prior parametric assumptions about the underlying data distribution nor depend on historical fault data, while simultaneously providing explanatory diagnostics. Beyond this, we develop a change point detection-base diagnosis technique for practical engineering applications. This online process monitoring technique guarantees timely intervention to uphold process stability and safety. We demonstrate the compelling performance, higher detection rate and lower alarm rate, of the CS divergence over prevalent divergence-based approaches, such as Kullback–Leibler divergence, Wasserstein distance and Mahalanobis distance. We also illustrate the explanatory insights offered by conditional CS divergence in fault isolation on synthetic data, benchmarks of continuous stirred-tank reactor process and continuous stirred-tank heater process and even a real-world continuous catalytic reforming process. Code of this CS divergence based fault detection and isolation is available at https://github.com/Feiya-Lv/Incipient-Fault-Detection-and-Isolation-with-Cauchy--Schwarz-Divergence.

Kullback-Leibler Control in Boolean Control Networks.

This article addresses the Kullback-Leibler (KL) control problem in Boolean control networks. In the considered problem, an extended stage cost function depending on the control inputs is introduced; in contrast to a stage cost of the conventional KL control problems in the Markov decision process cannot take into consideration the control inputs. An associated Bellman equation and a matrix-based iteration algorithm are presented. The theoretical analysis shows that the proposed KL control results in an approximated form of conventional dynamic programming (DP). Furthermore, the convergence analysis is presented, with the weight parameter converging to zero and diverging to infinity. In practical application examples, a comparison of the conventional DP and proposed KL control is illustrated.

The Rosenzweig–Porter model revisited for the three Wigner–Dyson symmetry classes

Interest in the Rosenzweig–Porter model, a parameter-dependent random-matrix model which interpolates between Poisson and Wigner–Dyson (WD) statistics describing the fluctuation properties of the eigenstates of typical quantum systems with regular and chaotic classical dynamics, respectively, has come up again in recent years in the field of many-body quantum chaos. The reason is that the model exhibits parameter ranges in which the eigenvectors are Anderson-localized, non-ergodic (fractal) and ergodic extended, respectively. The central question is how these phases and their transitions can be distinguished through properties of the eigenvalues and eigenvectors. We present numerical results for all symmetry classes of Dyson’s threefold way. We analyzed the fluctuation properties in the eigenvalue spectra, and compared them with existing and new analytical results. Based on these results we propose characteristics of the short- and long-range correlations as measures to explore the transition from Poisson to WD statistics. Furthermore, we performed in-depth studies of the properties of the eigenvectors in terms of the fractal dimensions, the Kullback–Leibler (KL) divergences and the fidelity susceptibility. The ergodic and Anderson transitions take place at the same parameter values and a finite size scaling analysis of the KL divergences at the transitions yields the same critical exponents for all three WD classes, thus indicating superuniversality of these transitions.

A fuzzy control algorithm based on artificial intelligence for the fusion of traditional Chinese painting and AI painting

Recently, artificial intelligence (AI)-generated resources have gained popularity because of their high effectiveness and reliability in terms of output and capacity to be customized and broadened, especially in image generation. Traditional Chinese paintings (TCPs) are incomplete because their color contrast is insufficient, and object reality is minimal. However, combining AI painting (AIP) with TCP remains inadequate and uncertain because of image features such as patterns, styles, and color. Hence, an algorithm named variational fusion-based fuzzy accelerated painting (VF2AP) has been proposed to resolve this challenge. Initially, the collected TCP data source is applied for preprocessing to convert it into a grayscale image. Then, the feature extraction process is performed via fuzzy-based local binary pattern (FLBP) and brushstroke patterns to enhance the fusion of intelligent fuzzy logic to optimize the local patterns of textures in a noisy image. Second, the extracted features are used as inputs to the variational autoencoder (VAE), which is used to avoid latent space irregularities in the image and the reconstructed image by maintaining minimum reconstruction loss. Third, fuzzy inference rules are applied to avoid variation in the fusion process of the reconstructed and original images. Fourth, the feedback mechanism is designed with evaluation metrics such as area under the curve-receiver operating characteristic (AUC-ROC) analysis, mean square error (MSE), structural similarity index (SSIM), and Kullback‒Leibler (KL) divergence to enhance the viewer's understanding of fused painting images.

Bayesian neural network and Bayesian physics-informed neural network via variational inference for seismic petrophysical inversion

Bayesian neural network via variational inference (BNN-VI) aims to estimate probability distribution of the training parameters of the neural network, which are difficult to compute with traditional methods, by proposing a family of densities and finding the candidates that are close to the target. Variational inference is a Bayesian method that provides a faster tool than Markov chain Monte Carlo sampling algorithms. However, to provide an accurate estimation of the most likely models, BNN-VI requires sufficient training data, which is not always available in geophysical inverse problems. We propose a method that combines a Bayesian approach with a physics-informed neural networks algorithms via variational inference, namely Bayesian physics-informed neural networks via variational inference (BPINN-VI) and apply it to a geophysical inverse problem for the estimation of petrophysical properties from seismic data. The method is based on a Bayesian network approach in which the objective function is defined by the Kullback–Leibler (KL) divergence. In addition, the physical relations between data (seismic measurements) and model variables (petrophysical properties) are embedded into neural networks through a physics-dependent loss term. We tested the proposed method on a pre-stack seismic dataset from the North Sea with limited direct measurements of the target petrophysical properties from well logs. The outcome is the most likely model of petrophysical properties. Compared to BNN-VI, the BPINN-VI results show higher correlation and R2 coefficients, and lower mean square error as well as lower uncertainty and higher lateral continuity.

VB-T PHD-SLAM: efficient SLAM under heavy-tailed noise

To address the challenge of simultaneous localization and mapping (SLAM) in the presence of heavy-tailed noise, this paper introduces a robust probability hypothesis density (PHD) SLAM algorithm. This algorithm models measurement noise using the Student's t-distribution, which better captures the heavy-tailed nature of the noise. Since the prior density is assumed to be Gaussian mixture form, the posterior density is no longer Gaussian mixture form after the likelihood update of the t-distribution. A variational Bayesian approach is employed to ensure computable multi-target densities during filtering, minimizing the Kullback-Leibler divergence to obtain an approximate solution for the new marginal likelihood function. Then a new closed-form recursion of PHD-SLAM is derived by using t-distribution. Simulation results and real-world validations demonstrate that the proposed algorithm outperforms PHD-SLAM 1.0 and PHD-SLAM 2.0 in terms of both localization and mapping accuracy while maintaining computational efficiency in SLAM scenarios affected by heavy-tailed noise.

Adaptive EMG decomposition in dynamic conditions based on online learning metrics with tunable hyperparameters

Objective. Developing neural decoders robust to non-stationary conditions is essential to ensure their long-term accuracy and stability. This is particularly important when decoding the neural drive to muscles during dynamic contractions, which pose significant challenges for stationary decoders. Approach. We propose a novel adaptive electromyography (EMG) decomposition algorithm that builds on blind source separation methods by leveraging the Kullback–Leibler divergence and kurtosis of the signals as metrics for online learning. The proposed approach provides a theoretical framework to tune the adaptation hyperparameters and compensate for non-stationarities in the mixing matrix, such as due to dynamic contractions, and to identify the underlying motor neuron (MN) discharges. The adaptation is performed in real-time (∼22 ms of computational time per 100 ms batches). Main results. The hyperparameters of the proposed adaptation captured anatomical differences between recording locations (forearm vs wrist) and generalised across subjects. Once optimised, the proposed adaptation algorithm significantly improved all decomposition performance metrics with respect to the absence of adaptation in a wide range of motion of the wrist (80∘). The rate of agreement, sensitivity, and precision were ⩾90% in ⩾80% of the cases in both simulated and experimentally recorded data, according to a two-source validation approach. Significance. The findings demonstrate the suitability of the proposed online learning metrics and hyperparameter optimisation to compensate the induced modulation and accurately decode MN discharges in dynamic conditions. Moreover, the study proposes an experimental validation method for EMG decomposition in dynamic tasks.

Understanding Higher-Order Interactions in Information Space.

Methods used in topological data analysis naturally capture higher-order interactions in point cloud data embedded in a metric space. This methodology was recently extended to data living in an information space, by which we mean a space measured with an information theoretical distance. One such setting is a finite collection of discrete probability distributions embedded in the probability simplex measured with the relative entropy (Kullback-Leibler divergence). More generally, one can work with a Bregman divergence parameterized by a different notion of entropy. While theoretical algorithms exist for this setup, there is a paucity of implementations for exploring and comparing geometric-topological properties of various information spaces. The interest of this work is therefore twofold. First, we propose the first robust algorithms and software for geometric and topological data analysis in information space. Perhaps surprisingly, despite working with Bregman divergences, our design reuses robust libraries for the Euclidean case. Second, using the new software, we take the first steps towards understanding the geometric-topological structure of these spaces. In particular, we compare them with the more familiar spaces equipped with the Euclidean and Fisher metrics.

Class incremental learning with KL constraint and multi-strategy exemplar selection for classification based on MMFA model

Class incremental learning (CIL) can learn new classes continuously by updating the model rather than retraining a model from scratch with all seen classes like traditional offline learning, therefore, CIL is more suitable for classification in dynamic environments, where new classes are captured progressively. However, the key knowledge of old classes will be lost due to the update mode of CIL, leading to the catastrophic forgetting (CF) problem. In this paper, a novel CIL method with Kullback-Leibler constraint and multi-strategy exemplar selection (CIL-KLMES) is proposed for classification based on max-margin factor analysis (MMFA) model. To handle the CF problem, CIL-KLMES imposes a Kullback-Leibler (KL) divergence term on the important parameters when updating the model to restrict the parameters' distribution to be similar, thus preventing the updated model from deviating too much from the previous model and preserving the knowledge of old classes. Moreover, CIL-KLMES selects a few representative exemplars from old classes based on the robust description of data distribution and classification decision boundary. By replaying representative exemplars to update the model together with new class data, the key knowledge of old classes can be further preserved. Therefore, the CF problem can be alleviated sufficiently. Experimental results demonstrate the effectiveness of CIL-KLMES.

Partial Bayes Estimation in Two-Parameter Rayleigh Distribution

In Bayesian paradigm, when we have no proper information regarding the joint parameters of the model of the variable and the aim is to estimate only one of them in presence of another nuisance parameter, we introduce an estimation approach known as ‘Partial Bayes estimation’. Here, two-parameter Rayleigh distribution is considered with location parameter μ and scale parameter λ and both are unknown. Partial Bayes estimates of the parameter of interest λ are derived both under non-informative uniform prior and informative gamma prior by plugging the estimate of another parameter μ , obtained by some other classical method using ordered samples. As, the maximum likelihood estimate of the parameters cannot be obtained explicitly, we also propose an estimation method closed to maximum likelihood estimate using the ordered samples. We use Kullback-Leibler distance loss derived using two Rayleigh density functions, precautionary loss and weighted squared error loss functions to compute the risk values. Simulation studies are performed to assess the performance of the Partial Bayes estimates through integrated risks under different loss functions. Two real datasets are used to show the applicability of the proposed Partial Bayes estimation technique.

Quantifying Quantum Chaos through Microcanonical Distributions of Entanglement

A characteristic feature of “quantum chaotic” systems is that their eigenspectra and eigenstates display universal statistical properties described by random matrix theory (RMT). However, eigenstates of local systems also encode structure beyond RMT. To capture this feature, we introduce a framework that allows us to compare the properties of eigenstates in local systems with those of pure random states. In particular, our framework defines a notion of distance between quantum state ensembles that utilizes the Kullback-Leibler divergence to compare the microcanonical distribution of entanglement entropy (EE) of eigenstates with a reference RMT distribution generated by pure random states (with appropriate constraints). This notion gives rise to a quantitative metric for quantum chaos that not only accounts for averages of the distributions but also higher moments. The differences in moments are compared on a highly resolved scale set by the standard deviation of the RMT distribution, which is exponentially small in system size. As a result, the metric can distinguish between chaotic and integrable behaviors and, in addition, quantify and compare the of chaos (in terms of proximity to RMT behavior) between two systems that are assumed to be chaotic. We implement our framework in local, minimally structured, Floquet random circuits, as well as a canonical family of many-body Hamiltonians, the mixed-field Ising model (MFIM). Importantly, for Hamiltonian systems, we find that the reference random distribution must be appropriately constrained to incorporate the effect of energy conservation in order to describe the ensemble properties of midspectrum eigenstates. The metric captures deviations from RMT across all models and parameters, including those that have been previously identified as strongly chaotic, and for which other diagnostics of chaos such as level spacing statistics look strongly thermal. In Floquet circuits, the dominant source of deviations is the second moment of the distribution, and this persists for all system sizes. For the MFIM, we find significant variation of the KL divergence in parameter space. Notably, we find a small region where deviations from RMT are minimized, suggesting that “maximally chaotic” Hamiltonians may exist in fine-tuned pockets of parameter space. Published by the American Physical Society 2024

Unveiling Malicious Network Flows Using Benford’s Law

The increasing proliferation of cyber-attacks threatening the security of computer networks has driven the development of more effective methods for identifying malicious network flows. The inclusion of statistical laws, such as Benford’s Law, and distance functions, applied to the first digits of network flow metadata, such as IP addresses or packet sizes, facilitates the detection of abnormal patterns in the digits. These techniques also allow for quantifying discrepancies between expected and suspicious flows, significantly enhancing the accuracy and speed of threat detection. This paper introduces a novel method for identifying and analyzing anomalies within computer networks. It integrates Benford’s Law into the analysis process and incorporates a range of distance functions, namely the Mean Absolute Deviation (MAD), the Kolmogorov–Smirnov test (KS), and the Kullback–Leibler divergence (KL), which serve as dispersion measures for quantifying the extent of anomalies detected in network flows. Benford’s Law is recognized for its effectiveness in identifying anomalous patterns, especially in detecting irregularities in the first digit of the data. In addition, Bayes’ Theorem was implemented in conjunction with the distance functions to enhance the detection of malicious traffic flows. Bayes’ Theorem provides a probabilistic perspective on whether a traffic flow is malicious or benign. This approach is characterized by its flexibility in incorporating new evidence, allowing the model to adapt to emerging malicious behavior patterns as they arise. Meanwhile, the distance functions offer a quantitative assessment, measuring specific differences between traffic flows, such as frequency, packet size, time between packets, and other relevant metadata. Integrating these techniques has increased the model’s sensitivity in detecting malicious flows, reducing the number of false positives and negatives, and enhancing the resolution and effectiveness of traffic analysis. Furthermore, these techniques expedite decisions regarding the nature of traffic flows based on a solid statistical foundation and provide a better understanding of the characteristics that define these flows, contributing to the comprehension of attack vectors and aiding in preventing future intrusions. The effectiveness and applicability of this joint method have been demonstrated through experiments with the CICIDS2017 public dataset, which was explicitly designed to simulate real scenarios and provide valuable information to security professionals when analyzing computer networks. The proposed methodology opens up new perspectives in investigating and detecting anomalies and intrusions in computer networks, which are often attributed to cyber-attacks. This development culminates in creating a promising model that stands out for its effectiveness and speed, accurately identifying possible intrusions with an F1 of nearly 80%, a recall of 99.42%, and an accuracy of 65.84%.

Neural Schrödinger bridge for unpaired real-world image deraining

Given the significant differences between domains, current unpaired learning methods struggle to accurately map the relationship between rainy and clear images. To this end, we introduce a neural Schrödinger bridge (NSB) for unpaired real-world image deraining, which utilizes the stochastic differential equations to capture the mapping relationships between rainy and clear domains. Meanwhile, we frame the deraining process as a Lagrangian problem using the Kullback-Leibler divergence between the data distribution and the model distribution. Additionally, by leveraging the capabilities of the contrastive language-image pre-training model (CLIP), our research shows that the CLIP prior helps differentiate between rainy and clear images. Building on this, we reformulate the Schrödinger bridge problem as a series of adversarial learning tasks using both image and prompt representations. To our knowledge, our approach is the first to use the Schrödinger bridge in unpaired image deraining. Extensive experiments show that our proposed NSB model outperforms existing unpaired deraining methods in both quantitative and qualitative evaluations.

Automated segmentation of MRI white matter hyperintensities in 8,421 patients with acute ischemic stroke.

To date, only a few small studies have attempted deep learning-based automatic segmentation of white matter hyperintensity (WMH) lesions in patients with cerebral infarction, which is complicated because stroke-related lesions can obscure WMH borders. We developed and validated deep learning algorithms to segment WMH lesions accurately in patients with cerebral infarction, using multisite datasets involving 8,421 patients with acute ischemic stroke. We included 8,421 stroke patients from 9 centers in Korea. 2D UNet and SE-Unet models were trained using 2,408 FLAIR MRI from 3 hospitals and validated using 6,013 FLAIR MRIs from 6 hospitals. WMH segmentation performance was assessed by calculating DSC, correlation coefficient, and concordance correlation coefficient compared to a human-segmented gold standard. In addition, we obtained an uncertainty index that represents overall ambiguity in the voxel classification for WMH segmentation in each patient based on the Kullback-Leibler divergence. In the training dataset, the mean age was 67.4±13.0 years and 60.4% were men. The mean (95% CI) DSCs for Unet in internal testing and external validation were respectively 0.659 (0.649-0.669) and 0.710 (0.707-0.714), which were slightly lower than the reliability between humans (DSC=0.744; 95% CI=0.738-0.751; P=.031). Compared with the Unet, the SE-Unet demonstrated better performance, achieving a mean DSC of 0.675 (0.666-0.685; P<.001) in the internal testing and 0.722 (0.719-0.726; P<.001) in the external validation; moreover, it achieved high DSC values (ranging from 0.672 to 0.744) across multiple validation datasets. We observed a significant correlation between WMH volumes that were segmented automatically and manually for the Unet (r=0.917, P<.0001) and even stronger for the SE-Unet (r=0.933, P<.0001). The SE-Unet also attained a high concordance correlation coefficient (ranging from 0.841 to 0.956) in external test datasets. In addition, the uncertainty indices in the majority of patients (86%) in the external datasets were below 0.35, with an average DSC of 0.744 in these patients. We developed and validated deep learning algorithms to segment WMH in patients with acute cerebral infarction using the largest-ever MRI datasets. In addition, we showed that the uncertainty index can be used to identify cases where automatic WMH segmentation is less accurate and requires human review. WMH = white matter hyperintensity; CNN = convolutional neural networks; SE = squeeze-and-excitation; KL = Kullback-Leibler; ReLU = rectified linear unit; LKW = last known well; mRS = modified Rankin Scale; NIHSS = National Institute of Health Stroke Scale; LAA = large artery atherosclerosis; SVO = small vessel occlusion; CE = cardioembolism.

Distributionally robust planning for power distribution network considering multi-energy station enabled integrated demand response

—To copy with the peak valley difference and peak load problem of power distribution network (PDN), this paper proposes a distributionally robust planning model for PDN considering the flexible management via the multi-energy station (MES) based integrated demand response (IDR). The planning covers reinforcement of the existing substation and feeders as well as the investment of MES and IDR. The IDR involves the incentive interruptible load response (ILR), transferable load response (TLR) and price-based power-gas substitution (PGS) which is implemented on MES. The MES integrates multiple energy storages and energy coupling devices, which is served as a platform for PDN operator to centrally track the final demand and improve the response potential of IDR. The proposed planning method aims at minimizing the present value of PDN investment cost and operation cost during the planning period. To handle the uncertainty of PDN end-use loads, a two-stage Kullback–Leibler divergence (KLD)-based distributionally robust optimization (DRO) planning model is formulated. Finally, the columns and constraints generation algorithm is utilized to solve the KLD-DRO planning model. Case studies are carried out on a practical 152-bus urban distribution system to validate the effectiveness of the proposed methodology and quantitatively analyze the key influence factors of PDN investment.

BFLIDS: Blockchain-Driven Federated Learning for Intrusion Detection in IoMT Networks.

The Internet of Medical Things (IoMT) has significantly advanced healthcare, but it has also brought about critical security challenges. Traditional security solutions struggle to keep pace with the dynamic and interconnected nature of IoMT systems. Machine learning (ML)-based Intrusion Detection Systems (IDS) have been increasingly adopted to counter cyberattacks, but centralized ML approaches pose privacy risks due to the single points of failure (SPoFs). Federated Learning (FL) emerges as a promising solution, enabling model updates directly on end devices without sharing private data with a central server. This study introduces the BFLIDS, a Blockchain-empowered Federated Learning-based IDS designed to enhance security and intrusion detection in IoMT networks. Our approach leverages blockchain to secure transaction records, FL to maintain data privacy by training models locally, IPFS for decentralized storage, and MongoDB for efficient data management. Ethereum smart contracts (SCs) oversee and secure all interactions and transactions within the system. We modified the FedAvg algorithm with the Kullback-Leibler divergence estimation and adaptive weight calculation to boost model accuracy and robustness against adversarial attacks. For classification, we implemented an Adaptive Max Pooling-based Convolutional Neural Network (CNN) and a modified Bidirectional Long Short-Term Memory (BiLSTM) with attention and residual connections on Edge-IIoTSet and TON-IoT datasets. We achieved accuracies of 97.43% (for CNNs and Edge-IIoTSet), 96.02% (for BiLSTM and Edge-IIoTSet), 98.21% (for CNNs and TON-IoT), and 97.42% (for BiLSTM and TON-IoT) in FL scenarios, which are competitive with centralized methods. The proposed BFLIDS effectively detects intrusions, enhancing the security and privacy of IoMT networks.

How well can persistent contrails be predicted? An update

Abstract. The total aviation effective radiative forcing is dominated by non-CO2 effects. The largest contributors to the non-CO2 effects are contrails and contrail cirrus. There is the possibility of reducing the climate effect of aviation by avoiding flying through ice-supersaturated regions (ISSRs), where contrails can last for hours (so-called persistent contrails). Therefore, a precise prediction of the specific location and time of these regions is needed. But a prediction of the frequency and degree of ice supersaturation (ISS) on cruise altitudes is currently very challenging and associated with great uncertainties because of the strong variability in the water vapour field, the low number of humidity measurements at the air traffic altitude, and the oversimplified parameterisations of cloud physics in weather models. Since ISS is more common in some dynamical regimes than in others, the aim of this study is to find variables/proxies that are related to the formation of ISSRs and to use these in a regression method to predict persistent contrails. To find the best-suited proxies for regressions, we use various methods of information theory. These include the log-likelihood ratios, known from Bayes' theorem, a modified form of the Kullback–Leibler divergence, and mutual information. The variables (the relative humidity with respect to ice, RHiERA5; the temperature, T; the vertical velocity, ω; the divergence, DIV; the relative vorticity, ζ; the potential vorticity, PV; the normalised geopotential height, Z; and the local lapse rate, γ) come from ERA5, and RHiM/I, which we assume as the truth, comes from MOZAIC/IAGOS (Measurement of Ozone and Water Vapour on Airbus In-service Aircraft/In-service Aircraft for a Global Observing System; commercial aircraft measurements). It turns out that RHiERA5 is the most important predictor of ice supersaturation, in spite of its weaknesses, and all other variables do not help much to achieve better results. Without RHiERA5, a regression to predict ISSRs is not successful. Certain modifications of RHiERA5 before the regression (as suggested in recent papers) do not lead to improvements of ISSR prediction. Applying a sensitivity study with artificially modified RHiERA5 distributions points to the origin of the problems with the regression: the conditional distributions of RHiERA5 (conditioned on ISS and non-ISS, from RHiM/I) overlap too heavily in the range of 70 %–100 %, so for any case in that range, it is not clear whether it belongs to an ISSR or not. Evidently, this renders the prediction of contrail persistence very difficult.

NEW APPROACH TOWARDS THE PROBLEM OF SOMATIC CRITERIA OF SECULAR TRENDS. SEXUAL SOMATIC DIMORPHISM

Introduction. The goal of study is description of secular dynamics of sex dimorphism (SD) of height, weight, chest girth of children aged 9 years and teenagers 13 years old from Russia and neighboring countries through 1920–2010. Materials and methods. The main source of data — materials of books on physical development of children and adolescents from cities and rural regions of Russia and USSR, collected and processed according to the universal methodical standards of Research Institute of Hygiene of children and adolescents. The quantitative estimation of SD value was held using Kullback divergence. One-year age groups, contrastive in biological essence, are under discussion. The estimation of the vector and level of associations of SD of anthropometric traits and secular factor was held using correlation analysis. The analysis of local patterns of secular SD dynamics of separate traits was carried out. Results. Secular variability of height and weight SD of 9-year-old children is not significant. While secular dynamics of three body dimensions of 13-year-old teenagers is significant, the highest determination coefficient is fixed for weight. For Moscow samples the decrease of height and weight SD is fixed through the historic interval 1960s – 70s with the following recovery up to the level of 1920s in 2000s. Temporal dynamics of SD of 9-year-old children of St. Petersburg through the interval 1960s – 2000s consists of macrosomization of girls; the tendency for 13-year-old teenagers is opposite. Temporal dynamics of somatic SD of 9-year-old Russian children of Kazan reveals stability through the interval 1960th-1990th; for 13-year-old teenagers’ parameters of transversal body development demonstrate secular stability, as to height – boys compensate their lag compared to girls. For Kirovsk city (Murmansk region) through the interval 1950s – 1970s temporal fluctuations of height and weight SD of 9-year-old children are less than 0,2 standard deviations; height and weight SD of 13-year-old teenagers reveal temporal increase (0,4–0,6 standard deviations). Conclusion. The results confirm the importance of somatic SD as the criterion of secular dynamics for 13-year-old teenagers and witness, as a whole, to the temporal stability of SD during second childhood (9 years). Local differences do not contribute significantly to the intensity of secular dynamics of SD. The process of secular macrosomization of one sex as compared to the other is not the exclusive prerogative of boys and male teenagers.

An uncertainty-aware domain adaptive semantic segmentation framework

Semantic segmentation is significant to realize the scene understanding of autonomous driving. Due to the lack of annotated real-world data, the technology of domain adaptation is applied so that the model is trained on the synthetic data and inferred on the real data. However, this domain gap leads to aleatoric and epistemic uncertainty. These uncertainties link to the potential safety issue of autonomous driving in normal weather and adverse weather. In this study, we explore the scientific problem that has received sparse attention previously. We postulate that the Dual Attention module can mitigate the uncertainty in the task of semantic segmentation and provide some empirical study to validate it. Furthermore, the utilization of Kullback-Leibler divergence (KL divergence) helps the estimation of aleatoric uncertainty and boosts the robustness of the segmentation model. Our empirical study on the diverse datasets of semantic segmentation demonstrates the effectiveness of our method in normal and adverse weather. Our code is available at: https://github.com/liubo629/Seg-Uncertainty-dual-attention.

Hardness and Approximability of Dimension Reduction on the Probability Simplex

Dimension reduction is a technique used to transform data from a high-dimensional space into a lower-dimensional space, aiming to retain as much of the original information as possible. This approach is crucial in many disciplines like engineering, biology, astronomy, and economics. In this paper, we consider the following dimensionality reduction instance: Given an n-dimensional probability distribution p and an integer m&lt;n, we aim to find the m-dimensional probability distribution q that is the closest to p, using the Kullback–Leibler divergence as the measure of closeness. We prove that the problem is strongly NP-hard, and we present an approximation algorithm for it.