KL Divergence Research Articles

Deconstructing Deep Active Inference: A Contrarian Information Gatherer.

Active inference is a theory of perception, learning, and decision making that can be applied to neuroscience, robotics, psychology, and machine learning. Recently, intensive research has been taking place to scale up this framework using Monte Carlo tree search and deep learning. The goal of this activity is to solve more complicated tasks using deep active inference. First, we review the existing literature and then progressively build a deep active inference agent as follows: we (1) implement a variational autoencoder (VAE), (2) implement a deep hidden Markov model (HMM), and (3) implement a deep critical hidden Markov model (CHMM). For the CHMM, we implemented two versions, one minimizing expected free energy, CHMM[EFE] and one maximizing rewards, CHMM[reward]. Then we experimented with three different action selection strategies: the ε-greedy algorithm as well as softmax and best action selection. According to our experiments, the models able to solve the dSprites environment are the ones that maximize rewards. On further inspection, we found that the CHMM minimizing expected free energy almost always picks the same action, which makes it unable to solve the dSprites environment. In contrast, the CHMM maximizing reward keeps on selecting all the actions, enabling it to successfully solve the task. The only difference between those two CHMMs is the epistemic value, which aims to make the outputs of the transition and encoder networks as close as possible. Thus, the CHMM minimizing expected free energy repeatedly picks a single action and becomes an expert at predicting the future when selecting this action. This effectively makes the KL divergence between the output of the transition and encoder networks small. Additionally, when selecting the action down the average reward is zero, while for all the other actions, the expected reward will be negative. Therefore, if the CHMM has to stick to a single action to keep the KL divergence small, then the action down is the most rewarding. We also show in simulation that the epistemic value used in deep active inference can behave degenerately and in certain circumstances effectively lose, rather than gain, information. As the agent minimizing EFE is not able to explore its environment, the appropriate formulation of the epistemic value in deep active inference remains an open question.

BEATRICE: Bayesian fine-mapping from summary data using deep variational inference.

We introduce a novel framework BEATRICE to identify putative causal variants from GWAS statistics. Identifying causal variants is challenging due to their sparsity and high correlation in the nearby regions. To account for these challenges, we rely on a hierarchical Bayesian model that imposes a binary concrete prior on the set of causal variants. We derive a variational algorithm for this fine-mapping problem by minimizing the KL divergence between an approximate density and the posterior probability distribution of the causal configurations. Correspondingly, we use a deep neural network as an inference machine to estimate the parameters of our proposal distribution. Our stochastic optimization procedure allows us to sample from the space of causal configurations, which we use to compute the posterior inclusion probabilities and determine credible sets for each causal variant. We conduct a detailed simulation study to quantify the performance of our framework against two state-of-the-art baseline methods across different numbers of causal variants and noise paradigms, as defined by the relative genetic contributions of causal and noncausal variants. We demonstrate that BEATRICE achieves uniformly better coverage with comparable power and set sizes, and that the performance gain increases with the number of causal variants. We also show the efficacy BEATRICE in finding causal variants from the GWAS study of Alzheimer's disease. In comparison to the baselines, only BEATRICE can successfully find the APOE ϵ2 allele, a commonly associated variant of Alzheimer's. BEATRICE is available for download at https://github.com/sayangsep/Beatrice-Finemapping.

Detection and Mitigation of Label-Flipping Attacks in FL Systems With KL Divergence

Detection and Mitigation of Label-Flipping Attacks in FL Systems With KL Divergence

Diffusion model conditioning on gaussian mixture model and negative gaussian mixture gradient

Diffusion models (DMs) are a type of generative model that has had a significant impact on image synthesis and beyond.They can incorporate a wide variety of conditioning inputs—such as text or bounding boxes—to guide generation. In this work, we introduce a novel conditioning mechanism that applies Gaussian mixture models (GMMs) for feature conditioning, which helps steer the denoising process in DMs. Drawing on set theory, our comprehensive theoretical analysis reveals that the conditional latent distribution based on features differs markedly from that based on classes. Consequently, feature-based conditioning tends to generate fewer defects than class-based conditioning. Experiments are designed and carried out and the experimental results support our theoretical findings as well as effectiveness of proposed feature conditioning mechanism. Additionally, we propose a new gradient function named the Negative Gaussian Mixture Gradient (NGMG) and incorporate it into the training of diffusion models alongside an auxiliary classifier. We theoretically demonstrate that NGMG offers comparable advantages to the Wasserstein distance, serving as a more effective cost function when learning distributions supported by low-dimensional manifolds, especially in contrast to many likelihood-based cost functions, such as KL divergences.

Mix-DDPM: Enhancing Diffusion Models through Fitting Mixture Noise with Global Stochastic Offset

Denoising diffusion probabilistic models (DDPM) have shown impressive performance in various domains as a class of deep generative models. In this article, we introduce the mixture noise-based DDPM (Mix-DDPM), which considers the Markov diffusion posterior as a Gaussian mixture model. Specifically, Mix-DDPM randomly selects a Gaussian component and then adds the chosen Gaussian noise, which can be demonstrated as a more efficient way to perturb the signals into a simple known distribution. We further define the reverse probabilistic model as a parameterized Gaussian mixture kernel. Due to the intractability in calculating the KL divergence between Gaussian mixture models, we derive a variational bound to maximize the likelihood, offering a concise formulation for optimizing the denoising model and valuable insights for designing the sampling strategies. Our theoretical derivation highlights that Mix-DDPM need only shift image which requires the inclusion of a global stochastic offset in both the diffusion and reverse processes , which can be efficiently implemented with just several lines of code. The global stochastic offset effectively fits a Gaussian mixture distribution enhancing the degrees of freedom of the entire diffusion model. Furthermore, we present three streamlined sampling strategies that interface with diverse fast dedicated solvers for diffusion ordinary differential equations, boosting the efficacy of image representation in the sampling phase and alleviating the issue of slow generation speed, thereby enhancing both efficiency and accuracy. Extensive experiments on benchmark datasets demonstrate the effectiveness of Mix-DDPM and its superiority over the original DDPM.

Suppressed possibilistic fuzzy c-means clustering based on shadow sets for noisy data with imbalanced sizes

Possibilistic fuzzy c-means clustering (PFCM) algorithm generates fewer overlapping clustering centers than the possibilistic c-means clustering (PCM) algorithm and possesses better noise immunity than fuzzy c-means clustering (FCM) algorithm. However, with the increasing noise intensity and number of clusters, PFCM still faces the problem of getting partially overlapping clustering centers or mislocated centers in noise regions. Moreover, the sample-size imbalance increasingly intensifies the difficulty of positioning centers of small clusters. To solve the above problems, a suppressed possibilistic fuzzy c-means clustering algorithm based on shadow sets (S-SPFCM) is proposed by introducing the shadow set theory and the "suppressed competitive learning" strategy. Firstly, KL divergence is introduced in the objective function of PFCM to increase the anti-noise robustness of fuzzy memberships against long-range noise and outliers. Secondly, to reduce the number of overlapping centers caused by possibilistic memberships, the shadow set theory is introduced to divide each class adaptively into three regions (core, shadow, and exclusion regions) by an uncertainty balance method. The suppressed competitive learning method is extended by modifying the memberships of points within the three regions, thus artificially guiding the iterative track of clustering centers. Meanwhile, to further reduce the influence of imbalanced sizes, a scheme to reset mislocated centers in the core and shadow regions is also designed. In addition, to improve the segmentation performance of S-SPFCM for noisy images, a suppressed possibilistic fuzzy c-means clustering algorithm based on shadow sets and local information (SL-SPFCM) is also proposed. The SL-SPFCM first improves the Euclidean distance by a distance filtering scheme. Then SL-SPFCM introduces the local median membership of each pixel into the KL divergence of the objective function. Finally, experiments on synthetic datasets and color images which are characteristic of imbalanced sizes and noise injection demonstrate the proposed S-SPFCM and SL-SPFCM algorithms achieve smaller center deviations and higher clustering accuracy compared with several state-of-the-art clustering algorithms.

3D mineral prospectivity modeling using deep adaptation network transfer learning: A case study of the Xiadian gold deposit, Eastern China

Mineralization distribution is commonly heterogeneous in space due to the various geology and structure conditions or fluid flow and evoluation, causing the possible difference in mineralization distribution regularity between shallow and deep at a deposit. This presents important challenges for three-dimensional mineral prospectivity modeling (3D MPM) for deep zones with the scarcity data conditions. Transfer learning has shown promising generalization performance in tasks involving shifts in data distribution, reducing reliance on labeled samples and enhancing learning capability with limited data. In this study, we propose an approach of 3D MPM, namely DAN-CBAM, based on the deep adaptation network (DAN) augmented with the convolutional block attention module (CBAM). It theoretically can harmonize the distribution of ore-controlling features between shallow and deep zones of deposits while effectively extract critical high-dimensional features and spatial patterns. The Xiadian orogenic gold deposit was selected as a case study to validate the approach. The DAN integrates multi-layer, multi-kernel adaptation at the top layer of the CNN, resulting in improved alignment of marginal distributions across domains. Metrics for distribution similarity such as Wasserstein distance, were reduced by 0.250 and KL divergence decreased by 0.032. Additionally, the inclusion of the CBAM module led to a lower MK-MMD loss and a faster convergence rate. And the DAN-CBAM model achieves superior prediction accuracy (0.85) compared to traditional deep neural network (DNN) models (0.81). These highlight CBAM's effectiveness in enhancing the model's ability to capture spatial similarities in mineralization. Furthermore, the area under the curve (AUC) value of the DAN-CBAM model (0.869) significantly outperforms traditional machine learning methods, including CNN (0.786) and Random Forest (0.703) models, underscoring its superior predictive efficiency in 3D MPM for deep mineralization. Therefore, the proposed DAN-CBAM model is promising to be applied in the 3D MPM, in particular for the deposits with different mineralization distribution regularities in space.

BEATRICE: Bayesian Fine-mapping from Summary Data using Deep Variational Inference.

We introduce a novel framework BEATRICE to identify putative causal variants from GWAS statistics. Identifying causal variants is challenging due to their sparsity and high correlation in the nearby regions. To account for these challenges, we rely on a hierarchical Bayesian model that imposes a binary concrete prior on the set of causal variants. We derive a variational algorithm for this fine-mapping problem by minimizing the KL divergence between an approximate density and the posterior probability distribution of the causal configurations. Correspondingly, we use a deep neural network as an inference machine to estimate the parameters of our proposal distribution. Our stochastic optimization procedure allows us to simultaneously sample from the space of causal configurations. We use these samples to compute the posterior inclusion probabilities and determine credible sets for each causal variant. We conduct a detailed simulation study to quantify the performance of our framework against two state-of-the-art baseline methods across different numbers of causal variants and different noise paradigms, as defined by the relative genetic contributions of causal and non-causal variants. We demonstrate that BEATRICE achieves uniformly better coverage with comparable power and set sizes, and that the performance gain increases with the number of causal variants. We also show the efficacy BEATRICE in finding causal variants from the GWAS study of Alzheimer's disease. In comparison to the baselines, only BEATRICE can successfully find the APOE allele, a commonly associated variant of Alzheimer's. Thus, we show that BEATRICE is a valuable tool to identify causal variants from eQTL and GWAS summary statistics across complex diseases and traits.

Disturbances of thalamus and prefrontal cortex contribute to cognitive aging: A structure-function coupling analysis based on KL divergence

Normal aging is accompanied by changes in brain structure and function associated with cognitive decline. Structural and functional abnormalities, particularly the prefrontal cortex (PFC) and subcortical regions, contributed to cognitive aging. However, it remains unclear how the synchronized changes in structure and function of individual brain regions affect the cognition in aging. Using 3D T1-weighted structural data and movie watching functional magnetic resonance imaging data in a sample of 422 healthy individuals (ages from 18 to 87 years), we constructed regional structure–function coupling (SFC) of cortical and subcortical regions by quantifying the distribution similarity of gray matter volume (GMV) and amplitude of low-frequency fluctuation (ALFF). Further, we investigated age-related changes in SFC and its relationship with cognition. With aging, increased SFC localized in PFC, thalamus and caudate nucleus, decreased SFC in temporal cortex, lateral occipital cortex and putamen. Moreover, the SFC in the PFC was associated with executive function and thalamus was associated with the fluid intelligence, and partially mediated age-related cognitive decline. Collectively, our results highlight that tighter structure–function synchron of the PFC and thalamus might contribute to age-related cognitive decline, and provide insight into the substrate of the thalamo-prefrontal pathway with cognitive aging.

GAP: A group-based automatic pruning algorithm via convolution kernel fusion

In recent years, the deployment and operation of convolution neural networks on edge devices with limited computing capabilities have become increasingly challenging due to the large network structure and computational cost. Currently, the mainstream structured pruning algorithms mainly compress the network at the filter or layer level. However, these methods introduce too much human intervention with large granularities, which may lead to unpredictable performance after compression. In this paper, we propose a group-based automatic pruning algorithm(GAP) via kernel fusion to automatically search for the optimal pruning structure in a more fine-grained manner. Specifically, we first adopt a novel nonlinear dimensionality reduction clustering algorithm to divide the filters of each convolution layer into groups of equal size. Afterwards, we encode the mutual distribution similarity of the kernels within each group, and its KL divergence is employed as an importance indicator to determine the retained kernel groups through weighted fusion. Subsequently, we introduce an intelligent searching module that automatically explore and optimize the pruned structure of each layer. Finally, the pruned filters are permutated to form a dense group convolution and fine-tuned. Sufficient experiments show that, on two image classification datasets, for five advanced CNN models, our GAP algorithm outperforms most extant SOTA schemes, reduces artificial intervention, and enables efficient end-to-end training of compact models.

Fund transfer fraud detection: Analyzing irregular transactions and customer relationships with self-attention and graph neural networks

This paper presents a method for identifying fraudulent fund transfers using real bank data, analyzing customer information, transactional activities, and customer relationships. The preprocessing step transforms high-dimensional, irregular transaction time series into regular time series, then further compresses them in a latent space using a self-attention-based autoencoder. To address the scarcity of fraudulent data samples and mitigate training issues caused by data imbalance, various deep generative models, including the conditional variational autoencoder and Wasserstein generative adversarial network, are applied to generate additional fraudulent raw data and augment fraud samples in latent space. The reparameterization trick is integrated into the encoder–decoder structure to boost the model’s generative capabilities. Additionally, a Graph Neural Network (GNN) is used to model customer relationships. The proposed approach utilizes end-to-end learning, integrating the autoencoder’s reconstruction loss, KL divergence loss (when reparameterization trick is applied), and classification loss for fraud detection. To optimize computational resources, neighborhood sampling for GNN is combined with mini-batch training for the autoencoder, improving both training efficiency and model reliability. Comprehensive experiments demonstrate the effectiveness of the proposed fusion network, highlighting the importance of each component and preprocessing step. For example, the areas under the precision–recall curves for fraud detection show notable improvements in our model. For suspicious transactions identified by the bank’s rules, other models range from 0.66% to 22.15%, while our model reached 27%. For non-suspicious transactions, other models range from 2.53% to 22.00%, with our model achieving 22.90%. This model has potential for wider applications in anomaly detection, particularly in datasets with irregular time series and complex customer relationships.

Learning the feature distribution similarities for online time series anomaly detection

Identifying anomalies in multi-dimensional sequential data is crucial for ensuring optimal performance across various domains and in large-scale systems. Traditional contrastive methods utilize feature similarity between different features extracted from multidimensional raw inputs as an indicator of anomaly severity. However, the complex objective functions and meticulously designed modules of these methods often lead to efficiency issues and a lack of interpretability. Our study introduces a structural framework called SimDetector, which is a Local–Global Multi-Scale Similarity Contrast network. Specifically, the restructured and enhanced GRU module extracts more generalized local features, including long-term cyclical trends. The multi-scale sparse attention module efficiently extracts multi-scale global features with pattern information. Additionally, we modified the KL divergence to suit the characteristics of time series anomaly detection, proposing a symmetric absolute KL divergence that focuses more on overall distribution differences. The proposed method achieves results that surpass or approach the State-of-the-Art (SOTA) on multiple real-world datasets and synthetic datasets, while also significantly reducing Multiply-Accumulate Operations (MACs) and memory usage.

Intrinsic Rewards for Exploration Without Harm From Observational Noise: A Simulation Study Based on the Free Energy Principle.

In reinforcement learning (RL), artificial agents are trained to maximize numerical rewards by performing tasks. Exploration is essential in RL because agents must discover information before exploiting it. Two rewards encouraging efficient exploration are the entropy of action policy and curiosity for information gain. Entropy is well established in the literature, promoting randomized action selection. Curiosity is defined in a broad variety of ways in literature, promoting discovery of novel experiences. One example, prediction error curiosity, rewards agents for discovering observations they cannot accurately predict. However, such agents may be distracted by unpredictable observational noises known as curiosity traps. Based on the free energy principle (FEP), this letter proposes hidden state curiosity, which rewards agents by the KL divergence between the predictive prior and posterior probabilities of latent variables. We trained six types of agents to navigate mazes: baseline agents without rewards for entropy or curiosity and agents rewarded for entropy and/or either prediction error curiosity or hidden state curiosity. We find that entropy and curiosity result in efficient exploration, especially both employed together. Notably, agents with hidden state curiosity demonstrate resilience against curiosity traps, which hinder agents with prediction error curiosity. This suggests implementing the FEP that may enhance the robustness and generalization of RL models, potentially aligning the learning processes of artificial and biological agents.

Multi-modal incomplete label information three-way bidirectional decision-making: Applications of disease assessment

Disease assessment involves two stages of decision-making process. The first process is to infer the development trend of the disease through the existing judgment conditions, which is to make a positive estimation (prediction) decision. The second process is to update the judgment conditions for different disease states and optimize the clinical assessment evidence by fusing disease progression data, that is, to make a reverse evaluation (optimization) decision by reviewing historical diagnostic and treatment data to summarize the best judgment conditions for the disease characteristics. Therefore, we construct a three-way bidirectional decision method for multi-modal incomplete labeling information, considering the mixed data features with no labels and labels, and the bidirectional decision features of positive prediction and reverse optimization. Specifically, the method first introduces semi-supervised learning (SSL) into the rough sets, constructs a rough sets model based on cosine similarity relation, generates pseudo-labels for incomplete information, and explains the positive prediction decision process principle. Second, the gradient space is constructed on the predictive information system by the three-way decision-making principle, the loss function formed by the combination of KL divergence and cross entropy loss function is used to find the direction of the fastest decline in attribute information value, and the optimal decision rule and the automatic updating strategy of the threshold value are obtained. At the same time, the principle of the reverse optimization decision process is explained. Finally, multimodal information from pathological tissue images is used to evaluate clinical progress problem. A pathological tissue images dataset includes 2000 colorectal cancer and 300 kidney disease cases used for simulation experiments. The proposed three-way bidirectional decision-making model can be widely applied to actual decision-making scenarios for evaluating clinical progress of different diseases. Compared with previous studies, this method can realize objective calculation and automatic update of threshold and loss values and effectively solve complex prediction and optimization decision-making problems.

Quantifying Quantum Chaos through Microcanonical Distributions of Entanglement

A characteristic feature of “quantum chaotic” systems is that their eigenspectra and eigenstates display universal statistical properties described by random matrix theory (RMT). However, eigenstates of local systems also encode structure beyond RMT. To capture this feature, we introduce a framework that allows us to compare the properties of eigenstates in local systems with those of pure random states. In particular, our framework defines a notion of distance between quantum state ensembles that utilizes the Kullback-Leibler divergence to compare the microcanonical distribution of entanglement entropy (EE) of eigenstates with a reference RMT distribution generated by pure random states (with appropriate constraints). This notion gives rise to a quantitative metric for quantum chaos that not only accounts for averages of the distributions but also higher moments. The differences in moments are compared on a highly resolved scale set by the standard deviation of the RMT distribution, which is exponentially small in system size. As a result, the metric can distinguish between chaotic and integrable behaviors and, in addition, quantify and compare the of chaos (in terms of proximity to RMT behavior) between two systems that are assumed to be chaotic. We implement our framework in local, minimally structured, Floquet random circuits, as well as a canonical family of many-body Hamiltonians, the mixed-field Ising model (MFIM). Importantly, for Hamiltonian systems, we find that the reference random distribution must be appropriately constrained to incorporate the effect of energy conservation in order to describe the ensemble properties of midspectrum eigenstates. The metric captures deviations from RMT across all models and parameters, including those that have been previously identified as strongly chaotic, and for which other diagnostics of chaos such as level spacing statistics look strongly thermal. In Floquet circuits, the dominant source of deviations is the second moment of the distribution, and this persists for all system sizes. For the MFIM, we find significant variation of the KL divergence in parameter space. Notably, we find a small region where deviations from RMT are minimized, suggesting that “maximally chaotic” Hamiltonians may exist in fine-tuned pockets of parameter space. Published by the American Physical Society 2024

An uncertainty-aware domain adaptive semantic segmentation framework

Semantic segmentation is significant to realize the scene understanding of autonomous driving. Due to the lack of annotated real-world data, the technology of domain adaptation is applied so that the model is trained on the synthetic data and inferred on the real data. However, this domain gap leads to aleatoric and epistemic uncertainty. These uncertainties link to the potential safety issue of autonomous driving in normal weather and adverse weather. In this study, we explore the scientific problem that has received sparse attention previously. We postulate that the Dual Attention module can mitigate the uncertainty in the task of semantic segmentation and provide some empirical study to validate it. Furthermore, the utilization of Kullback-Leibler divergence (KL divergence) helps the estimation of aleatoric uncertainty and boosts the robustness of the segmentation model. Our empirical study on the diverse datasets of semantic segmentation demonstrates the effectiveness of our method in normal and adverse weather. Our code is available at: https://github.com/liubo629/Seg-Uncertainty-dual-attention.

IoT traffic classification and anomaly detection method based on deep autoencoders

This study investigates anomaly detection of IoT device traffic using Convolutional Neural Networks (CNN) and Variational Autoencoders (VAE) to enhance the detection capability of security threats in IoT environments. A series of hardware configurations, software environments, and hyperparameters were utilized to optimize the training and testing processes of the models. The CNN model demonstrates robust classification performance, achieving an accuracy rate of 95.85% on the test dataset, effectively distinguishing between different types of IoT device traffic. Meanwhile, the VAE model exhibits proficient anomaly detection capabilities by effectively capturing abnormal patterns in the data using reconstruction loss and KL divergence. The combined use of CNN and VAE models offers a comprehensive solution to cybersecurity challenges in IoT environments. Future research directions include exploring diverse IoT traffic data, practical deployment for validation, and further optimization of model structures and parameters to improve performance and applicability.

Similarity measure method of near-infrared spectrum combined with multi-attribute information

Due to the high-dimensionality, redundancy, and non-linearity of the near-infrared (NIR) spectra data, as well as the influence of attributes such as producing area and grade of the sample, which can all affect the similarity measure between samples. This paper proposed a t-distributed stochastic neighbor embedding algorithm based on Sinkhorn distance (St-SNE) combined with multi-attribute data information. Firstly, the Sinkhorn distance was introduced which can solve problems such as KL divergence asymmetry and sparse data distribution in high-dimensional space, thereby constructing probability distributions that make low-dimensional space similar to high-dimensional space. In addition, to address the impact of multi-attribute features of samples on similarity measure, a multi-attribute distance matrix was constructed using information entropy, and then combined with the numerical matrix of spectral data to obtain a mixed data matrix. In order to validate the effectiveness of the St-SNE algorithm, dimensionality reduction projection was performed on NIR spectral data and compared with PCA, LPP, and t-SNE algorithms. The results demonstrated that the St-SNE algorithm effectively distinguishes samples with different attribute information, and produced more distinct projection boundaries of sample category in low-dimensional space. Then we tested the classification performance of St-SNE for different attributes by using the tobacco and mango datasets, and compared it with LPP, t-SNE, UMAP, and Fisher t-SNE algorithms. The results showed that St-SNE algorithm had the highest classification accuracy for different attributes. Finally, we compared the results of searching the most similar sample with the target tobacco for cigarette formulas, and experiments showed that the St-SNE had the highest consistency with the recommendation of the experts than that of the other algorithms. It can provide strong support for the maintenance and design of the product formula.

Variational Autoencoders for Generative Drug-Gene Interactions in Periodontal Bone Resorption.

Introduction Periodontal bone resorption is a significant dental problem causing tooth loss and impaired oral function. It is influenced by factors such as bacterial plaque, genetic predisposition, smoking, systemic diseases, medications, hormonal changes, and poor oral hygiene. This condition disrupts bone remodeling, favoring resorptive processes. Variational autoencoders (VAEs) can learn the distribution of drug-gene interactions from existing data, identify potential drug targets, and predict therapeutic effects. This study investigates the generation of drug-gene interactions in periodontal bone resorption using VAEs. Methods A bone resorptive drugs dataset was retrieved from Probes and Drugs and analyzed using Cytoscape (https://cytoscape.org/) and CytoHubba (https://apps.cytoscape.org/apps/cytohubba), powerful tools for studying drug-gene interactions in bone resorption. The dataset was then prepared for matrix representation, with normalized input data. It was subsequently divided into training, validation, and testing sets. We then built an encoder-decoder network, defined a loss function, optimized parameters, and fine-tuned hyperparameters. Using VAEs, we generated new drug-gene interactions, assessed model performance, and visualized the latent space with reconstructed drug-gene interactions for further insights. Results The analysis revealed the top hub genes in drug-gene interactions, including Matrix Metalloproteinase (MMP) 14, MMP 9, HIF1A, STAT1, MAPT, CAS9, MMP2, CASP3, MMP1, and MAK1. The VAE's reconstruction accuracy was measured using mean squared error (MSE), with an average squared difference of 0.077. Additionally, the KL divergence value was 2.349, and the average reconstruction log-likelihood was -246. Conclusion The generative variational encoder model for drug-gene interactions in bone resorption demonstrates high accuracy and reliability in representing complex drug-gene relationships within this context.

An Advanced Nonlinear Framework for Early Detection and Prognosis of Fatigue Cracks in Plate-Like Structures

Plate-like structures are prone to the initiation of fatigue cracks during operation, primarily due to the harmonic loading imposed by the external environment. These micro-cracks possess the potential to progress into more severe forms, compromising structural integrity and, in extreme cases, leading to catastrophic accidents. The primary focus of this research is to introduce an online nonlinear framework for the early diagnosis and prognosis of fatigue cracks. This framework aims to detect nascent fatigue cracks in plate-like structures, facilitating proactive measures to prevent structural failure and preserve the overall integrity. The initial phase of this study involves capturing Lamb waves generated by subjecting the inspected plate-like structures to an excitation signal. The excitation signal, along with the S0 mode Lamb wave, is employed to identify a Nonlinear Autoregressive with Exogenous inputs (NARX) model, serving as a representation of the underlying nonlinear dynamic mapping intrinsic to the inspected specimen. Subsequently, the Nonlinear Output Frequency Response Functions (NOFRFs) are identified based on the established NARX model. An early fatigue crack damage index, denoted as optimal NOFRFs weighted contribution rate based on the KL divergence (KLRm), is extracted from the NOFRFs and employed for estimating crack lengths. Additionally, the Particle Filter (PF) is utilized for predicting the Remain Useful Life (RUL). To validate the effectiveness of the proposed framework, fatigue experiments are conducted on aluminum lug joint specimens. The experimental results demonstrate the effectiveness and precision of the framework in estimating crack lengths and predicting RUL for nonlinear early fatigue cracks.