Invertible Neural Networks Research Articles

Inferring stellar parameters and their uncertainties from high-resolution spectroscopy using invertible neural networks

New spectroscopic surveys will increase the number of astronomical objects in need of characterisation by more than an order of magnitude. Machine learning tools are required to address this data deluge in a fast and accurate fashion. Most machine learning algorithms cannot directly estimate error, making them unsuitable for reliable science. We aim to train a supervised deep-learning algorithm tailored for high-resolution observational stellar spectra. This algorithm accurately infers precise estimates while providing coherent estimates of uncertainties by leveraging information from both the neural network and the spectra. We trained a conditional invertible neural network (cINN) on observational spectroscopic data obtained from the GIRAFFE spectrograph (HR10 and HR21 setups) within the Gaia-ESO survey. A key feature of cINN is its ability to produce the Bayesian posterior distribution of parameters for each spectrum. By analysing this distribution, we inferred stellar parameters and their corresponding uncertainties. We carried out several tests to investigate how parameters are inferred and errors are estimated. We achieved an accuracy of 28K in $T_ eff $, 0.06 dex in $ g$, 0.03 dex in Fe/H $, and between 0.05 dex and 0.17 dex for the other abundances for high-quality spectra. Accuracy remains stable with low signal-to-noise ratio (between 5 and 25) spectra, with an accuracy of 39K in $T_ eff $, 0.08 dex in $ g$, and 0.05 dex in Fe/H $. The uncertainties obtained are well within the same order of magnitude. The network accurately reproduces astrophysical relationships both on the scale of the Milky Way and within smaller star clusters. We created a table containing the new parameters generated by our cINN. This neural network represents a compelling proposition for future astronomical surveys. These derived uncertainties are coherent and can therefore be reused in future works as Bayesian priors.

Model uncertainty quantification of a degradation model of miter gates using normalizing flow-based likelihood-free inference

Degradation modeling is essential for failure prognostics of miter gates and subsequent risk-informed maintenance or operational decision-making. Typically, degradation models are formulated based on certain assumptions and simplifications and may not fully capture the complexity of underlying degradation mechanisms. Even minor errors in such models may lead to significant discrepancies in failure prognostics due to error accumulation over time. Aiming to improve the accuracy of the degradation model and ensure reliable failure prognostics, this article proposes a degradation model updating framework for miter gates using normalizing flow-based likelihood-free inference, accounting for both model parameter uncertainty and model form uncertainty (i.e., model bias). The proposed work consists of two phases: the offline training phase and the online updating phase. During the offline training phase, the unknown model bias term is modeled as a random noise with uncertain standard deviation. Synthetic data are then generated using a physical model based on prior knowledge of the uncertain model parameters and model bias. The synthetic data are utilized to train an inference model, which has a summary network for data compression and a conditional invertible neural network for parameter estimation. In the online updating phase, the trained inference model uses strain observations to obtain posterior samples. The Gaussian mixture model and dual particle filter techniques are utilized to recursively update posterior samples, thereby improving the estimation accuracy. Subsequently, model bias is inversely estimated using the degradation model, which uses the updated model parameters and the gap length from the inference model. Following that, a regression model is constructed to correct the biases inherent in the simplified degradation model. This process is implemented iteratively over time to perform continuous model updating and failure prognostics. Results of a case study demonstrate the efficacy of the proposed framework in reducing both model parameter uncertainty and recovering model biases and thereby improving the accuracy of failure prognostics of miter gates.

Multi‐colour recipe prediction through invertible neural network and clustering algorithm

AbstractComputer‐assisted colour prediction has become increasingly significant for various consumer items in the industrial sector. Many professional colourists find it challenging to develop a suitable colour recipe. The most difficult aspect is predicting which dye mix will result in the required shade on a given fabric. This study introduces an advanced alternative to the traditional colour‐making method using an invertible neural network (INN) model. The INN model effectively addresses real‐world inverse problems due to its bi‐directional nature. In the forward phase, the model retrieves information about the colour recipe; in the backward phase, this information is combined with latent space data to predict the recipe. Furthermore, unsupervised data generated by the INN model is fed into clustering algorithms, such as K‐means and the Gaussian mixture model (GMM), to obtain multiple recipes. The forward procedure was reintroduced with a predicted recipe to assess the efficacy of the proposed model. An analysis was then conducted on the colour differences between the anticipated and actual recipes. The colour differences, rounded to perceptually significant precision, from 30,000 samples with 50 centre points, are as follows: 1.4, 2.2, 2.5, 3.7, 1.2, and 2.1. These results indicate that the INN model and the GMM clustering approach together provide a highly accurate and efficient solution to automating the colour‐matching process, offering a more precise and practical solution for the colour‐manufacturing industry.

Rethinking Pan-Sharpening in Closed-Loop Regularization.

It is generally known that pan-sharpening is fundamentally a PAN-guided multispectral (MS) image super-resolution problem that involves learning the nonlinear mapping from low-resolution (LR) to high-resolution (HR) MS images. Since an infinite number of HR-MS images can be downsampled to produce the same corresponding LR-MS image, learning the mapping from LR-MS to HR-MS image is typically ill-posed and the space of the possible pan-sharpening functions can be extremely large, making it difficult to estimate the optimal mapping solution. To address the above issue, we propose a closed-loop scheme that learns the two opposite mapping including the pan-sharpening and its corresponding degradation process simultaneously to regularize the solution space in a single pipeline. More specifically, an invertible neural network (INN) is introduced to perform a bidirectional closed-loop: the forward operation for LR-MS pan-sharpening and the backward operation for learning the corresponding HR-MS image degradation process. In addition, given the vital importance of high-frequency textures for the Pan-sharpened MS images, we further strengthen the INN by designing a specified multiscale high-frequency texture extraction module. Extensive experimental results demonstrate that the proposed algorithm performs favorably against state-of-the-art methods qualitatively and quantitatively with fewer parameters. Ablation studies also verify the effectiveness of the closed-loop mechanism in pan-sharpening. The source code is made publicly available at https://github.com/manman1995/pan-sharpening-Team-zhouman/.

A Transformer-based invertible neural network for robust image watermarking

For the existing encoder-noise-decoder (END) based watermarking models, since the coupling between the encoder and the decoder is weak, the encoder generally embeds certain redundant features into the cover image to enable the decoder to extract watermark completely, which will affect watermarking invisibility. To address this problem, this paper proposes a Transformer-based invertible neural network (INN) for robust image watermarking (IWFormer). In order to effectively reduce redundant features, the INN framework is utilized for the watermark embedding and extracting processes, so that the encoded features are highly consistent with the features required for decoding. For enhancing watermarking robustness, an affine Transformer module is designed by mining the global correlation of the cover image. In addition, considering that the human visual system is sensitive to low-frequency variations, the wavelet low-frequency sub-band loss is deployed to guide watermark to be embedded in middle- and high-frequency components, thus further increasing the quality of the encoded images. Experimental results demonstrate that compared with the existing state-of-the-art watermarking models, the proposed IWFormer owns remarkable advantages in terms of both watermarking invisibility and robustness.

Reconstructing and analyzing the invariances of low-dose CT image denoising networks.

Deep learning-based methods led to significant advancements in many areas of medical imaging, most of which are concerned with the reduction of artifacts caused by motion, scatter, or noise. However, with most neural networks being black boxes, they remain notoriously difficult to interpret, hindering their clinical implementation. In particular, it has been shown that networks exhibit invariances w.r.t. input features, that is, they learn to ignore certain information in the inputdata. To improve the interpretability of deep learning-based low-dose CT image denoisingnetworks. We learn a complete data representation of low-dose input images using a conditional variational autoencoder (cVAE). In this representation, invariances of any given denoising network are then disentangled from the information it is not invariant to using a conditional invertible neural network (cINN). At test time, image-space invariances are generated by applying the inverse of the cINN and subsequent decoding using the cVAE. We propose two methods to analyze sampled invariances and to find those that correspond to alterations of anatomicalstructures. The proposed method is applied to four popular deep learning-based low-dose CT image denoising networks. We find that the networks are not only invariant to noise amplitude and realizations, but also to anatomicalstructures. The proposed method is capable of reconstructing and analyzing invariances of deep learning-based low-dose CT image denoising networks. This is an important step toward interpreting deep learning-based methods for medical imaging, which is essential for their clinical implementation.

Photoacoustic Quantification of Tissue Oxygenation Using Conditional Invertible Neural Networks.

Intelligent systems in interventional healthcare depend on the reliable perception of the environment. In this context, photoacoustic tomography (PAT) has emerged as a non-invasive, functional imaging modality with great clinical potential. Current research focuses on converting the high-dimensional, not human-interpretable spectral data into the underlying functional information, specifically the blood oxygenation. One of the largely unexplored issues stalling clinical advances is the fact that the quantification problem is ambiguous, i.e. that radically different tissue parameter configurations could lead to almost identical photoacoustic spectra. In the present work, we tackle this problem with conditional Invertible Neural Networks (cINNs). Going beyond traditional point estimates, our network is used to compute an approximation of the conditional posterior density of tissue parameters given the photoacoustic spectrum. To this end, an automatic mode detection algorithm extracts the plausible solution from the sample-based posterior. According to a comprehensive validation study based on both synthetic and real images, our approach is well-suited for exploring ambiguity in quantitative PAT.

Invertible neural network combined with dynamic mode decomposition applied to flow field feature extraction and prediction

Invertible neural network combined with dynamic mode decomposition applied to flow field feature extraction and prediction

Identification of nonlinear system model and inverse model based on conditional invertible neural network

In applications such as adaptive inverse control, internal model control, and active noise control, the identification accuracy of the system model and the inverse model directly affects the performance. However, it is not easy to identify inverse models for nonlinear systems. Moreover, existing methods require two identification calculations to obtain the system model and the inverse model. Therefore, an identification method of nonlinear system model and inverse model based on conditional invertible neural network (cINN) is proposed. The invertible structure of cINN enables simultaneous approximation of complex nonlinear functions and simultaneous acquisition of the corresponding inverse functions. Consequently, both the nonlinear system model and the inverse model can be identified concurrently through cINN. Moreover, the identification performance of the cINN-based method is validated and applied to disturbance cancellation in a classical nonlinear simulation system. Finally, the inverse model of the actuator is identified by cINN, and the inverse model is applied to the nonlinear compensation of the actuator.

RIHINNet: A robust image hiding method against JPEG compression based on invertible neural network

AbstractImage hiding is a task that embeds secret images in digital images without being detected. The performance of image hiding has been greatly improved by using the invertible neural network. However, current image hiding methods are less robust in the face of Joint Photographic Experts Group (JPEG) compression. The secret image cannot be extracted from the stego image after JPEG compression of the stego image. Some methods show good robustness for some certain JPEG compression quality factors but poor robustness for other common JPEG compression quality factors. An image‐hiding network (RIHINNet) that is robust to all common JPEG compression quality factors is proposed. First of all, the loss function is redesigned; thus, the secret image is hidden as much as possible in the area that is less likely to be changed after JPEG compression. Second, the classifier is designed, which can help the model to select the extractor according to the range of JPEG compression degree. Finally, the interval robustness of the secret image extraction is improved through the design of a denoising module. Experimental results show that this RIHINNet outperforms other state‐of‐the‐art image‐hiding methods in the face of JPEG compressed noise with random compression quality factors, with more than 10 dB peak signal‐to‐noise ratio improvement in secret image recovery on ImageNet, COCO and DIV2K datasets.

An improved YOLOv8 model enhanced with detail and global features for underwater object detection

Underwater object detection is significant for the practical research of mastering existing marine biological resources. In response to the challenges posed by complex underwater environments such as water scattering and variations in object scales, researchers have developed YOLOv8 for object detection, driven by the rising popularity and iteration of deep learning. Building upon this model, we propose an enhanced underwater object detection model named YOLOv8-DGF. Firstly, we replace the convolutional layers of Spatial Pyramid Pooling Fusion (SPPF) with Invertible Neural Networks to further augment the fusion capacity of detailed features, facilitating the preservation of pivotal information while mitigating the impact of noise. Additionally, we introduce a global attention mechanism into Convolution to Fully Connected (C2f), which weights the input features, thereby emphasizing or suppressing feature information from different locations. Through our ‘Detail to Global’ strategy, the model achieved mAP@0.5 scores of 87.7% and 84.8% on the RUOD and URPC2020 datasets, respectively, with improved processing speed. Extensive ablation experiments on the Pascal VOC dataset demonstrate that YOLOv8-DGF outperforms other methods, achieving the best overall performance.

A cross-modal feature aggregation and enhancement network for hyperspectral and LiDAR joint classification

Advancements in Earth observation technologies have greatly enhanced the potential of integrating hyperspectral (HS) images with Light Detection and Ranging (LiDAR) data for land use and land cover classification. Despite this, most existing methods primarily focus on employing deep network layers to extract features from two heterogeneous data modalities, often overlooking a gradual modeling data representation approach from shallow to deep layers. Furthermore, excessive network layers can result in the deterioration of modality-specific features, therefore lowering the classification performance. The paper proposed a novel cross-modal feature aggregation and enhancement network for the joint classification of HSI and LiDAR data. Initially, a cross-modal feature fusion module is developed to exploit spatial scale consistency to complete the interchange and fusion of feature embedding at the pixel level, preserving the original information from the two heterogeneous modalities to a certain degree. Then two straightforward strategies (i.e., addition and concatenation) are employed in the shallow network layers before being sent to the transformer encoder. The former facilitates the model’s ability to discern more subtle distinctions and refine spatial location details. The latter ensures the preservation of information integrity, effectively mitigating the risk of feature loss. Moreover, invertible neural networks and a feature enhancement module are introduced, leveraging the complementary information of HSI and LiDAR data to enhance the detail and texture information extracted in deeper layers. Extensive experiments on Houston2013, Trento, and MUUFL datasets demonstrate that the proposed method outperforms several state-of-the-art models in three evaluation metrics, achieving an accuracy improvement of up to 2%. The proposed model brings new inspirations for HSI and LiDAR classification, which is critical for accurate environmental monitoring, urban planning, and precision agriculture. The source code is publicly accessible at https://github.com/zhangyiyan001/CMFAEN.

Event-by-event comparison between machine-learning- and transfer-matrix-based unfolding methods

The unfolding of detector effects is a key aspect of comparing experimental data with theoretical predictions. In recent years, different Machine-Learning methods have been developed to provide novel features, e.g. high dimensionality or a probabilistic single-event unfolding based on generative neural networks. Traditionally, many analyses unfold detector effects using transfer-matrix-based algorithms, which are well established in low-dimensional unfolding. They yield an unfolded distribution of the total spectrum, together with its covariance matrix. This paper proposes a method to obtain probabilistic single-event unfolded distributions, together with their uncertainties and correlations, for the transfer-matrix-based unfolding. The algorithm is first validated on a toy model and then applied to pseudo-data for the pp→Zγγ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$pp\\rightarrow Z\\gamma \\gamma $$\\end{document} process. In both examples the performance is compared to the Machine-Learning-based single-event unfolding using an iterative approach with conditional invertible neural networks (IcINN).

Search Me Knot, Render Me Knot: Embedding Search and Differentiable Rendering of Knots in 3D

AbstractWe introduce the problem of knot‐based inverse perceptual art. Given multiple target images and their corresponding viewing configurations, the objective is to find a 3D knot‐based tubular structure whose appearance resembles the target images when viewed from the specified viewing configurations. To solve this problem, we first design a differentiable rendering algorithm for rendering tubular knots embedded in 3D for arbitrary perspective camera configurations. Utilizing this differentiable rendering algorithm, we search over the space of knot configurations to find the ideal knot embedding. We represent the knot embeddings via homeomorphisms of the desired template knot, where the weights of an invertible neural network parametrize the homeomorphisms. Our approach is fully differentiable, making it possible to find the ideal 3D tubular structure for the desired perceptual art using gradient‐based optimization. We propose several loss functions that impose additional physical constraints, enforcing that the tube is free of self‐intersection, lies within a predefined region in space, satisfies the physical bending limits of the tube material, and the material cost is within a specified budget. We demonstrate through results that our knot representation is highly expressive and gives impressive results even for challenging target images in both single‐view and multiple‐view constraints. Through extensive ablation study, we show that each proposed loss function effectively ensures physical realizability. We construct a real‐world 3D‐printed object to demonstrate the practical utility of our approach.

Dynamical analysis of high-order Hopfield neural network with application in WBANs

Given the increasing demand for privacy preservation of medical data, a novel medical image encryption scheme is proposed based on Invertible Neural Network (INN) in this paper. Firstly, a new High-Order Hopfield Neural Network (HOHNN) is designed to yield a cipher flow as the conditional input, thereby regulating the encryption process. Moreover, its intrinsic properties as well as various dynamic behaviors are both demonstrated through theoretical analysis and two-parameter Lyapunov exponential charts. Secondly, guided by a multi-objective loss function and a known prior distribution, the original medical image is encrypted into a noise-like cipher image through the proposed encryption scheme. Besides, it is worth mentioning that unlike the existing deep learning-based data encryption schemes, the decryption network shares the identical weight matrix and network structure with the corresponding encryption one in this work. Finally, extensive simulation experiments have validated the feasibility as well as the security of the proposed scheme.

Efficient Bayesian inference using physics-informed invertible neural networks for inverse problems

This paper presents an innovative approach to tackle Bayesian inverse problems using physics-informed invertible neural networks (PI-INN). Serving as a neural operator model, PI-INN employs an invertible neural network (INN) to elucidate the relationship between the parameter field and the solution function in latent variable spaces. Specifically, the INN decomposes the latent variable of the parameter field into two distinct components: the expansion coefficients that represent the solution to the forward problem, and the noise that captures the inherent uncertainty associated with the inverse problem. Through precise estimation of the forward mapping and preservation of statistical independence between expansion coefficients and latent noise, PI-INN offers an accurate and efficient generative model for resolving Bayesian inverse problems, even in the absence of labeled data. For a given solution function, PI-INN can provide tractable and accurate estimates of the posterior distribution of the underlying parameter field. Moreover, capitalizing on the INN’s characteristics, we propose a novel independent loss function to effectively ensure the independence of the INN’s decomposition results. The efficacy and precision of the proposed PI-INN are demonstrated through a series of numerical experiments.

State-space reconstruction from partial observables using an invertible neural network with structure-preserving properties for nonlinear structural dynamics

State-space reconstruction from partial observables using an invertible neural network with structure-preserving properties for nonlinear structural dynamics

DeFuseDTI: Interpretable drug target interaction prediction model with dual-branch encoder and multiview fusion

Predicting the interaction between drugs and targets is a crucial step in drug development, and computer-based deep learning approaches have the potential to significantly reduce costs. Existing models using a single encoder often suffer from insufficient cross-modal feature extraction, with most models tending to overly focus on extracting locally aggregated information, thereby diluting the detailed features of each target residue and drug atom. Additionally, the lack of effective interaction fusion between drug and target lead to prediction results lacking reliable interpretability, posing a more urgent issue. To address these challenges, we propose a dual-branch encoder model, DeFuseDTI, which includes base encoder and detail encoder to extract locally aggregated features and detailed features of each target residue and drug atom. The detail encoder (utilizing Invertible Neural Networks for targets and graph transformers for drugs) can capture furtherly the features of each atom and residue, providing rich and precise features for model interpretability. For better achieve interactive learning of drug and target features, the Multiview Fusion Attention learning module was introduced to integrate multiview features and generate a unified representations for decoding prediction results. Based on the module's unique attention mechanism, drug-target importance matrices can be obtained, which offer interpretable analysis at the molecular level. Experimental results and analyses demonstrate that DeFuseDTI outperforms several state-of-the-art models on four public datasets. Its significant interpretability holds promise for providing accurate and scientifically meaningful guidance for biochemical experiments at the molecular level.

Bayesian Model Updating of Multiscale Simulations Informing Corrosion Prognostics Using Conditional Invertible Neural Networks

Abstract Physics-based multi-scale corrosion simulation plays a vital role in predicting the evolution of pitting corrosion on large civil infrastructure such as miter gates, contributing to a model-informed structural health monitoring (SHM) strategy for risk-based asset health management. The physics-based analysis, however, may not accurately reflect the underlying true physics due to various uncertainty sources and needs to be updated using Bayesian inference methods based on observations to make the prediction closer to field observations. However, traditional Bayesian inference requires the evaluation of a likelihood function, which is often unavailable due to the complex model architecture and various surrogate models used in the analysis. Therefore, likelihood-free inference approaches are required for the updating of the multi-scale corrosion simulation models. This paper meets this need by proposing a conditional invertible neural network (cINN)-based Bayesian model updating method for an existing corrosion simulation model. We first train an cINN model based on simulated observations generated from a high-fidelity forward corrosion analysis model. A convolutional neural network (CNN)-based feature extraction algorithm is then employed to extract key features from corrosion images. After that, the extracted corrosion features from CNN is used as inputs of the cINN model to directly obtain posterior distributions of uncertain corrosion model parameters without evaluating the likelihood function. The results show that the proposed cINN-based model updating approach can provide more accurate inference results with a reduced computational cost in comparison to the classical approximate Bayesian computation (ABC) approach.

Bayesian model-based damage detection in engineering systems using Invertible Neural Networks

This study explores the application of BayesFlow, a state-of-the-art deep learning technique for approximate Bayesian inference, in the context of Structural Health Monitoring (SHM), and specifically, model-based damage detection. BayesFlow diverges from other likelihood-free methods by not relying on predefined summary statistics, but learning optimal ones from the simulations via invertible neural networks. It is also capable of performing almost instantaneous inference for an arbitrary number of datasets, which is ideal for real-time applications such as bridge monitoring systems. The effectiveness of the proposed approach is demonstrated with a small scale bridge use case with synthetic measurements for different testing scenarios. Across all sub-cases, BayesFlow is able to determine both the damage location and severity with a high accuracy, while the credible intervals are close to those obtained under an exact Bayesian inference approach with the true likelihood.