Invertible Neural Networks Research Articles

CURTAINs for your sliding window: Constructing unobserved regions by transforming adjacent intervals.

We propose a new model independent technique for constructing background data templates for use in searches for new physics processes at the LHC. This method, called Curtains, uses invertible neural networks to parameterise the distribution of side band data as a function of the resonant observable. The network learns a transformation to map any data point from its value of the resonant observable to another chosen value. Using Curtains, a template for the background data in the signal window is constructed by mapping the data from the side-bands into the signal region. We perform anomaly detection using the Curtains background template to enhance the sensitivity to new physics in a bump hunt. We demonstrate its performance in a sliding window search across a wide range of mass values. Using the LHC Olympics dataset, we demonstrate that Curtains matches the performance of other leading approaches which aim to improve the sensitivity of bump hunts, can be trained on a much smaller range of the invariant mass, and is fully data driven.

Spectral imaging enables contrast agent–free real-time ischemia monitoring in laparoscopic surgery

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Conditional invertible image re-scaling

One of the key issue for image re-scaling is modeling the down-sampling loss. The loss can be approached by a prior distribution through an invertible network in recent works. However, this assumption of independence between images and down-sampling losses is not always satisfied in practice. To address the above problem, we design a module to learn a latent variable representing the down-scaling loss conditional on low-resolution(LR) images according to the information entropy theory. Unlike previous methods, the down-scaling loss is recovered through the proposed module whose inputs are both LR images and predefined distributions. The difference between the high-resolution(HR) image and the low-resolution(LR) image is represented with a latent variable through a lossless invertible neural network. In addition, a conditional entropy loss is proposed to train the invertible neural network to suppress the conditional entropy between LR images and latent variables. It helps to decrease the effect of the latent variable to generate the HR image from LR images during the up-scaling procedure. We evaluate the proposed method on widely used data sets, and the experimental results demonstrate that our proposed method performs favorably against SOTA methods in terms of PSNR and SSIM metrics.

Invertible network for unpaired low-light image enhancement

Existing unpaired image enhancement approaches prefer to employ the traditional two-way generative adversarial network (GAN) framework, in which two convolutional neural network (CNN) generators are deployed for enhancement and degradation separately. However, such data-driven models ignore the inherent characteristics of transformation between the low-light and normal-light images, leading to unstable training and artifacts. Here, we propose to leverage the invertible neural network to enhance the low-light images in the forward process and degrade the unpaired normal-light photograph inversely. The generated and real images are then fed into discriminators for adversarial learning. In addition to the adversarial loss, we design transformation-consistent loss to ensure the stability of training, detail-preserving loss to preserve more image details, and reversibility loss to alleviate the over-exposure problem. Moreover, we present a progressive self-guided enhancement process in inference and achieve favorable performance against the state-of-the-art methods.

Investigation of inverse design of multilayer thin-films with conditional invertible neural networks

In this work, we apply conditional invertible neural networks (cINN) to inversely design multilayer thin-films given an optical target in order to overcome limitations of state-of-the-art optimization approaches. Usually, state-of-the-art algorithms depend on a set of carefully chosen initial thin-film parameters or employ neural networks which must be retrained for every new application. We aim to overcome those limitations by training the cINN to learn the loss landscape of all thin-film configurations within a training dataset. We show that cINNs can generate a stochastic ensemble of proposals for thin-film configurations that are reasonably close to the desired target depending only on random variables. By refining the proposed configurations further by a local optimization, we show that the generated thin-films reach the target with significantly greater precision than comparable state-of-the-art approaches. Furthermore, we tested the generative capabilities on samples which are outside of the training data distribution and found that the cINN was able to predict thin-films for out-of-distribution targets, too. The results suggest that in order to improve the generative design of thin-films, it is instructive to use established and new machine learning methods in conjunction in order to obtain the most favorable results.

A Novel Denoising Approach Based on Improved Invertible Neural Networks for Real-Time Conveyor Belt Monitoring

Powerful belt conveyor is the crucial transportation equipment for coal mining. Keeping its stable operation is essential to ensure the coal mining efficiency. At present, damage detection systems based on machine vision are widely studied and deployed. Clear images are the foundation for successful damage detection. However, in the harsh environment of underground coal seam, it is difficult to capture high-quality images, which limits the performance of the machine vision-based techniques for the conveyor belt damage detection. To solve this issue, an image denoising and enhancement approach is proposed based on the invertible neural network (INN). First, a new res-block is introduced to improve the feature extraction ability of the proposed model. Then, to take advantage of the INN architecture, input images are split by the generative flow (GLOW) coupling block in the forward path and the noise among the high-frequency (HF) features is discarded in the inverse path. An improved trainable guided filter (TGF) is proposed to reconstruct the HF images. Besides, an improved multiple loss function is designed to keep detailed feature information and eliminate the effect of noise. The comparisons and experimental results demonstrate that the proposed denoising INN (DeINN) presents excellent image denoising ability and outperforms several existing popular methods. Finally, a real-world application demonstrates that the proposed DeINN satisfies the industrial requirements on belt damage detection.

Inference of astrophysical parameters with a conditional invertible neural network

Conditional Invertible Neural Networks (cINNs) provide a new technique for the inference of free model parameters by enabling the creation of posterior distributions. With these distributions, the mean parameter values, their uncertainties and the correlations between the parameters can be estimated. In this contribution, we summarize the functionality of cINNs, which are based on normalizing flows, and present the application of this new method to a scenario from astroparticle physics. We show that it is possible to constrain properties of the currently unknown sources of ultra-high-energy cosmic rays and compare the posterior distributions obtained with the network to those acquired using the classic Markov Chain Monte Carlo method.

Autoencoder-extended Conditional Invertible Neural Networks for Unfolding Signal Traces

The reconstruction of cosmic ray-induced air showers from measurements of radio waves constitutes a major challenge. In this work, we focus on recovering the full three-dimensional electromagnetic field from two recorded signal traces of an antenna station covering two horizontal polarization directions. The simulated field is folded by a direction and frequency-dependent characteristic antenna response pattern, resulting in voltage signal traces as a function of time. Both signal traces are contaminated by simulated background noise. We use conditional Invertible Neural Networks (cINNs) to learn posterior distributions, from which the most likely electromagnetic field given a measured signal trace can be inferred. To improve robustness, we extend the method with an autoencoder by reducing the parameter phase space and decoupling the cINN from specific data shapes. Thereby, each signal trace is condensed into a small number of abstract parameters in the latent space on which the cINN operates. The presented method shows promising results and can be transferred to other unfolding problems where the recovery of the pre-measurement state is of interest.

Predicting reaction pathways and accelerating protein conformation sampling with machine learning.

It is challenging to sample molecular processes with large activation energies using molecular dynamics simulations. Current enhanced sampling methodologies rely on the identification of appropriate reaction coordinates that describe reaction pathways in the free energy surface (FES). A new method, log-probability estimation via invertible neural networks for enhanced sampling (LINES), has been developed for identifying reaction coordinates and facilitating the sampling. The iterative scheme employs a normalizing flow machine learning model to learn the underlying FES in a molecular system with data generated from molecular dynamics and then applies a gradient-based optimization method to identify reaction coordinates. The model has been rigorously validated through the prediction of a water-based reaction coordinate that improves the rate of reversible binding for a β-cyclodextrin/cyclobutanol system. In addition, LINES has also been successfully applied towards the study of unfolding/refolding of chignolin, a small peptide with a stable folded conformation. Most recently, LINES has been implemented to extract the binding sites from the predicted reaction coordinates for ligand binding with human T-cell immunoglobulin and mucin domain containing protein-3 (TIM3). TIM3-ligand binding plays key roles in the immune system. In all examples, LINES demonstrates a robust approach for reaction coordinate identification that can accurately capture characteristics of a high-dimensional FES and then predict reaction pathways between different metastable states in the FES.

Noise-Net: determining physical properties of H iiregions reflecting observational uncertainties

ABSTRACTStellar feedback, the energetic interaction between young stars and their birthplace, plays an important role in the star formation history of the Universe and the evolution of the interstellar medium. Correctly interpreting the observations of star-forming regions is essential to understand stellar feedback, but it is a non-trivial task due to the complexity of the feedback processes and degeneracy in observations. In our recent paper, we introduced a conditional invertible neural network (cINN) that predicts seven physical properties of star-forming regions from the luminosity of 12 optical emission lines as a novel method to analyse degenerate observations. We demonstrated that our network, trained on synthetic star-forming region models produced by the warpfield-emission predictor (warpfield-emp), could predict physical properties accurately and precisely. In this paper, we present a new updated version of the cINN that takes into account the observational uncertainties during network training. Our new network named Noise-Net reflects the influence of the uncertainty on the parameter prediction by using both emission-line luminosity and corresponding uncertainties as the necessary input information of the network. We examine the performance of the Noise-Net as a function of the uncertainty and compare it with the previous version of the cINN, which does not learn uncertainties during the training. We confirm that the Noise-Net outperforms the previous network for the typical observational uncertainty range and maintains high accuracy even when subject to large uncertainties.

Fast Bayesian Inversion of Airborne Electromagnetic Data Based on the Invertible Neural Network

The inversion of airborne electromagnetic (AEM) data suffers from severe non-uniqueness of the solution. Bayesian inference provides the means to estimate structural uncertainty with a rich suite of statistical information. However, conventional Bayesian methods are computationally demanding in nonlinear inversions, especially considering the huge volumes of observational data, and thus are not feasible in practice. In this study, we develop a fast Bayesian inversion operator based on the invertible neural network (INN) to fully explore the posterior distribution and quantitatively evaluate the model uncertainty. The INN uses a latent variable to capture the information loss during measurement and constructs bijective mappings between AEM data and the resistivity model. We also introduce another noise variable into the INN to account for data uncertainties. Synthetic tests demonstrate that the INN can effectively recover the posterior distribution by a relatively small ensemble of predicted resistivity models whose AEM responses show a significant agreement with the true signal. We also apply the INN inversion operator to a field data set and obtain results consistent with previous studies. The INN shows considerable adaptability to field observations and strong noise robustness. Meanwhile, the INN delivers the inversion result with posterior model distribution for 23366 AEM time series in 20 seconds on a common PC. The inversion efficiency can be further improved for large data set due to its natural parallelizability. The proposed INN method can support fast Bayesian inversion of AEM data and offer tremendous potential for near real-time uncertainty evaluation of underground structures.

Probabilistic invertible neural network for inverse design space exploration and reasoning

<abstract> <p>Invertible neural network (INN) is a promising tool for inverse design optimization. While generating forward predictions from given inputs to the system response, INN enables the inverse process without much extra cost. The inverse process of INN predicts the possible input parameters for the specified system response qualitatively. For the purpose of design space exploration and reasoning for critical engineering systems, accurate predictions from the inverse process are required. Moreover, INN predictions lack effective uncertainty quantification for regression tasks, which increases the challenges of decision making. This paper proposes the probabilistic invertible neural network (P-INN): the epistemic uncertainty and aleatoric uncertainty are integrated with INN. A new loss function is formulated to guide the training process with enhancement in the inverse process accuracy. Numerical evaluations have shown that the proposed P-INN has noticeable improvement on the inverse process accuracy and the prediction uncertainty is reliable.</p> </abstract>

DeepMIH: Deep Invertible Network for Multiple Image Hiding.

Multiple image hiding aims to hide multiple secret images into a single cover image, and then recover all secret images perfectly. Such high-capacity hiding may easily lead to contour shadows or color distortion, which makes multiple image hiding a very challenging task. In this paper, we propose a novel multiple image hiding framework based on invertible neural network, namely DeepMIH. Specifically, we develop an invertible hiding neural network (IHNN) to innovatively model the image concealing and revealing as its forward and backward processes, making them fully coupled and reversible. The IHNN is highly flexible, which can be cascaded as many times as required to achieve the hiding of multiple images. To enhance the invisibility, we design an importance map (IM) module to guide the current image hiding based on the previous image hiding results. In addition, we find that the image hidden in the high-frequency sub-bands tends to achieve better hiding performance, and thus propose a low-frequency wavelet loss to constrain that no secret information is hidden in the low-frequency sub-bands. Experimental results show that our DeepMIH significantly outperforms other state-of-the-art methods, in terms of hiding invisibility, security and recovery accuracy on a variety of datasets.

Factorized Industrial Anomaly Detection and Localization

Convolutional neural networks trained on large datasets can generalize various down-streaming tasks, including industrial anomaly detection and localization, which is critical in modern large-scale industrial manufacturing. Whereas previous methods have demonstrated that the feature fusion strategy across multiple layers is effective for better performance on industrial anomaly detection and localization, they lack flexibility in intervening and manipulating the local and global information composition process. Through experiments, we demonstrate that the brute-force feature fusion strategy used in previous methods leads to sub-optimal performance in most industrial anomaly detection scenarios. To this end, we propose a novel feature factorization and reversion framework based on invertible neural networks, enabling the selective emphasis or suppression of distinct information in a continuous space by request to fit various preferences for detecting different abnormalities. The preferred local and global info-combination for detecting different defects on 15 objects is studied by experiments exhaustively on the popular benchmark MVTec-AD. Based on feature factorization and reversion, our method is able to outperform previous state-of-the-art methods by a noticeable margin, achieving an image-level anomaly detection AUROC score of up to 99.67% (previously 99.4%), pixel-level anomaly localization AUROC score of 98.61% (previously 98.5%), and AUPRO of 95.15% (previously 94.6%), which validates the effectiveness of the proposed method for industrial anomaly detection and localization.

Neural Information Squeezer for Causal Emergence.

Conventional studies of causal emergence have revealed that stronger causality can be obtained on the macro-level than the micro-level of the same Markovian dynamical systems if an appropriate coarse-graining strategy has been conducted on the micro-states. However, identifying this emergent causality from data is still a difficult problem that has not been solved because the appropriate coarse-graining strategy can not be found easily. This paper proposes a general machine learning framework called Neural Information Squeezer to automatically extract the effective coarse-graining strategy and the macro-level dynamics, as well as identify causal emergence directly from time series data. By using invertible neural network, we can decompose any coarse-graining strategy into two separate procedures: information conversion and information discarding. In this way, we can not only exactly control the width of the information channel, but also can derive some important properties analytically. We also show how our framework can extract the coarse-graining functions and the dynamics on different levels, as well as identify causal emergence from the data on several exampled systems.

Lossless Image Steganography Based on Invertible Neural Networks.

Image steganography is a scheme that hides secret information in a cover image without being perceived. Most of the existing steganography methods are more concerned about the visual similarity between the stego image and the cover image, and they ignore the recovery accuracy of secret information. In this paper, the steganography method based on invertible neural networks is proposed, which can generate stego images with high invisibility and security and can achieve lossless recovery for secret information. In addition, this paper introduces a mapping module that can compress information actually embedded to improve the quality of the stego image and its antidetection ability. In order to restore message and prevent loss, the secret information is converted into a binary sequence and then embedded in the cover image through the forward operation of the invertible neural networks. This information will then be recovered from the stego image through the inverse operation of the invertible neural networks. Experimental results show that the proposed method in this paper has achieved competitive results in the visual quality and safety of stego images and achieved 100% accuracy in information extraction.

Invertible Neural BRDF for Object Inverse Rendering

We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering, i.e., joint estimation of reflectance and natural illumination from a single image of an object of known geometry. The BRDF is expressed with an invertible neural network, namely, normalizing flow, which provides the expressive power of a high-dimensional representation, computational simplicity of a compact analytical model, and physical plausibility of a real-world BRDF. We extract the latent space of real-world reflectance by conditioning this model, which directly results in a strong reflectance prior. We refer to this model as the invertible neural BRDF model (iBRDF). We also devise a deep illumination prior by leveraging the structural bias of deep neural networks. By integrating this novel BRDF model and reflectance and illumination priors in a MAP estimation formulation, we show that this joint estimation can be computed efficiently with stochastic gradient descent. We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data and demonstrate its use in object inverse rendering on a number of synthetic and real images. The results show new ways in which deep neural networks can help solve challenging radiometric inverse problems.

Modelling internal structure of differentiated asteroids via data-driven approach

ABSTRACT This paper is devoted to an interdisciplinary method modelling the internal structure of differentiated asteroids via a data-driven approach called invertible neural networks (INNs). The model estimation of the internal structure can be generalized as an inverse problem of estimating internal parameters from a set of observations. Previous works (e.g. Park et al. 2014; Takahashi and Scheeres 2014) used the full gravity field data measures to derive the heterogeneous mass distribution. However, in our method, only the flight state of the spacecraft is adopted as the observation data. Since the internal parameters may not be uniquely determined, typical feedforward neural networks cannot simply be applied to such an inverse problem. The INNs adopted in this paper can ‘read’ the interior information from a flight trajectory of the spacecraft directly. The INNs are employed to establish the two-directional mapping between the group of physical parameters and the set of flight state observations of position and velocity. The INNs are trained in a bi-directional way using four losses. Finally, the performances of the trained networks are shown in both overfit and underfit situations where the internal structure of asteroids can be estimated by this INNs-based method accurately and effectively. The results also show that even when the degeneracy occurs, the true solution still falls inside the estimation distribution.

Variational inference of fractional Brownian motion with linear computational complexity.

We introduce a simulation-based, amortized Bayesian inference scheme to infer the parameters of random walks. Our approach learns the posterior distribution of the walks' parameters with a likelihood-free method. In the first step a graph neural network is trained on simulated data to learn optimized low-dimensional summary statistics of the random walk. In the second step an invertible neural network generates the posterior distribution of the parameters from the learned summary statistics using variational inference. We apply our method to infer the parameters of the fractional Brownian motion model from single trajectories. The computational complexity of the amortized inference procedure scales linearly with trajectory length, and its precision scales similarly to the Cramér-Rao bound over a wide range of lengths. The approach is robust to positional noise, and generalizes to trajectories longer than those seen during training. Finally, we adapt this scheme to show that a finite decorrelation time in the environment can furthermore be inferred from individual trajectories.

ERGO-ML I: inferring the assembly histories of IllustrisTNG galaxies from integral observable properties via invertible neural networks

ABSTRACT A fundamental prediction of the ΛCDM cosmology is the hierarchical build-up of structure and therefore the successive merging of galaxies into more massive ones. As one can only observe galaxies at one specific time in the cosmic history, this merger history remains, in principle, unobservable. By using the TNG100 simulation of the IllustrisTNG project, we show that it is possible to infer the unobservable stellar assembly and merger history of central galaxies from their observable properties by using machine learning techniques. In particular, in this first paper of ERGO-ML (Extracting Reality from Galaxy Observables with Machine Learning), we choose a set of seven observable integral properties of galaxies to infer the stellar ex-situ fraction, the average merger lookback times and mass ratios, and the lookback time and stellar mass of the last major merger. To infer the posterior distribution for these parameters and hence estimate the uncertainties in the predictions, we use a conditional Invertible Neural Network (cINN). We find that the stellar ex-situ fraction and the time of the last major merger are well-determined by the selected set of observables, that the mass-weighted merger mass ratio is unconstrained, and that, beyond stellar mass, stellar morphology and stellar age are the most informative properties. Finally, we show that the cINN recovers the remaining unexplained scatter and secondary cross-correlations. Overall, this is a first step towards a tool that can be applied to large galaxy surveys in order to infer unobservable properties of the galaxies’ past, enabling empirical studies of galaxy evolution enriched by cosmological simulations.