Low-dimensional Model Research Articles

Deposition of graphenic nanomaterials from elevated temperature premixed stagnation flames

The work examines the unique nanostructure of carbon nanoparticles deposited from sooting premixed flames with flame temperatures exceeding 2200 K. This flame temperature regime has previously been shown to transition from typical soot formation conditions to a regime whereby the flame-form carbon adopts a nanostructure considerably more ordered than soot. Graphenic carbon deposits observed by High-resolution TEM (HRTEM) are reported here corroborating previous Raman spectroscopy evidence. The use of premixed stretch-stabilized flames enables particle production in the high-temperature regime under a flow field amenable to low-dimensional flame modeling. Although the flame flow configuration is relatively simple, three sample preparation methods are used to assess the representation of true carbon properties as they exist in the flame. HRTEM imaging is carried out on carbon particle samples prepared by rapid-insertion deposition, aerosol dilution probe deposition and carbon particle film deposition. Images from rapid-insertion samples show amorphous particles in the lightly sooting flame and turbostratic particles in the heavy sooting flame. There is trace evidence of graphenic structure in rapid-insertion samples but the most striking particles on the TEM grid are graphite nanocrystals presumably formed by a new artificial crystallization process. HRTEM images of particles collected over time by diluted aerosol deposition and film deposition show clear graphenic structures. Overall, the carbon nanostructure observed by HRTEM is a mixture of amorphous, turbostratic and graphenic carbon lattices depending on the flame condition and sampling method. The current work highlights potential impacts of higher flame temperatures and higher equivalence ratio on deposited flame-formed carbon. Namely, graphenic particle structure is observed in rapid-insertion deposition samples but graphene portions are most abundant in aerosol dilution and carbon particle film deposition samples. This may indicate that graphene structures grow on the deposition surface over time.

Variational deep learning of equilibrium transition path ensembles.

We present a time-dependent variational method to learn the mechanisms of equilibrium reactive processes and efficiently evaluate their rates within a transition path ensemble. This approach builds off of the variational path sampling methodology by approximating the time-dependent commitment probability within a neural network ansatz. The reaction mechanisms inferred through this approach are elucidated by a novel decomposition of the rate in terms of the components of a stochastic path action conditioned on a transition. This decomposition affords an ability to resolve the typical contribution of each reactive mode and their couplings to the rare event. The associated rate evaluation is variational and systematically improvable through the development of a cumulant expansion. We demonstrate this method in both over- and under-damped stochastic equations of motion, in low-dimensional model systems, and in the isomerization of a solvated alanine dipeptide. In all examples, we find that we can obtain quantitatively accurate estimates of the rates of the reactive events with minimal trajectory statistics and gain unique insights into transitions through the analysis of their commitment probability.

Natural statistics support a rational account of confidence biases

Previous work has sought to understand decision confidence as a prediction of the probability that a decision will be correct, leading to debate over whether these predictions are optimal, and whether they rely on the same decision variable as decisions themselves. This work has generally relied on idealized, low-dimensional models, necessitating strong assumptions about the representations over which confidence is computed. To address this, we used deep neural networks to develop a model of decision confidence that operates directly over high-dimensional, naturalistic stimuli. The model accounts for a number of puzzling dissociations between decisions and confidence, reveals a rational explanation of these dissociations in terms of optimization for the statistics of sensory inputs, and makes the surprising prediction that, despite these dissociations, decisions and confidence depend on a common decision variable.

Classical structural identifiability methodology applied to low-dimensional dynamic systems in receptor theory.

Mathematical modelling has become a key tool in pharmacological analysis, towards understanding dynamics of cell signalling and quantifying ligand-receptor interactions. Ordinary differential equation (ODE) models in receptor theory may be used to parameterise such interactions using timecourse data, but attention needs to be paid to the theoretical identifiability of the parameters of interest. Identifiability analysis is an often overlooked step in many bio-modelling works. In this paper we introduce structural identifiability analysis (SIA) to the field of receptor theory by applying three classical SIA methods (transfer function, Taylor Series and similarity transformation) to ligand-receptor binding models of biological importance (single ligand and Motulsky-Mahan competition binding at monomers, and a recently presented model of a single ligand binding at receptor dimers). New results are obtained which indicate the identifiable parameters for a single timecourse for Motulsky-Mahan binding and dimerised receptor binding. Importantly, we further consider combinations of experiments which may be performed to overcome issues of non-identifiability, to ensure the practical applicability of the work. The three SIA methods are demonstrated through a tutorial-style approach, using detailed calculations, which show the methods to be tractable for the low-dimensional ODE models.

Low-dimensional data-based surrogate model of a continuum-mechanical musculoskeletal system based on non-intrusive model order reduction

Over the last decades, computer modeling has evolved from a supporting tool for engineering prototype design to an ubiquitous instrument in non-traditional fields such as medical rehabilitation. This area comes with unique challenges, e.g. the complex modeling of soft tissue or the analysis of musculoskeletal systems. Conventional modeling approaches like the finite element (FE) method are computationally costly when dealing with such models, limiting their usability for real-time simulation or deployment on low-end hardware, if the model at hand cannot be simplified without losing its expressiveness. Non-traditional approaches such as surrogate modeling using data-driven model order reduction are used to make complex high-fidelity models more widely available regardless. They often involve a dimensionality reduction step, in which the high-dimensional system state is transformed onto a low-dimensional subspace or manifold, and a regression approach to capture the reduced system behavior. While most publications focus on one dimensionality reduction, such as principal component analysis (PCA) (linear) or autoencoder (nonlinear), we consider and compare PCA, kernel PCA, autoencoders, as well as variational autoencoders for the approximation of a continuum-mechanical system. In detail, we demonstrate the benefits of the surrogate modeling approach on a complex musculoskeletal system of a human upper-arm with severe nonlinearities and physiological geometry. We consider both, the model’s deformation and the internal stress as the two main quantities of interest in a FE context. By doing so we are able to create computationally low-cost surrogate models which capture the system behavior with high approximation quality and fast evaluations.

Improving human intuition for vision-based freshness prediction of Citrus reticulata Blanco using machine learning

Prediction of citrus freshness is essential for the fruit market to better arrange storage, transportation, and wholesale. A rough judgement is often conducted using visual inspection due to its equipment-free nature and high efficiency, but the accuracy is relatively low due to its unsatisfactory objectiveness. This study explored and verified the visual features adopted by machine learning models capable of precise freshness prediction, and these features were then used to train human intuition for improved performance. Images of mandarins (Citrus reticulata Blanco) collected at 5 different storage periods in a continuous 28 days were first categorized by a high-dimensional Convolutional Neural Network (CNN) model, achieving an accuracy of 95.6% significantly higher than that of human inspectors. Visualization of the model indicated corresponding features that might be adopted for freshness prediction, and these features were further analyzed using a low-dimensional and explainable decision tree model for validation. For the consumer sensory panels, the test group was trained using the validated features, and they eventually obtained a freshness prediction accuracy of 87.3%, which was 10.0% higher than that of the control group and demonstrated the efficacy of the proposed training process. The presented pipeline can be readily repeated to improve human intuition for freshness prediction for different citrus species and other fruit crops.

IterDE: An Iterative Knowledge Distillation Framework for Knowledge Graph Embeddings

Knowledge distillation for knowledge graph embedding (KGE) aims to reduce the KGE model size to address the challenges of storage limitations and knowledge reasoning efficiency. However, current work still suffers from the performance drops when compressing a high-dimensional original KGE model to a low-dimensional distillation KGE model. Moreover, most work focuses on the reduction of inference time but ignores the time-consuming training process of distilling KGE models. In this paper, we propose IterDE, a novel knowledge distillation framework for KGEs. First, IterDE introduces an iterative distillation way and enables a KGE model to alternately be a student model and a teacher model during the iterative distillation process. Consequently, knowledge can be transferred in a smooth manner between high-dimensional teacher models and low-dimensional student models, while preserving good KGE performances. Furthermore, in order to optimize the training process, we consider that different optimization objects between hard label loss and soft label loss can affect the efficiency of training, and then we propose a soft-label weighting dynamic adjustment mechanism that can balance the inconsistency of optimization direction between hard and soft label loss by gradually increasing the weighting of soft label loss. Our experimental results demonstrate that IterDE achieves a new state-of-the-art distillation performance for KGEs compared to strong baselines on the link prediction task. Significantly, IterDE can reduce the training time by 50% on average. Finally, more exploratory experiments show that the soft-label weighting dynamic adjustment mechanism and more fine-grained iterations can improve distillation performance.

Improving the Accuracy of Coal-Rock Dynamic Hazard Anomaly Detection Using a Dynamic Threshold and a Depth Auto-Coding Gaussian Hybrid Model

A coal-rock dynamic disaster is a rapid instability and failure process with dynamic effects and huge disastrous consequences that occurs in coal-rock mass under mining disturbance. Disasters lead to catastrophic consequences, such as mine collapse, equipment damage, and casualties. Early detection can prevent the occurrence of disasters. However, due to the low accuracy of anomaly detection, disasters still occur frequently. In order to improve the accuracy of anomaly detection for coal-rock dynamic disasters, this paper proposes an anomaly detection method based on a dynamic threshold and a deep self-encoded Gaussian mixture model. First, pre-mining data were used as the initial threshold, and the subsequent continuously arriving flow data were used to dynamically update the threshold to solve the impact of artificially setting the threshold on the detection accuracy. Second, feature dimensionality reduction and reorganization of the data were carried out, and low-dimensional feature representation and feature reconstruction error modeling were used to solve the difficulty of extracting features from high-dimensional data in real time. Finally, through the end-to-end optimization calculation of the energy probability distribution between different categories for anomaly detection, the problem that key abnormal information may be lost due to dimensionality reduction was solved and accurate detection of monitoring data was realized. Experimental results showed that this method has better performance than other methods.

Proper Orthogonal Decomposition Mode Coefficient Interpolation: A Non-Intrusive Reduced-Order Model for Parametric Reactor Kinetics

In this paper, a non-intrusive reduced-order model (ROM) for parametric reactor kinetics simulations is presented. Time-dependent ROMs are notoriously data intensive and difficult to implement when nonlinear multiphysics phenomena are considered. These challenges are exacerbated when parametric dependencies are included. The proper orthogonal decomposition mode coefficient interpolation (POD-MCI) ROM presented in this work can be constructed directly from lower-dimensional full-order model (FOM) outputs and is independent of the underlying model. This greatly alleviates the data requirement of many existing ROMs and can be used without modification on arbitrarily complex models or experimental data. The POD-MCI ROM is demonstrated on a number of examples and yields accurate characterizations of the parametric behaviors of both FOM outputs and derived quantities of interest within the selected parameter spaces, at extremely attractive computational speedup factors relative to FOMs.

Interactions between two adjacent convection rolls in turbulent Rayleigh-Bénard convection

Neighboring convection rolls are experimentally found to be stable in either counter-rotating or co-rotating states, with stochastic switching between the two states. These states are quantitatively described by analytic solutions of a low-dimensional stochastic model in which turbulent thermal diffusion between the rolls provides the forcing that stabilizes both states, using the same turbulent thermal diffusivity parameter that characterizes heat transport. This is presented within a generalized model framework that can account for the geometry of the convection cells and other forces.

Learning digital emulators for closed architecture machine tool controllers

Machine tool controllers (MTCs) are evolving rapidly in response to the growing precision requirements, industry-wide thrusts to integrate cloud applications, external sensors and other data sources, as well to assure cybersecurity. However, most of the MTCs have closed architecture. This severely impedes the innovations to enhance their performance as well as assure their safety and security. Digital emulators, as alternatives to real closed architecture MTCs, are considered essential to assess the performance of the closed MTCs and various innovations therein. We present a machine learning method to create digital emulators that can mimic the dynamics of closed architecture MTCs. The proposed method is based on interrogating the controller using a set of production recipes (e.g., G-codes) and employing the response of the controller to learn a low-dimensional parameterization of the underlying architecture. The low-dimensional base model captures the key aspects of the structure and dynamic connectivity among the various components of an MTC. The parameters of the base model are tuned based on the outputs of an MTC gathered using standard data exchange protocols (e.g. OPC UA, MTConnect). We applied the proposed approach to develop emulators for a SIEMENS controller that mimics X–Y motion of their 2-axis motion control systems. The current implementation of the emulator can track the measured path trajectories up to 0.23 mm accuracy for various geometries tested. Our approach can be used to generalize emulator development for different types of real world MTCs.

Difference Analysis of Different Explosion Load Simulation Methods Based on LS-DYNA

LS-DYNA provides a variety of different explosion load simulation methods, which are widely used in the field of explosion shock. Compared with the direct application of explosion load on the structural surface, the simulation method of establishing the air domain is more complex, but it can reproduce the evolution process of the explosion flow field and consider the interaction between the shock wave and the structure. The simulation results are more consistent with the actual situation. In this paper, the four commonly used explosion load simulation methods in LS-DYNA software are compared and analyzed, and the sensitivity analysis of grid size is carried out. The differences in the results of the four explosion load simulation methods under typical explosion conditions are compared and analyzed from the aspects of explosion shock wave waveform, arrival time, overpressure peak and specific impulse. The analysis results show that the proportional detonation distance of 0.8 m/kg1/3 is the boundary point of grid sensitivity ; under the same grid size, different explosion load simulation methods have little effect on shock wave arrival time, but have obvious influence on overpressure peak value and specific impulse. It is critical to reasonably select the termination time of the low-dimensional model when using the mapping method ; method 1 and method 2 are preferred for simulation without considering the computational cost or small model size. When the model is large, method 3 and method 4 should be preferred to improve the computational efficiency.

Learning perturbation-inducible cell states from observability analysis of transcriptome dynamics

A major challenge in biotechnology and biomanufacturing is the identification of a set of biomarkers for perturbations and metabolites of interest. Here, we develop a data-driven, transcriptome-wide approach to rank perturbation-inducible genes from time-series RNA sequencing data for the discovery of analyte-responsive promoters. This provides a set of biomarkers that act as a proxy for the transcriptional state referred to as cell state. We construct low-dimensional models of gene expression dynamics and rank genes by their ability to capture the perturbation-specific cell state using a novel observability analysis. Using this ranking, we extract 15 analyte-responsive promoters for the organophosphate malathion in the underutilized host organism Pseudomonas fluorescens SBW25. We develop synthetic genetic reporters from each analyte-responsive promoter and characterize their response to malathion. Furthermore, we enhance malathion reporting through the aggregation of the response of individual reporters with a synthetic consortium approach, and we exemplify the library’s ability to be useful outside the lab by detecting malathion in the environment. The engineered host cell, a living malathion sensor, can be optimized for use in environmental diagnostics while the developed machine learning tool can be applied to discover perturbation-inducible gene expression systems in the compendium of host organisms.

Health-aware model predictive energy management for fuel cell electric vehicle based on hybrid modeling method

Fuel cell lifetime is strongly affected by dynamic conditions. Most existing energy management works only focus on the fuel cell durability protection from the perspective of output power slope, without deeply considering the influence of the important parameters inside the stack. However, considering the variation of stack internal parameters (mechanism analysis) is more significant for fuel cell lifetime evaluation. In this paper, a health-aware model predictive control (HA-MPC) energy management strategy is proposed for fuel cell electric vehicle. A fuel cell health state model is established from the perspective of stack hydrogen excess ratio (HER), oxygen excess ratio (OER) and humidity through the hybrid modeling method. The fuel cell mechanism model and the low-dimensional data-driven model are established through the grey-box model estimation method and genetic algorithm-based radial basis function (GA-RBF) neural network. Then the objective function of energy management strategy is developed considering the total equivalent hydrogen consumption and stack improper parameter changes of HER, OER and humidity. Comparing with model predictive control strategy based on the typical power cost function, the HA-MPC can effectively reduce the steep drop of HER and OER in low power region by 3.58% and 4.41%, which can protect the fuel cell system lifetime.

The Mori–Zwanzig formulation of deep learning

We develop a new formulation of deep learning based on the Mori–Zwanzig (MZ) formalism of irreversible statistical mechanics. The new formulation is built upon the well-known duality between deep neural networks and discrete dynamical systems, and it allows us to directly propagate quantities of interest (conditional expectations and probability density functions) forward and backward through the network by means of exact linear operator equations. Such new equations can be used as a starting point to develop new effective parameterizations of deep neural networks and provide a new framework to study deep learning via operator-theoretic methods. The proposed MZ formulation of deep learning naturally introduces a new concept, i.e., the memory of the neural network, which plays a fundamental role in low-dimensional modeling and parameterization. By using the theory of contraction mappings, we develop sufficient conditions for the memory of the neural network to decay with the number of layers. This allows us to rigorously transform deep networks into shallow ones, e.g., by reducing the number of neurons per layer (using projection operators), or by reducing the total number of layers (using the decay property of the memory operator).

Frequency-domain analysis of membrane polarization in two-compartment model neurons with weak alternating electric fields.

Transcranial alternating current stimulation (tACS) is widely used in studying brain functions and the treatment of neuropsychiatric diseases in a frequency-specific manner. However, how tACS works on neuronal activity has been poorly understood. In this paper, we use linear system analysis to investigate how weak alternating electric fields (EFs) affect the membrane polarization of neurons in the frequency domain. Two biophysically realistic conductance-based two-compartment models of cortical pyramidal neurons are developed to simulate subthreshold membrane polarization with weak alternating EFs. We linearize the original nonlinear models at the stable equilibrium points and further simplify them to the two- or three-dimensional linear systems. Thus, we calculate the transfer functions of the low-dimensional linear models to model neuronal polarization patterns. Based on the transfer functions, we compute the amplitude- and phase-frequency characteristics to describe the relationship between weak EFs and membrane polarization. We also computed the parameters (gain, zeros, and poles) and structures (the number of zeros and poles) of transfer functions to reveal how neuronal intrinsic properties affect the parameters and structure of transfer functions and thus the frequency-dependent membrane polarization with alternating EFs. We find that the amplitude and phase of membrane polarization both strongly depended on EF frequency, and these frequency responses are modulated by the intrinsic properties of neurons. The compartment geometry, internal coupling conductance, and ionic currents (except Ih) affect the frequency-dependent polarization by mainly changing the gain and pole of transfer functions. Larger gain contributes to larger amplitude-frequency characteristics. The closer the pole is to the imaginary axis, the lower phase-frequency characteristics. However, Ih changes the structure of transfer function in the dendrite by introducing a new pair of zero-pole points, which decrease the amplitude at low frequencies and thus lead to a visible resonance. These results highlight the effects of passive properties and active ion currents on subthreshold membrane polarization with alternating EFs in the frequency domain, which provide an explainable connection of how intrinsic properties of neurons modulate the neuronal input-output functions with weak EF stimulation.

Risk Estimation of Typhoon Disaster Based on Three-Dimensional Information Diffusion Method

In the context of the increasing frequency and intensity of natural disasters, assessing the risk of typhoon disasters can provide significant assistance for risk control and emergency management of typhoon disasters. In this paper, based on the three-dimensional information diffusion method, the formal expected loss model is transformed into a computable typhoon risk assessment model. The fuzzy information in the small sample data is deeply mined, and the typhoon disaster risk assessment with the expected loss as the connotation is carried out, and the probability density distribution estimation of disaster-causing factors at different levels and the functional relationship identification between disaster-causing factors at different levels and direct economic loss rate are realized by using the information matrix. At the same time, combined with the frequency of typhoon occurrence, the annual risk of disasters is predicted to make up for the problem of insufficient marine environmental data and improve the calculation accuracy of risk assessment models. Taking Guangdong Province as an example, a typhoon risk assessment was conducted, estimating the probability distribution, direct economic loss rate distribution, and annual loss expectation of typhoon disasters under different wind speed scales and extreme wave heights. The results indicate that the risk estimation value of the three-dimensional information diffusion model is higher than that of the traditional model, which weakens the limitations of the low-dimensional information diffusion model and makes the evaluation results more reasonable and reliable.

Low-Dimensional Models for Aerofoil Icing Predictions

Determining the aero-icing characteristics is key for safety assurance in aviation, but it may be a computationally expensive task. This work presents a framework for the development of low-dimensional models for application to aerofoil icing. The framework builds on: an adaptive sampling strategy to identify the local, nonlinear features across the icing envelope for continuous intermittent icing; a classic technique based on Proper Orthogonal Decomposition, and a modern Neural Network architecture. The extreme diversity in simulated ice shapes, from smooth and streamlined to rugged and irregular shapes, motivated the use of an unsupervised classification of the ice shapes. This allowed deploying the Proper Orthogonal Decomposition locally within each sub-region, sensibly improving the prediction accuracy over the global model. On the other hand, the Neural Network architecture and the convolutional auto-encoder were found insensitive to the complexity in ice shapes. A strong correlation was found to exist between the ice shape, resulting ice mass and aerodynamic performance of the iced aerofoil, both in terms of the average and variance. On average, rime ice causes a loss of maximum lift coefficient of 21.5% compared to a clean aerofoil, and the average ice thickness is 0.9% of the aerofoil chord. For glaze ice, the average loss of maximum lift coefficient is 46.5% and the average ice thickness is 2.1%. Glaze ice was also found to have three times more surface coverage than rime ice.

Predicting rare events using neural networks and short-trajectory data

Estimating the likelihood, timing, and nature of events is a major goal of modeling stochastic dynamical systems. When the event is rare in comparison with the timescales of simulation and/or measurement needed to resolve the elemental dynamics, accurate prediction from direct observations becomes challenging. In such cases a more effective approach is to cast statistics of interest as solutions to Feynman-Kac equations (partial differential equations). Here, we develop an approach to solve Feynman-Kac equations by training neural networks on short-trajectory data. Our approach is based on a Markov approximation but otherwise avoids assumptions about the underlying model and dynamics. This makes it applicable to treating complex computational models and observational data. We illustrate the advantages of our method using a low-dimensional model that facilitates visualization, and this analysis motivates an adaptive sampling strategy that allows on-the-fly identification of and addition of data to regions important for predicting the statistics of interest. Finally, we demonstrate that we can compute accurate statistics for a 75-dimensional model of sudden stratospheric warming. This system provides a stringent test bed for our method.

Reduced order model predictive control for parametrized parabolic partial differential equations

Model Predictive Control (MPC) is a well-established approach to solve infinite horizon optimal control problems. Since optimization over an infinite time horizon is generally infeasible, MPC determines a suboptimal feedback control by repeatedly solving finite time optimal control problems. Although MPC has been successfully used in many applications, applying MPC to large-scale systems – arising, e.g., through discretization of partial differential equations – requires the solution of high-dimensional optimization problems and thus poses immense computational effort. We consider systems governed by parametrized parabolic partial differential equations and employ the reduced basis method as a low-dimensional surrogate model for the finite time optimal control problem. The reduced order optimal control serves as feedback control for the original large-scale system. We analyze the proposed RB-MPC approach by first developing a posteriori error bounds for the errors in the optimal control and associated cost functional. These bounds can be evaluated efficiently in an offline-online computational procedure and allow us to guarantee asymptotic stability of the closed-loop system using the RB-MPC approach in several practical scenarios. We also propose an adaptive strategy to choose the prediction horizon of the finite time optimal control problem. Numerical results are presented to illustrate the theoretical properties of our approach.