Original Full Model Research Articles

Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding

Current natural language understanding (NLU) models have been continuously scaling up, both in terms of model size and input context, introducing more hidden and input neurons. While this generally improves performance on average, the extra neurons do not yield a consistent improvement for all instances. This is because some hidden neurons are redundant, and the noise mixed in input neurons tends to distract the model. Previous work mainly focuses on extrinsically reducing low-utility neurons by additional post- or pre-processing, such as network pruning and context selection, to avoid this problem. Beyond that, can we make the model reduce redundant parameters and suppress input noise by intrinsically enhancing the utility of each neuron? If a model can efficiently utilize neurons, no matter which neurons are ablated (disabled), the ablated submodel should perform no better than the original full model. Based on such a comparison principle between models, we propose a cross-model comparative loss for a broad range of tasks. Comparative loss is essentially a ranking loss on top of the task-specific losses of the full and ablated models, with the expectation that the task-specific loss of the full model is minimal. We demonstrate the universal effectiveness of comparative loss through extensive experiments on 14 datasets from three distinct NLU tasks based on five widely used pre-trained language models and find it particularly superior for models with few parameters or long input.

Thermodynamics for reduced models of chemical reactions by PEA and QSSA

The partial equilibrium approximation (PEA) and the quasi-steady-state approximation (QSSA) are two classical methods for reducing complex macroscopic chemical reactions into simple computable ones. Previous studies have primarily focused on the accuracy of solutions before and after applying model reduction. In this paper, however, we adopt a thermodynamic view and aim to establish a quantitative connection on the essential thermodynamic quantities, such as entropy production rate, free-energy dissipation rate, and entropy flow rate, between the original chemical mass-action equations and the reduced models obtained by PEA or QSSA. Our findings reveal that the reduced models by PEA and QSSA may not preserve the thermodynamic structure of the original full model, particularly when algebraic relations are used instead of differential equations. This is evident in the loss of non-negativity of the free-energy dissipation rate. To validate our results, we apply them to the Michaelis-Menten reactions as a prototype, both analytically and numerically. We anticipate that our study will inspire a reevaluation of the effectiveness of various model reduction or approximation methods from the perspective of nonequilibrium thermodynamics. Published by the American Physical Society 2024

Optimizing androgen receptor prioritization using high-throughput assay-based activity models.

Introduction: Computational models using data from high-throughput screening assays have promise for prioritizing and screening chemicals for testing under the U.S. Environmental Protection Agency's Endocrine Disruptor Screening Program (EDSP). The purpose of this work was to demonstrate a data processing method for the determination of optimal minimal assay batteries from a larger comprehensive model, to provide a uniform method of evaluating the performance of future minimal assay batteries compared with the androgen receptor (AR) pathway model, and to incorporate chemical cluster analysis into this evaluation. Although several of the assays in the AR pathway model are no longer available through the original vendor, this approach could be used for future evaluations of minimal assay models for prioritization and screening. Methods: We compared two previously published models and found that an expanded 14-assay model had higher sensitivity for antagonists, whereas the original 11-assay model had slightly higher sensitivity for agonists. We then investigated subsets of assays in the original AR pathway model to optimize overall testing strategies that minimize cost while maintaining sensitivity across a broad chemical space. Results and Discussion: Evaluation of the critical assays across subset models derived from the 14-assay model identified three critical assays for predicting antagonism and two critical assays for predicting agonism. A minimum of nine assays is required for predicting agonism and antagonism with high sensitivity (95%). However, testing workflows guided by chemical structure-based clusters can reduce the average number of assays needed per chemical by basing the assays selected for testing on the likelihood of a chemical being an AR agonist, according to its structure. Our results show that a multi-stage testing workflow can provide 95% sensitivity while requiring only 48% of the resources required for running all assays from the original full models. The resources can be reduced further by incorporating in silico activity predictions. Conclusion: This work illustrates a data-driven approach that incorporates chemical clustering and simultaneous consideration of antagonism and agonism mechanisms to more efficiently screen chemicals. This case study provides a proof of concept for prioritization and screening strategies that can be utilized in future analyses to minimize the overall number of assays needed for predicting AR activity, which will maximize the number of chemicals that can be tested and allow data-driven prioritization of chemicals for further screening under the EDSP.

Dynamic modeling and parameter sensitivity analysis of AUV by using the POD method and the HB-AFT method

Autonomous Underwater Vehicle (AUV) parameterization is a geometric modification method, and can quickly obtain excellent structure with low vibration and noise performance. Blindness and inefficiency are common outcomes of the lack of an efficient parameter selection approach. But not enough research has been done in this area. In this paper, the finite element (FE) method is used to establish the propeller–shaft–shell double beam system with ball bearings, which is more similar to the real model. The basic theory of the proper orthogonal decomposition (POD) method is introduced in detail, and the original full order model (FOM) is reduced to 4 DOFs. It is found that the accuracy of the reduced order model (ROM) is high enough and the calculation time is greatly reduced. The semi-numerical semi-analytical expression of the ROM system is solved by the harmonic balance and alternating frequency/time domain (HB-AFT) method, whose results are in good agreement with those by the Runge–Kutta method. The sensitivities of the shell responses to the spring stiffness are discussed. The AUV setup is designed to analysis of numerical and experimental results. These results in this paper can provide an advanced theoretical basis for the structural parameter design of the AUV.

Towards Reduced Order Models via Robust Proper Orthogonal Decomposition to capture personalised aortic haemodynamics

Data driven, reduced order modelling has shown promise in tackling the challenges associated with computational and experimental haemodynamic models. In this work, we focus on the use of Reduced Order Models (ROMs) to reconstruct velocity fields in a patient-specific dissected aorta, with the objective being to compare the ROMs obtained from Robust Proper Orthogonal Decomposition (RPOD) to those obtained from the traditional Proper Orthogonal Decomposition (POD). POD and RPOD are applied to in vitro, haemodynamic data acquired by Particle Image Velocimetry and compare the decomposed flows to those derived from Computational Fluid Dynamics (CFD) data for the same geometry and flow conditions. In this work, PIV and CFD results act as surrogates for clinical haemodynamic data e.g. MR, helping to demonstrate the potential use of ROMS in real clinical scenarios. The flow is reconstructed using different numbers of POD modes and the flow features obtained throughout the cardiac cycle are compared to the original Full Order Models (FOMs).Robust Principal Component Analysis (RPCA), the first step of RPOD, has been found to enhance the quality of PIV data, allowing POD to capture most of the kinetic energy of the flow in just two modes similar to the numerical data that are free from measurement noise. The reconstruction errors differ along the cardiac cycle with diastolic flows requiring more modes for accurate reconstruction. In general, modes 1–10 are found sufficient to represent the flow field. The results demonstrate that the coherent structures that characterise this aortic dissection flow are described by the first few POD modes suggesting that it is possible to represent the macroscale behaviour of aortic flow in a low-dimensional space; thus significantly simplifying the problem, and allowing for more computationally efficient flow simulations or machine learning based flow predictions that can pave the way for translation of such models to the clinic.

Evaluating efficient SENSE algorithms to deblur spiral MRI with fat/water separation.

The combination of SENSE and spiral imaging with fat/water separation enables high temporal efficiency. However, the corresponding computation increases due to the blurring/deblurring operation across the multi-channel data. This study presents two alternative models to simplify computational complexity in the original full model (model 1). The performances of the models are evaluated in terms of the computation time and reconstruction error. Two approximated spiral MRI reconstruction models were proposed: the comprehensive blurring before coil operation (model 2) and the regional blurring before coil operation (model 3), respectively, by altering the order of coil-sensitivity encoding process to distribute signals among the multi-channel coils. Four subjects were recruited for scanning both fully sampled T1 - and T2 -weighted brain image data with simulated undersampling for testing the computational efficiency and accuracy on the approximation models. Based on the examples, the computation time can be reduced to 31%-47% using model 2, and to 39%-56% using model 3.The quality of the water image remains unchanged among the three models, whereas the primary difference in image quality is in the fat channel. The fat images from model 3 are consistent with those from model 1, but those from model 2 have higher normalized error, differing by up to 4.8%. Model 2 provides the fastest computation but exhibits higher error in the fat channel, particularly in the high field and with long acquisition window. Model 3, an abridged alternative, is also faster than the full model and can maintain high accuracy in reconstruction.

Enhancing deep neural networks for multivariate uncertainty analysis of cracked structures by POD-RBF

An efficient Monte Carlo (MC) simulation method is proposed to address multivariate uncertainties in the dynamic fracture analysis of cracked structures. Deep neural networks (DNN) based on model order reduction are special surrogate models for enhancing the sampling efficiency of MC simulation. Proper orthogonal decomposition-radial basis functions (POD-RBF) bridge the original full model and the DNN, enabling the training datasets of the neural network to be evaluated rapidly from a reduced-order model. The full-order model snapshot of the cracked structure is obtained using the scaled boundary finite element method (SBFEM). We innovatively tackle multidimensional uncertainties, including individual crack lengths, material parameters, and their combinations. Numerical examples show that the POD-RBF-DNN-MC surrogate model benefits from the uncertainty analysis of large scale and multidimensional random input variables. The method is simpler than full-order sampling and eliminates the scale problem of POD-RBF. In addition, the POD-RBF-DNN-MC surrogate model is more efficient than full-order MC, POD-RBF-MC only, and DNN-MC only, with respect to accelerating the stochastic response analysis based on MC. Finally, we validate the proposed algorithm and show that it significantly improves the efficiency of uncertainty analysis.

A sample-efficient deep learning method for multivariate uncertainty qualification of acoustic–vibration interaction problems

We propose an efficient Monte Carlo simulation method to address the multivariate uncertainties in acoustic–vibration interaction systems. The deep neural network acts as a general surrogate model to enhance the sampling efficiency of Monte Carlo Simulation. Singular Value Decomposition - Radial Basis Functions (SVD-RBF) acts as a bridge between the original full model and the neural network, enabling the training datasets of the neural network to be evaluated rapidly from a reduced-order model. The snapshots of full order models are obtained with isogeometric analysis, in which we couple two numerical schemes for vibro–acoustic interaction problems: the isogeometric finite element method for simulating vibration of Kirchhoff–Love shells and isogeometric boundary element method for exterior acoustic waves. Numerical results show that the proposed algorithm can significantly improve the efficiency of uncertainty analysis.

A finite-volume moving-mesh method for two-phase flow in dynamically fracturing porous media

Multiphase flow in fractured porous media can be described by discrete fracture matrix models that represent the fractures as dimensionally reduced manifolds embedded in the bulk porous medium. Generalizing earlier work on this approach we focus on immiscible two-phase flow in time-dependent fracture geometries, i.e., the fracture itself and the aperture of the fractures might evolve in time. For dynamic fracture geometries of that kind, neglecting capillary forces, we deduce by transversal averaging of a full dimensional description a dimensionally reduced model that governs the geometric evolution and the flow dynamics. The core computational contribution is a mixed-dimensional finite-volume discretization based on a conforming moving-mesh ansatz. This finite-volume moving-mesh (FVMM) algorithm is tracking the fractures' motions as a family of unions of facets of the mesh. Notably, the method permits arbitrary movement of facets of the triangulation while keeping the mass conservation constraint. In a series of numerical examples we investigate the modeling error of the reduced model as it compares to the original full dimensional model. Moreover, we show the performance of the finite-volume moving-mesh algorithm for the complex wave pattern that is induced by the interaction of saturation fronts and evolving fractures.

Nonlinear dynamics for the 3D ideal viscous gas flow over the cylinder

The system of governing equations for the dynamics of the compressible viscous ideal gas is considered in the 3D bounded domain with the inflow and outflow boundary conditions. The cylinder is located in the domain. Such problem is simulated using the high order WENO-scheme for inviscid part of the equations and using 4-th order central approximation for the viscous tensor part with the third order temporal discretization.The method of Proper Orthogonal Decomposition (POD) is applied to the problem at hand in order to extract the most active nodes. Cascades of bifurcations of periodic orbits and invariant tori are found that correspond to the excitation in different POD modes. The approximation of the reduced order model is analyzed and it is shown that one cannot make parameter extrapolations for the reduced order model to capture the same dynamics as is observed in the original full size model.

Laser Powder Bed Fusion of a Topology Optimized and Surface Textured Rudder Bulb with Lightweight and Drag-Reducing Design

This work demonstrates the advantages of using laser powder bed fusion for producing a rudder bulb of a moth class sailing racing boat via laser powder bed fusion (LPBF). The component was designed to reduce weight using an AlSi7Mg0.6 alloy and incorporated a biomimetic surface texture for drag reduction. For the topological optimization, the component was loaded structurally due to foil wing’s lift action as well as from the environment due to hydrodynamic resistance. The aim was to minimize core mass while preserving stiffness and the second to benefit from drag reduction capability in terms of passive surface behavior. The external surface texture is inspired by scales of the European sea bass. Both these features were embedded to the component and produced by LPBF in a single run, with the required resolution. Drag reduction was estimated in the order of 1% for free stream velocity of 2.5 m s−1. The production of the final part resulted in limited geometrical error with respect to scales 3D model, with the desired mechanical properties. A reduction in weight of approximately 58% with respect to original full solid model from 452 to 190 g was achieved thanks to core topology optimization. Sandblasting was adopted as finishing technique since it was able to improve surface quality while preserving fish scale geometries. The feasibility of producing the biomimetic surfaces and the weight reduction were validated with the produced full-sized component.

Conductance heterogeneities induced by multistability in the dynamics of coupled cardiac gap junctions

In this paper, we study the propagation of the cardiac action potential in a one-dimensional fiber, where cells are electrically coupled through gap junctions (GJs). We consider gap junctional gate dynamics that depend on the intercellular potential. We find that different GJs in the tissue can end up in two different states: a low conducting state and a high conducting state. We first present evidence of the dynamical multistability that occurs by setting specific parameters of the GJ dynamics. Subsequently, we explain how the multistability is a direct consequence of the GJ stability problem by reducing the dynamical system’s dimensions. The conductance dispersion usually occurs on a large time scale, i.e., thousands of heartbeats. The full cardiac model simulations are computationally demanding, and we derive a simplified model that allows for a reduction in the computational cost of four orders of magnitude. This simplified model reproduces nearly quantitatively the results provided by the original full model. We explain the discrepancies between the two models due to the simplified model’s lack of spatial correlations. This simplified model provides a valuable tool to explore cardiac dynamics over very long time scales. That is highly relevant in studying diseases that develop on a large time scale compared to the basic heartbeat. As in the brain, plasticity and tissue remodeling are crucial parameters in determining the action potential wave propagation’s stability.

Efficient hybrid multiobjective optimization of pressure swing adsorption

Pressure swing adsorption (PSA) is an energy-efficient technology for gas separation, while the multiobjective optimization of PSA is a challenging task. To tackle this, we propose a hybrid optimization framework (TSEMO + DyOS), which integrates two steps. In the first step, a Bayesian stochastic multiobjective optimization algorithm (i.e., TSEMO) searches the entire decision space and identifies an approximated Pareto front within a small number of simulations. Within TSEMO, Gaussian process (GP) surrogate models are trained to approximate the original full process models. In the second step, a gradient-based deterministic algorithm (i.e., DyOS) is initialized at the approximated Pareto front to further refine the solutions until local optimality. Therein, the full process model is used in the optimization. The proposed hybrid framework is efficient, because it benefits from the coarse-to-fine function evaluations and stochastic-to-deterministic searching strategy. When the result is far away from the optima, TSEMO can efficiently approximate a trade-off curve as good as a commonly used evolutional algorithm, i.e., Nondominated Sorting Genetic Algorithm II (NSGA-II), while TSEMO only uses around 1/16th of CPU time of NSGA-II. This is because the GP-based surrogate model is utilized for function evaluations in the initial coarse search. When the result is near the optima, the searching efficiency of TSEMO dramatically decreases, while DyOS can accelerate the searching efficiency by over 10 times. This is because, in the proximity of optima, the exploitation capacity of DyOS is significantly higher than that of TSEMO.

Development and External Validation of Prognostic Models for COVID-19 to Support Risk Stratification in Secondary Care

Background: Existing UK prognostic models for patients admitted to hospital with COVID-19 are limited by reliance on comorbidities, which are under-recorded in secondary care, and lack of imaging data among the candidate predictors. Our aims were to develop and externally validate novel prognostic models for adverse outcomes (death, intensive therapy unit (ITU) admission) in UK secondary care; and externally validate the existing 4C score. Methods: Patients with COVID-19 admitted to University Hospitals Birmingham (UHB) January-August 2020 were included. Candidate predictors included demographic variables, symptoms, physiological measures, imaging, laboratory tests. Final models used logistic regression with stepwise selection. External validation was performed in the CovidCollab dataset. Findings: 1040 patients with COVID-19 were included in the derivation cohort; 288 (28%) died and 183 (18%) were admitted to ITU within 28 days of admission. Area under the receiver operating curve (AUROC) for mortality was 0.791 (95%CI 0.761-0.822) in UHB and 0.767 (95%CI 0.754-0.780) in CovidCollab; AUROC for ITU admission was 0.906 (95%CI 0.883-0.929) in UHB and 0.811 (95%CI 0.795-0.828) in CovidCollab. Models showed good calibration. Addition of comorbidities to candidate predictors did not improve model performance. AUROC for the 4C score in the UHB dataset was 0.754 (95%CI 0.721-0.786). Interpretation: The novel prognostic models showed good discrimination and calibration in derivation and external validation datasets, and outperformed the existing 4C score. The models can be integrated into electronic medical records systems to calculate each individual patient’s probability of death or ITU admission at the time of hospital admission. Implementation of the models and clinical utility should be evaluated. Funding: Medical Research Council UK Research and Innovation.Declaration of Interests: NJA, ES, KN, MP, AD, CS, TT and YT report a grant from UKRI MRC during the conduct of the study. ES reports grants from National Institute for Health Research (NIHR), Wellcome Trust, MRC, Health Data Research UK (HDR-UK), British Lung Foundation, and Alpha 1 Foundation outside the submitted work. KN reports grants from MRC and HDR-UK outside the submitted work. DP reports grants from NIHR, MRC, and Chernakovsky Foundation outside the submitted work. All other authors have nothing to declare.Ethics Approval Statement: Ethical approval was provided by the East Midlands – Derby REC (reference: 20/EM/0158) for the PIONEER Research Database.

POD-DEIM model order reduction technique for model predictive control in continuous chemical processing

In this study, a model order reduction (MOR) technique is proposed to address the challenges of controlling large-scale problems for model predictive control (MPC) development in continuous chemical processing to meet the real-time control requirements. In particular, the proper orthogonal decomposition (POD) technique is employed to project the original large-scale full chemical process model onto a small system of a reduced model space, while the discrete empirical interpolation method is used to evaluate the nonlinear functions at a small set of the interpolation points. By using MOR, the original full chemical process model has been further represented by a much smaller number of state variables (about 2 to 3 orders of magnitude smaller in dimension). Thus, instead of solving the original full model, the MOR method solves the reduced sub-set model iteratively in the control process. In such a way, the MOR solution enables a much faster computational time and opens opportunities for various real-time control applications. In addition, an optimal snapshot selection algorithm is implemented to obtain the global basis vectors, which cover enough information of the model parameter(s) and input(s) in large control window(s) for an accurate MOR construction. For control demonstration, the in-silico control scenarios are performed for both applications (which are multiple scale chemical reactions and multiple pathway reactions), while onsite control is only performed for multiple pathway chemical reactions. Particularly, for soft-launch control demonstrations, the implemented framework is applied for multiple input (MI)/ multiple output (MO) and with the input disturbance and output noise control scenarios, while the onsite control demonstration is applied for SI/SO with input disturbance and MI/SO control scenarios. The obtained results show that the use of the developed MOR-MPC approach can significantly reduce the computational time of about two-orders of magnitude with the relative error of 1.0 × 10−3 compared to original full model. It implies that the MOR can be applied to the real-time control of certain applications, where it is impossible for the full original model.

Parameter Preserving Model Order Reduction of Large Sparse Small-Signal Electromechanical Stability Power System Models

This paper proposes a method for the model order reduction (MOR) of large scale power system models that produces reduced order models (ROM) which preserve the access to selected parameters exactly like the original full order model (FOM). The preserved parameters are related to decentralized power system devices, including, but not limited to, power system stabilizers that are used for the damping control of system electromechanical oscillations. The problem formulation of this parametric MOR (PMOR) is described, as well as the implementation details regarding efficiency and flexibility. Test results are described for large practical power system models used in small-signal electromechanical stability studies. The ROM produced sufficiently accurate responses following parameters changes, when equivalently compared to the original FOM results. The proposed method allows preserving specified nonlinearities in chosen devices of the parameterized ROMs. It can also be applied to the reduction of differential algebraic equation models related to other areas of engineering.

A full orthotropic micropolar peridynamic formulation for linearly elastic solids

An original full orthotropic model for in-plane linear elasticity is proposed in the micropolar peridynamic analysis framework. The analytical formulation is derived from the definition of a specific microelastic energy function for micropolar nonlocal lattices which allows to obtain, for the first time, an orthotropic bond-based model characterized by four independent elastic moduli. An important feature of the model is that the bond properties, i.e. the elastic constants, are continuous functions of the bond orientation in the principal material axes. The introduction of the bond shear stiffness and the definition of a bond shear deformation measure which accounts for particle’s rotation, on one hand eliminates the restriction of two independent constants that affects other bond-based orthotropic peridynamic formulations, and on the other makes the model suitable in predicting the mechanical behavior of a wide variety of Cauchy orthotropic materials undergoing homogeneous and non-homogeneous deformations. The accuracy of the proposed model in linear elasticity has been verified through simulating uniaxial extension test of a composite lamina with a central circular hole and natural frequency analyses considering different orientations of the principal material reference system.

Prediction of the grain size evolution during thermal treatments at the mesoscopic scale: a numerical framework and industrial examples

Recently, an original full field model working at the mesoscopic scale using the level set (LS) method in a finite element (FE) framework has been introduced. This approach has demonstrated its potential for the simulation of grain growth and recrystallization problems. Through the development of the DIGIMU® software, this methodology is now considered for industrial applications. The paper presents (i) the recent developments made on the LS approach and (ii) some examples of large scale simulations in two and three dimensions considering thermal treatments applied on materials. Grain boundaries motion considering the presence or not of second phase particles (like precipitates) are investigated.

A very low speech model based on frequency selection-GA approach

Using a new model order reduction based on frequency selection and optimal genetic algorithm, a second order speech model is calculated. In our algorithm, the modeling process starts with a full order classical all poles model obtained by Burg method. The full order model AR is reduced using the proposed approach based on the genetic algorithms and the original speech production system dominant frequencies. The model reduction yields to a ARMA second order model which interestingly preserves the key properties of the original full order model in the time and frequency domains. To illustrate the performance and the effectiveness of the proposed approach, some computer simulations are conducted on some practical speech segments. To show the novelty of our approach, a comparative study with an approximant given by the robust SVD-Schur technique is presented.

Improving firefly algorithm using hybrid strategies

In this paper, we present a new firefly algorithm (FA) by using hybrid strategies to obtain a good optimisation performance. The proposed approach, namely HFA, employs three strategies. First, an adaptive parameter method is utilised to dynamically changing the step factor. Second, HFA uses a modified search strategy and eliminates the concept of attractiveness. So, HFA does not include two parameters, absorption coefficient and initial attractiveness. Third, a probabilistic attraction model is used to replace the original full attraction model. Experiments on some benchmark problems show that HFA is superior to mimetic FA (MFA) and probabilistic attraction-based FA (PAFA).