Model Reduction Techniques Research Articles

TEC-CNN: Towards Efficient Compressing Convolutional Neural Nets with Low-rank Tensor Decomposition

Most state-of-the-art convolutional neural networks (CNNs) are characterised by excessive parameterisation, leading to a high computational burden. Tensor decomposition has emerged as a model reduction technique for compressing deep neural networks. Previous approaches have predominantly relied on either Tucker decomposition or Canonical Polyadic (CP) decomposition for CNNs. However, CP decomposition exhibits exceptional compression capabilities in comparison to Tucker decomposition, which results in a more pronounced accuracy loss. This paper introduces an efficient model compression method, termed TEC-CNN, designed to achieve significant compression while preserving accuracy levels comparable to those of the original models. In TEC-CNN, convolutional layers are identified to obtain convolutional kernels by analysing given models under the principles of low-rank tensor decomposition, and then, calculating the ranks of convolutional kernels. Furthermore, an efficient decomposition schema for the convolutional kernel is proposed with approximate kernel tensor for reducing parameters. Additionally, a novel format of a convolutional sequence is presented and constructed with a reduced number of parameters to replace the original convolutional layers. Finally, the effectiveness of TEC-CNN is assessed across a range of computer vision tasks. For instance, in CIFAR-100 classification, ResNet18 is compressed to 4.1 MB, while Unext, when applied to image segmentation using the International Skin Imaging Collaboration (ISIC) dataset, is reduced to 3.419 MB. When employed for fire object detection with Yolov7, TEC-CNN achieves a model size reduction of 71.6 MB. Comprehensive experimental results underscore that our approach achieves significant model compression while preserving model performance.

Phenomenological constitutive laws for the dissipative behaviour of highly compressible elastomers and their finite element implementation

This paper deals with the development of constitutive equations to model the mechanical behaviour of compressible elastomers. These materials are naturally incompressible but can be made compressible by the addition of hollow microspheres, for example. Such a material is referred to as syntactic foam. The CEA (French Commission for Atomic and Alternative Energies) employs such compressible materials as seals in complex structures to reduce the internal stresses in vulnerable components and prevent their failure. The behaviour of these structures is predicted by finite element simulations. It is important to know and model the mechanical behaviour of the seals. Like elastomers, they can undergo large deformations. The microspheres enable the material to undergo large volume change unlike pure elastomer that is nearly incompressible. This compressibility also intensifies dissipative phenomena encountered in elastomers such as viscosity or plasticity. Furthermore, the Mullins stress softening effect is also intensified even for loadings that only bring about volumetric changes. To model these behaviours, a phenomenological approach was developed based on the isochoric/volumetric decomposition of the deformation gradient. The method of intermediate dissipative configurations was employed to introduce multiple phenomena, including viscosity (with several characteristic times) and viscoplasticity, for these two parts of the deformation. The constitutive equations and their flow rules were implemented in Abaqus through a UMAT subroutine and using a numerical approach to define the tangent operator. The parameters of the behaviour law were identified using a model reduction technique known as shape manifold approach. The resulting model can be compared with experimental data.

Model order reduction and sensitivity analysis for complex heat transfer simulations inside the human eyeball.

Heat transfer in the human eyeball, a complex organ, is significantly influenced by various pathophysiological and external parameters. Particularly, heat transfer critically affects fluid behavior within the eye and ocular drug delivery processes. Overcoming the challenges of experimental analysis, this study introduces a comprehensive three-dimensional mathematical and computational model to simulate the heat transfer in a realistic geometry. Our work includes an extensive sensitivity analysis to address uncertainties and delineate the impact of different variables on heat distribution in ocular tissues. To manage the model's complexity, we employed a very fast model reduction technique with certified sharp error bounds, ensuring computational efficiency without compromising accuracy. Our results demonstrate remarkable consistency with experimental observations and align closely with existing numerical findings in the literature. Crucially, our findings underscore the significant role of blood flow and environmental conditions, particularly in the eye's internal tissues. Clinically, this model offers a promising tool for examining the temperature-related effects of various therapeutic interventions on the eye. Such insights are invaluable for optimizing treatment strategies in ophthalmology.

Automated model order reduction for building thermal load prediction using smart thermostats data

This paper presents a methodology to automatically determine the structure of sufficiently accurate grey-box models for model predictive control, energy efficiency and flexibility applications in buildings. The methodology is based on model reduction and system identification techniques, with a path that enhances data pre-processing, a multistage order reduction, and parameter estimation. The model structure is determined with a cascade approach that either neglects, keeps, or aggregates thermal zones by using discrete and continuous frequency domain techniques. Once the optimal structure is identified, the parameters are calibrated with the measured data from smart thermostats, using the model predictive control relevant identification method. The methodology is applied to a monitored house located in Québec, Canada. The developed algorithm identifies adjacent zones, even when the building layout is unknown, by studying indoor temperature fluctuations. The results concerning the model creation suggest that, for this specific building, the aggregation by floor is the most efficient way for creating reduced order thermal models, limiting uncertainty due to thermal zone interaction. This methodology provides control-oriented models that accurately predict response up to 24-h ahead with Root Mean Square Error less than 0.5 °C and acceptable Fitness Function values for the minimum number of selected parameters. Finally, several scenarios demonstrate the insights gained from using grey-box building thermal models for design, control, and retrofitting applications.

Numerical and computational analysis on a dissipative dynamical system: Slow invariant manifold for complex chemical mechanism

This article presents the notion about Slow Invariant Manifold (SIM) and their fundamental role in model reduction techniques (MRTs) for challenges encountered in mechanical engineering within dissipative systems of chemical kinetics. Focusing on the reaction routes of complex mechanisms, we construct and compare primary approximations of the SIM through MRTs, including the Spectral Quasi Equilibrium Manifold (SQEM) and Intrinsic Low Dimensional Manifold (ILDM). These methods effectively transform high-dimensional complex problems into lower dimensions, solving them without compromising crucial information about the complex systems modified for homogeneous reactive systems. Employing the sensitivity analysis by using the MATLAB's toolbox, we present the numerical findings in a tabular format obtained through MRTs. This study provides the understanding about the accessible exploration of numerical solutions, improving insights of the complex variation within the system.

Model order reduction by convex displacement interpolation

We present a nonlinear interpolation technique for parametric fields that exploits optimal transportation of coherent structures of the solution to achieve accurate performance. The approach generalizes the nonlinear interpolation procedure introduced in [Iollo, Taddei, J. Comput. Phys., 2022] to multi-dimensional parameter domains and to datasets of several snapshots. Given a library of high-fidelity simulations, we rely on a scalar testing function and on a point set registration method to identify coherent structures of the solution field in the form of sorted point clouds. Given a new parameter value, we exploit a regression method to predict the new point cloud; then, we resort to a boundary-aware registration technique to define bijective mappings that deform the new point cloud into the point clouds of the neighboring elements of the dataset, while preserving the boundary of the domain; finally, we define the estimate as a weighted combination of modes obtained by composing the neighboring snapshots with the previously-built mappings. We present several numerical examples for compressible and incompressible, viscous and inviscid flows to demonstrate the accuracy of the method. Furthermore, we employ the nonlinear interpolation procedure to augment the dataset of simulations for linear-subspace projection-based model reduction: our data augmentation procedure is designed to reduce offline costs — which are dominated by snapshot generation — of model reduction techniques for nonlinear advection-dominated problems.

Damage detection of truss structures using meta-heuristic algorithms and optimized group method of data handling surrogate model

Optimization-based methods are increasingly being implemented for structural damage detection (SDD) problems through the minimization of the objective functions based on vibration data. Meanwhile, the challenge of high computational cost hinders the applied use of SDD methods, especially upon addressing large-scale structures. In this study, a two-stage damage detection approach is proposed for damage localization as well as extent quantification which reduces the computational time of SDD problems. In the first stage of the proposed framework, suspected locations of damage are identified by employing a residual force vector. Then in the second stage, damage extent of already identified elements will be assessed by model updating method through three different optimization algorithms, namely particle swarm optimization (PSO), differential evolution (DE), and enhanced colliding bodies optimization (ECBO). To spare time-consuming modal analysis in each iteration of optimization algorithms, a PSO-based group method of data handling (GMDH) polynomial neural network (PNN) surrogate model is proposed. Performance of GMDH PNN strongly depends on the input variables, the number of input variables, and the order of the polynomial. The PSO-based GMDH PNN, developed in this study, employs PSO for the optimum selection of these parameters. Furthermore, the efficiency of Elemental Energy Quotient Indicator (EEQI) as a new damage index proposed in this paper is compared with the approach based on the natural frequency-based index. This paper also deals with the incomplete modal data measured from limited number of sensors. This problem is addressed by employing the model reduction technique and solving the SDD problem with modal information obtained from the reduced model. Numerical examples show that the suggested PSO-based GMDH surrogate model is an accurate tool for various types of SDD problems, and it can provide promising results with a significant reduction in computational time.

Hermite Expansion Technique for Model Reduction of Circuit Systems with Delay Components

Hermite Expansion Technique for Model Reduction of Circuit Systems with Delay Components

Topology optimization method for transient heat conduction using the Lyapunov equation

This study presents a novel topology optimization approach for transient heat conduction during the transition period based on the Lyapunov equation. For the problem under consideration, traditional topology optimization methods face computational challenges due to the time dependence of the transient responses. To address this issue, this article introduces a new optimization formulation that contains a modified governing equation for transient heat conduction and a new objective function. The new formulation aims to simplify the evaluation of transient response performance by utilizing the Lyapunov equation, thereby avoiding time-consuming transient analysis during the optimization iteration process. Additionally, this study employs model reduction techniques to reduce the size of the analysis model and further enhance the efficiency of the optimization process. After comparing two classic methods, the Proper Orthogonal Decomposition (POD) method is selected for its higher efficiency. Numerical examples consider both heat conduction and temperature control problems, employing 2D and 3D models to validate the proposed method's effectiveness. The results of the point-to-point heat conduction example indicate that, compared to the conventional design, the optimized topology design obtained using the proposed method successfully reduce the heating time by approximately 35%. Furthermore, the optimized designs display clear distinguishing features from conventional designs, further emphasizing the significance of implementing transient heat conduction topology optimization.

Model simplification and sensitivity analysis of chemical species for oxidation behavior in multi route complex chemical mechanism

This study employs a Model Reduction Technique (MRT) to simplify the four-step catalytic carbon monoxide (CO) oxidation reaction. The C-matrix method identifies key elements, key/non key components, and key reactions, while the Intrinsic Low-Dimensional Manifold (ILDM) pinpoints a Slow-Invariant Manifold (SIM) important for understanding key species behavior. Sensitivity analysis can be considered for measuring the efficiency of the chemical species in detailed mechanism. This systematic approach contributes to optimizing and controlling complex reactions offering broad application potential. In addition to the mathematical proof, the validation of the given chemical model is rectified. The comparison between the slow invariant manifold of both reaction routes is reported and the computational based results performed in this study are obtained through MATLAB.

Comparative analysis of model reduction techniques for flapping wing dynamics

Comparative analysis of model reduction techniques for flapping wing dynamics

Model reduction to spectral submanifolds in piecewise smooth dynamical systems

We develop a model reduction technique for piecewise smooth dynamical systems using spectral submanifolds. Specifically, we construct low-dimensional, sparse, nonlinear and piecewise smooth models on unions of slow and attracting spectral submanifolds (SSMs) for each smooth subregion of the phase space and then properly match them. We applied this methodology to both equation-driven and data-driven problems, with and without external forcing.

A Novel Mayfly Algorithm with Response Surface for Static Damage Identification Based on Multiple Indicators

This paper proposes a novel structural damage identification approach coupling the Mayfly algorithm (MA) with static displacement-based response surface (RS). Firstly, a hybrid optimal objective function is established that simultaneously considers the sensitivity-based residual errors of static damage identification equation and the static displacement residual. In the objective function, the static damage identification equation is addressed by the Tikhonov regularization technique. The MA is subsequently employed to conduct an optimal search and pinpoint the location and intensity of damages at the structural element level. To handle the inconformity of the static loading points and the measurement points of displacements, the model reduction and displacement extension techniques are implemented to reconstruct the static damage identification equation. Meanwhile, the static displacement-based RS is constructed to calculate the displacement residual in the hybrid objective function, thereby circumventing the time-consuming finite element calculations and improving computational efficiency. The identification results of the numerical box girder bridge demonstrate that the proposed method outperforms the particle swarm optimization, differential evolution, Jaya and whale optimization algorithms about both convergence rate in optimal searching and identification accuracy. The proposed method enables more accurate damage identification compared to methods solely based on the indicator of the residual of static damage identification equations or displacement residual. The results of identifying damage in the 21 element-truss structure and the static experiments on identifying damage in an aluminum alloy cantilever beam confirm the high efficiency of the proposed approach.

Finite element model updating and damage detection using strain-based modal data in the Bayesian framework

This paper proposes a novel methodology for Bayesian model updating and damage detection of a structural system using strain-based modal data. The strain-based modal data are assumed to consist of the posterior statistics of natural frequencies and strain mode shapes corresponding to measured strains from a classically damped system. The dynamic strain-based equation of the motion is formulated utilizing the energy functional of the structural system and the strain displacement relation. The state variables in the considered strain-based formulation consist of the strains and the elemental rigid body displacements. The methodology is developed by incorporating strain-based formulation into the Bayesian framework for Finite Element (FE) model updating. Besides, additional uncertain parameters in the form of system strain-based modal parameters are introduced in the eigenvalue equation to avoid mode-matching. The model reduction technique is utilized to eliminate the rigid body displacements and the strains that are not measured. Metropolis-Hastings (MH) within Gibbs sampling is proposed to simulate samples from the posterior Probability Density Function (PDF) of the model parameters of the FE model. The effectiveness of the proposed methodology is demonstrated using simulated examples. An experimental study is also conducted to validate the proposed methodology.

Hybrid vibro-acoustic model reduction for model updating in nuclear power plant pipeline with undetermined boundary conditions

In this work, the hybrid vibro-acoustic model reduction technique that is a physical-modal combined formulation is proposed to accelerate the finite element model updating process of the vibro-acoustic pipeline system. Particularly, the new formulation could provide an effective way of the model updating by preserving the physical DOFs for the direct calibration of the undetermined boundary conditions. The sensitivity based vibro-acoustic model updating is first conducted, and then the undetermined spring constant at the displacement boundary condition is then directly and effectively calibrated by using the proposed hybrid model reduction formulation. The proposed method is implemented in the real nuclear facility to evaluate its performance. In addition, an experimental implementation test using the inverse force identification process is also conducted to demonstrate the reliability of the generated vibro-acoustic FE model through the proposed method.

Inference of complex reaction mechanisms applying model reduction techniques

Both structural (number of species and reactions) and temporal (extremely diverse reaction rates) aspects of complexity are considered when describing large chemical reaction networks. A consistent way to make model reduction is to construct the invariant manifold, which describes the asymptotic system behavior. Preliminary approximations to SIM (Slow Invariant Manifold) are constructed using the model reduction techniques (MRTs): the Quasi-Equilibrium Manifold (QEM), the Spectral Quasi-Equilibrium Manifold (SQEM), and the Intrinsic Low-Dimension Manifold (ILDM). In this paper, the activities of the concerned species and the overall dynamics of the system are examined. Two examples are used to demonstrate the techniques: the Michaelis–Menten mechanism, which is a single reaction mechanism, and a multi-route route reaction mechanism. The behavior of each species on the available route is covered separately. As a result, the reduced invariant solution curve of several approaches is illustrated, along with a comparison of these methods in various graphs. Sensitivity analysis is applied using the SimBiology toolbox in MATLAB to monitor the role of each parameter involved. All the results of model reduction techniques are simulated through MATLAB.

Enhanced Tracking in Legged Robots through Model Reduction and Hybrid Control Techniques: Addressing Disturbances, Delays, and Saturation

This study introduces a reduced-order leg dynamic model to simplify the controller design and enhance robustness. The proposed multi-loop control scheme tackles tracking control issues in legged robots, including joint angle and contact-force regulation, disturbance suppression, measurement delay, and motor saturation avoidance. Firstly, model predictive control (MPC) and sliding mode control (SMC) schemes are developed using a simplified model, and their stability is analyzed using the Lyapunov method. Numerical simulations under two disturbances validate the superior tracking performance of the SMC scheme. Secondly, an Nth-order linear active disturbance rejection control (LADRC) is designed based on a simplified model and optimization problems. The second-order LADRC-SMC scheme reduces the contact-force control error in the SMC scheme by ten times. Finally, a fourth-order LADRC-SMC with a Smith Predictor (LADRC-SMC-SP) scheme is formulated, employing each loop controller independently. This scheme simplifies the design and enhances performance. Compared to numerical simulations of the above and existing schemes, the LADRC-SMC-SP scheme eliminates delay oscillations, shortens convergence time, and demonstrates fast force-position tracking responses, minimal overshoot, and strong disturbance rejection. The peak contact-force error in the LADRC-SMC-SP scheme was ten times smaller than that in the LADRC-SMC scheme. The integral square error (ISE) values for the tracking errors of joint angles θ1 and θ2, and contact force f, are 1.6636×10−28 rad2⋅s, 1.7983×10−28 rad2⋅s, and 1.8062×10−30 N2⋅s, respectively. These significant improvements in control performance address the challenges in single-leg dynamic systems, effectively handling disturbances, delays, and motor saturation.

Robustness improvement of optimal control in terms of RBFNN with empirical model reduction and transfer learning

This paper proposes a method to compute solutions of optimal controls for dynamic systems in terms of radial basis function neural networks (RBFNN) with Gaussian neurons. The RBFNN is used to compute the value function from Hamilton–Jacobi–Bellman equation with the policy iteration. The concept of dominant system is introduced to create initial coefficients of the neural networks to stabilise unstable systems and guarantee the convergence of policy iteration. Model reduction and transfer learning techniques are introduced to improve robustness of RBFNN optimal control, and reduce computational time. Numerical and experimental results show that the resulting optimal control has excellent control performance in stabilisation and trajectory tracking, and much-improved robustness to disturbances and model uncertainties even when the system responses move to a domain of the state space that is larger than the domain where the neural networks are trained.

Electrochemical aging model of lithium-ion battery with impedance output and its parameter sensitivity analysis and identification

The electrochemical model of the battery has gained widespread recognition due to its clear physical interpretability. However, achieving an accurate electrochemical model heavily relies on precise parameter settings. To address this challenge, various model-based parameter optimization methods have been developed. Currently, most research relies on model reduction techniques and heuristic random search methods for parameter identification. These approaches often lead to time-consuming processes and a loss of physical interpretability in the electrochemical model. To overcome these limitations, we propose a novel technical framework that directly parametrizes the electrochemical model. First, we establish a battery electrochemical model that accounts for lithium plating and SEI film thickening, serving as the main focus of this study. Secondly, in order to expedite parameter identification without compromising the model's physical interpretability, we innovate by performing comprehensive sensitivity analysis using voltage and impedance characteristics. This allows us to identify the most influential parameters, which become the key for subsequent research. Finally, to ensure reasonable initialization of finite element computations and to avoid convergence issues in the identification algorithm, we combine machine learning and optimization algorithms into a two-step parameter identification method. By utilizing the proposed method to identify the model parameters, the simulations exhibit high accuracy.

Novel reduction schemes for a dissipative dynamical system: A study on slow invariant manifolds in chemical kinetics

This research explores the intricate concept of the Slow Invariant Manifold (SIM) and its pivotal role in developing model reduction techniques (MRTs) for challenges within dissipative systems in chemical kinetics, specifically in mechanical engineering. Focusing on the multi-step mechanism with two intermediates, primary approximations of the SIM are constructed and compared using two prominent MRTs: The Spectral Quasi Equilibrium Manifold (SQEM) and Intrinsic Low Dimensional Manifold (ILDM). At the given rate coefficient, a special computational experiment was performed in which the efficiency of chemical species has been compared. Noteworthy innovation involves evaluating SIM separately for reduced species, departing from the conventional approach of considering every species within the mechanism. The study employs local sensitivity analysis with MATLAB's Sim-Biology toolbox, presenting quantitative findings in a tabular format for a comprehensive MRT comparison. Beyond contributing to a deeper understanding of model reduction techniques in complex chemical kinetics, this research marks the first computational exploration of dissipative systems. The novel perspective of evaluating SIM for reduced species offers nuanced insights, emphasizing the critical role of SIM in effectively addressing challenges in mechanical engineering applications. In summary, this study introduces computational advancements and novel approaches, advancing model reduction techniques and highlighting the significance of SIM in addressing challenges within mechanical engineering contexts.