Model-free Method Research Articles

Prediction of order parameters based on protein NMR structure ensemble and machine learning.

The fast motions of proteins at the picosecond to nanosecond timescale, known as fast dynamics, are closely related to protein conformational entropy and rearrangement, which in turn affect catalysis, ligand binding and protein allosteric effects. The most used NMR approach to study fast protein dynamics is the model free method, which uses order parameter S2 to describe the amplitude of the internal motion of local group. However, to obtain order parameter through NMR experiments is quite complex and lengthy. In this paper, we present a machine learning approach for predicting backbone 1H-15N order parameters based on protein NMR structure ensemble. A random forest model is used to learn the relationship between order parameters and structural features. Our method achieves high accuracy in predicting backbone 1H-15N order parameters for a test dataset of 10 proteins, with a Pearson correlation coefficient of 0.817 and a root-mean-square error of 0.131.

Research on improved partial format MFAC greenhouse temperature control method based on low energy consumption optimization

Temperature is critical to the growth of crops in agricultural greenhouses. Thus, designing a greenhouse temperature controller that maximizes energy savings while maintaining control accuracy is very important. This paper proposes an improved partial format model-free adaptive control method and designs a greenhouse temperature controller based on this method to balance control accuracy and energy consumption. Firstly, a limited energy consumption term is added to the control input cost function of the traditional partial format model-free adaptive control to penalize excessive control input. We derive an improved partial format model-free adaptive control input algorithm and design a greenhouse temperature controller using this algorithm. Then, the controller's sensitive parameters are selected using a Monte Carlo-based parameter sensitivity method. Finally, the whale optimization algorithm is used to optimize the sensitive parameters. The insensitive parameters are set according to experience. This paper, a theoretical study based on simulated experiments, proves the convergent tracking error of the proposed improved partial format model-free adaptive algorithm. Simulation results show that the improved controller minimizes energy consumption while ensuring control accuracy, reducing power consumption by 12.35 % compared to the traditional controller.

Sample Efficient Reinforcement Learning with Partial Dynamics Knowledge

The problem of sample complexity of online reinforcement learning is often studied in the literature without taking into account any partial knowledge about the system dynamics that could potentially accelerate the learning process. In this paper, we study the sample complexity of online Q-learning methods when some prior knowledge about the dynamics is available or can be learned efficiently. We focus on systems that evolve according to an additive disturbance model of the form S_{h+1} = ƒ(S_h, A_h) + W_h, where ƒ represents the underlying system dynamics, and W_h are unknown disturbances independent of states and actions. In the setting of finite episodic Markov decision processes with S states, A actions, and episode length H, we present an optimistic Q-learning algorithm that achieves Õ(Poly(H)√T) regret under perfect knowledge of ƒ, where T is the total number of interactions with the system. This is in contrast to the typical Õ(Poly(H)√SAT) regret for existing Q-learning methods. Further, if only a noisy estimate ƒ_hat of ƒ is available, our method can learn an approximately optimal policy in a number of samples that is independent of the cardinalities of state and action spaces. The sub-optimality gap depends on the approximation error ƒ_hat − ƒ, as well as the Lipschitz constant of the corresponding optimal value function. Our approach does not require modeling of the transition probabilities and enjoys the same memory complexity as model-free methods.

Adaptive Control for Virtual Synchronous Generator Parameters Based on Soft Actor Critic.

This paper introduces a model-free optimization method based on reinforcement learning (RL) aimed at resolving the issues of active power and frequency oscillations present in a traditional virtual synchronous generator (VSG). The RL agent utilizes the active power and frequency response of the VSG as state information inputs and generates actions to adjust the virtual inertia and damping coefficients for an optimal response. Distinctively, this study incorporates a setting-time term into the reward function design, alongside power and frequency deviations, to avoid prolonged system transients due to over-optimization. The soft actor critic (SAC) algorithm is utilized to determine the optimal strategy. SAC, being model-free with fast convergence, avoids policy overestimation bias, thus achieving superior convergence results. Finally, the proposed method is validated through MATLAB/Simulink simulation. Compared to other approaches, this method more effectively suppresses oscillations in active power and frequency and significantly reduces the setting time.

Thermogravimetric analysis and kinetic modeling of the pyrolysis of different biomass types by means of model-fitting, model-free and network modeling approaches

AbstractThis work aims at comparing the ability of 7 modeling approaches to simulate the pyrolysis kinetics of spruce wood, wheat straw, swine manure, miscanthus and switchgrass. Measurements were taken using a thermogravimetric analyzer (TGA) with 4 heating rates comprised between 5 and 30 K min−1. The obtained results were processed using 3 isoconversional methods (Kissinger–Akahira–Sunose (KAS), Ozawa–Flynn–Wall (OFW) and Friedman), 1-step and 3-step Kissinger models, as well as an advanced fitting method recently proposed by Bondarchuk et al. [1] (Molecules 28:424, 2023, https://doi.org/10.3390/molecules28010424). Seventeen reaction models were considered to derive rate constant parameters, which were used to simulate the variation of the fuel conversion degree $$\alpha$$ α as a function of the temperature $$T$$ T . To complement this benchmarking analysis of the modeling approaches commonly used to simulate biomass pyrolysis, a network model, the bio-CPD (chemical percolation devolatilization), was additionally considered. The suitability of each model was assessed by computing the root-mean-square deviation between simulated and measured $$\alpha =f(T)$$ α = f ( T ) profiles. As highlights, the model-free methods were found to accurately reproduce experimental results. The agreement between simulated and measured data was found to be higher with the Friedman model, followed by the KAS, FWO, 3-step, and 1-step Kissinger models. As for the bio-CPD, it failed to predict measured data as well as the above-listed models. To conclude, although it was less efficient than the Friedman, KAS or OFW models, the fitting approach from Bondarchuk et al. [1] (Molecules 28:424, 2023, https://doi.org/10.3390/molecules28010424) still led to satisfactory results, while having the advantage of not requiring the selection of a reaction model a priori.

Investigation on prospective bioenergy from pyrolysis of macadamia nut shell waste using multicomponent Fraser-Suzuki kinetic model, kinetic triplet and thermodynamic parameters

The investigation of macadamia nut shell waste (MNSW) was conducted to elucidate its bioenergy potential through the analysis of physicochemical properties, kinetic triplet, reaction mechanism models, and thermodynamic parameters. The thermal degradation experiments were carried out using a thermogravimetric analyzer under nitrogen atmosphere at four different heating rates. The multicomponent Fraser-Suzuki kinetic model revealed that the primary decomposition stage could be successfully divided as the parallel devolatilization process of two, three, or four pseudo components. The thermal degradation process of MNSW can be divided into pseudo hemicellulose (PE-H), hemicellulose-lignin (PE-HL), hemicellulose-cellulose (PE-HC), cellulose (PE-C), and lignin (PE-L) employing the multicomponent Fraser-Suzuki kinetic model. The apparent activation energy (E) was calculated using Friedman, Ozawa-Flynn-Wall, Starink, and Kissinger-Akahira-Sunose four model-free methods. The average apparent activation energy calculated through the Starink model-free method was found to be 190.13 and 174.18 kJ/mol for PE-HL and PE-C, 143.79, 169.99, and 249.88 kJ/mol for PE-H, PE-C and PE-L, and 142.98, 157.15, 169.84, and 244.62 kJ/mol for PE-H, PE-HC, PE-C and PE-L, respectively. Evaluation of thermodynamic parameters showed that the pyrolysis process of each pseudo component was an endothermic non-spontaneous reaction independent of surface area and required additional heat and energy input. The pyrolysis process of all pseudo components was accurately characterized using order-based, nucleation, geometrical contraction, power law, and diffusional reaction mechanism models using master plots method. This work provides a theoretical basis for MNSW or other biomass wastes as attractive and environmentally friendly alternatives for bioenergy production, offering crucial insights for reactor design and improving conversion efficiency.

Robust hierarchical games of linear discrete-time systems based on off-policy model-free reinforcement learning

An off-policy model-free reinforcement learning (RL) algorithm is proposed for a robust hierarchical game while considering incomplete information and input constraints. The robust hierarchical game exhibits characteristics of a Stackelberg–Nash (SN) game, where equilibrium points are designated as Stackelberg–Nash–Saddle equilibrium (SNE) points. An off-policy method is employed for the RL algorithm, addressing input constraints by using excitation input instead of real-time update polices as control inputs. Moreover, a model-free method is implemented for the off-policy RL algorithm, accounting for the challenge posed by incomplete information. The goal of this paper is to develop an off-policy model-free RL algorithm to obtain approximate SNE polices of the robust hierarchical game with incomplete information and input constraints. Furthermore, the convergence and effectiveness of the off-policy model-free RL algorithm are guaranteed by proving the equivalence of Bellman equation between nominal SNE policies and approximate SNE policies. Finally, a simulation is provided to verify the advantage of the developed algorithm.

Finite-time model-free robust synchronous control of multi-lift overhead cranes based on iterative learning

A model-free control method based on iterative learning law combined with adaptive super-twisting is proposed to realize the synchronous coordination control of multi-lift overhead crane system for the problems of inaccurate modeling, system parameter variation, and disturbance uncertainty that exist in multi-lift overhead crane system. First, a load-coupling model of the double-container overhead crane considering the deformation tangential force in the interlocking mode is established. Second, a time-varying sliding mode surface (TSMC) designed using nonlinear functions effectively improves the convergence speed of the system state. The method of iterative learning control is introduced to compensate the system dynamics to achieve model-free control, and the dynamic iterative learning control (DILC) is designed to improve the convergence speed of the error of the system and the steady-state performance. To suppress uncertainty disturbances and avoid control gain overestimation, an adaptive gain is added to the generalized super-twisting algorithm, which has the advantages of both finite-time convergence and chattering suppression, and improves the robustness and tracking performance of the multi-lift overhead crane system. The stability of the controlled system is analyzed using Lyapunov stability theory. The simulation experiments illustrate the effectiveness of the proposed synchronization control scheme.

Pyrolysis of macadamia nut peel using multicomponent Gaussian kinetic modeling and ANN analysis

The physicochemical properties of macadamia nut peel (MNP) were explored based on pyrolysis characteristics, kinetics, thermodynamic parameters, reaction mechanism, and artificial neural network (ANN) application to evaluate its suitability and potential for pyrolytic conversion into bioenergy products. Pyrolysis experiments of MNP were performed using a thermogravimetric analyzer in a nitrogen atmosphere. The thermal degradation process of MNP can be successfully modeled into pseudo hemicellulose (PS–H1), cellulose (PS–C2), and lignin (PS-L3) using a multicomponent Gaussian kinetic model with high correlation coefficient (R2 > 0.99). The average apparent activation energy was 179.28 kJ/mol for PS-H1, 192.31 kJ/mol for PS-C2, and 129.42 kJ/mol for PS-L3 using Ozawa-Flynn-Wall, Kissinger-Akahira-Sunose, Starink, and Tang four model-free methods. Master plots method was used to determine the reaction mechanism models of three pseudo components, and the pyrolysis progress can be described well using order-based and nucleation mechanism models. The thermal degradation behaviors of MNP were predicted using artificial neural network, and the best artificial neural network prediction model was ANN (3*15*1). Based on the physicochemical properties, pyrolysis characteristics, kinetics, thermodynamic parameters, reaction mechanism and ANN prediction model, MNP is suitable for pyrolytic conversion into bioenergy products.

Model-Free Subsampling Method Based on Uniform Designs

Model-Free Subsampling Method Based on Uniform Designs

A model free method of predicting transient dynamics in anaerobic digestion.

Transient dynamics pose unique challenges when dealing with predictions and management of ecological systems yet little headway has been made on understanding when an ecological system might be in a transient state. As a start we consider a specific model, here focusing on a canonical model for anaerobic digestion. Through a series of simplifications, we analyse the potential of the model for transient dynamics, and the driving mechanisms. Using a stochastic analogue of this model, we create synthetic ecological data. Thus, combining our understanding of the deterministic transient dynamics with the use of empirical dynamical modelling, we propose several new metrics to indicate when the synthetic time series is leaving a transient state.

Kinetic Studies of Effects of Heat on PET Plastics: TGA, DTA, XRD and DSC Analysis

In this work, the effects of heat on plastic waste (PET plastic bottles) were investigated. During thermogravimetric analysis, the kinetics of the PET pyrolysis was studied using a combination of model-fitting and model-free methods. About 12mg of the grinded PET sample was weighed into the ceramic crucible and then heated from temperature of 30 °C to 950 °C at two heating rates of 10°C/min and 20°C /min. Thermogravimetric analysis indicated that the decomposition of plastic waste in a Nitrogen atmosphere can be divided into three stages: the minor loss stage (100°C), the major loss stage (450–520°C) and the stable loss stage (500–900°C). The corresponding weight loss at two different heating rates of 10°C/min and 20°C /min were determined to of 70% and 77.7% respectively. Based on R2 (regression coefficient) values, heating rate of 20oC/min gave the values (0.9279) for the model free method employed (KAS model). When compared to the model fitting, 0.75th order of reaction gave the closest R2 value (0.9643) which also translated to a very close Activation energies of 264 KJ/mol and 255 KJ/mol respectively. All in all, it can be deduced that the activation energy of PET pyrolysis falls within 250 – 280 KJ/mol based on this study. Also, X-Ray Diffraction and Differential Scanning Calorimetry analysis were carried out. The narrow and high nature of the XRD peaks depicted that the sample is crystalline. And lastly, DSC analysis investigated the heat flowing in and out of the PET sample. The exodermal (where cold re-crystallization occurred) and endothermal (where melting occur) peaks were recorded as 165oC and 138oC respectively. It is recommended that other model-free and model-fitting methods should be invested in further research in order to generate a wide range of data for kinetics of PET.

Improved model-free synergetic control of permanent magnet synchronous motor using unknown input observer

Parametric uncertainties and inverter nonlinearity exist in the permanent magnet synchronous motor (PMSM) drive system, which may lead to performance degradation or failure of model-based control. This paper presents an enhanced model-free synergetic control method for PMSMs, addressing the limitations of relying on accurate motor models in existing speed control strategies. The proposed approach aims to achieve robust control performance in the presence of parameter perturbations without the need for an accurate motor model. Firstly, a new ultra-local model of the speed loop for the PMSM is established based on a newly developed ultra-local theory. Subsequently, a model-free synergetic controller is designed using the principles of synergetic control theory. To address the issue of output chattering in the sliding mode observer used to estimate the unknown part of the ultra-local model, a synergetic observer method is proposed. Simulation results are presented and compared with those obtained using a proportional-integral (PI) controller and a conventional model-free sliding mode controller. The results demonstrate that the model-free synergetic controller exhibits robust performance and provides accurate estimation of the unknown part without output chattering.

Kinetics Study of PVA Polymer by Model-Free and Model-Fitting Methods Using TGA.

Thermogravimetric Analysis (TGA) serves a pivotal technique for evaluating the thermal behavior of Polyvinyl alcohol (PVA), a polymer extensively utilized in the production of fibers, films, and membranes. This paper targets the kinetics of PVA thermal degradation using high three heating rate range 20, 30, and 40 K min-1. The kinetic study was performed using six model-free methods: Freidman (FR), Flynn-Wall-Qzawa (FWO), Kissinger-Akahira-Sunose (KAS), Starink (STK), Kissinger (K), and Vyazovkin (VY) for the determination of the activation energy (Ea). TGA showed two reaction stages: the main one at 550-750 K and the second with 700-810 K. But only the first step has been considered in calculating Ea. The average activation energy values for the conversion range (0.1-0.7) are between minimum 104 kJ mol-1 by VY to maximum 199 kJ mol-1 by FR. Model-fitting has been applied by combing Coats-Redfern (CR) with the master plot (Criado's) to identify the most convenient reaction mechanism. Ea values gained by the above six models were very similar with the average value of (126 kJ mol-1) by CR. The reaction order models-Second order (F2) was recommended as the best mechanism reaction for PVA pyrolysis. Mechanisms were confirmed by the compensation effect. Finally, (∆H, ∆G, and ∆S) parameters were presented and proved that the reaction is endothermic.

Pyrolysis of natural rubber–cellulose composites: isoconversional kinetic analysis based on thermogravimetric data

Despite the current growing interest in rubber composites with natural organic fillers, there is a lack of kinetic analyses that describe the decomposition of these materials during pyrolysis. For this reason, the main objective of this study was the kinetic analysis and determination of formal kinetic parameters for the pyrolytic decomposition of NR–CEL composites with different cellulose content (0, 30, 45, and 55 phr). Thermogravimetric measurements were made at heating rates of 2, 4, 6, 8, 10, and 20 °C min–1 in the temperature range of 20–600 °C. First, Friedman and KAS model-free methods were applied. Therefore, model-based methods and the model-fitting procedure were used to find the optimal multi-step kinetic model. The proposed final model consists of two parallel processes, which are kinetically independent: A → B → C and D → E → F. For each step, a kinetic triplet was calculated: the apparent activation energy, the pre-exponential factor, and the kinetic parameters of the extended empirical Prout–Tompkins model. The master plots method was used to determine the kinetic decomposition mechanism of the individual steps. It was found that step A → B has the shape of an nth-order model, step B → C mainly follows the diffusion model, the mechanism of step D → E transfers from a random scission kinetics model to an nth-order model with an increasing amount of CEL, and step E → F obeys the chain scission mechanism.

Comparative investigation of low temperature oxidized and pyrolyzed cokes from the inferior heavy oil: Structural feather, thermo-oxidation behavior, and kinetics

It was crucial to study the low temperature coking process for the heavy oil, which affected the fuel availability and subsequent in-situ combustion progress. From this perspective, low temperature coking experiments with different atmospheres were accomplished for the inferior heavy oil to obtain oxidized (cokeLTO) and pyrolytic (cokeLTP) cokes, which were further employed to trace structural feathers via joint analysis of elemental analysis, Fourier transform infrared spectrophotometry (FTIR) and X-ray photoelectron spectroscopy (XPS) spectra, and to characterize thermo-oxidative characteristics and kinetics via different thermal analyses (TG-DTG/DSC) and model-free methods. The results showed that a relatively less aliphatic and more aromatic structures were contained in cokeLTO with a relatively higher proportion of C-O, C = O, O-C = O groups and C/H ratio, implying that oxygen atom was served as the promoter to replace hydrogen atom and accelerate macromolecular structure formation. While the decomposition and conversion of N- and S- functional groups were complicated and insensitive to the coking atmosphere. Furthermore, cokeLTO had a more dominant thermo-oxidation behavior than cokeLTP with better exothermic characteristic and lower activation energy distribution. From this study, it could provide a vital piece of updated insights on the role of low temperature oxidation process in coke deposition and feature.

Size-dependent thermal oxidation kinetics in Sm2(Co, Fe, Cu, Zr)17 alloy powder

This study investigates the interrelationship between the oxide growth kinetics and the microstructural changes induced by dry, thermal oxidation in Sm2(Co, Fe, Cu, Zr)17 alloy powder (of varying particle sizes and compositions). Whereas, model-free iso-conversional methods are used to determine the kinetic-triplets during the non-isothermal oxidation stage, power-law dependent kinetic model is considered during the entire oxidation process to calculate the reaction order, rate constant and the apparent-activation energy. A distinctly different oxide growth kinetics, with strong dependency on the initial particle size(s) and the alloy composition-induced oxide microstructure is seen here.

Protein design using structure-based residue preferences

Recent developments in protein design rely on large neural networks with up to 100s of millions of parameters, yet it is unclear which residue dependencies are critical for determining protein function. Here, we show that amino acid preferences at individual residues—without accounting for mutation interactions—explain much and sometimes virtually all of the combinatorial mutation effects across 8 datasets (R2 ~ 78-98%). Hence, few observations (~100 times the number of mutated residues) enable accurate prediction of held-out variant effects (Pearson r > 0.80). We hypothesized that the local structural contexts around a residue could be sufficient to predict mutation preferences, and develop an unsupervised approach termed CoVES (Combinatorial Variant Effects from Structure). Our results suggest that CoVES outperforms not just model-free methods but also similarly to complex models for creating functional and diverse protein variants. CoVES offers an effective alternative to complicated models for identifying functional protein mutations.

Pyrolytic characteristics of abutilon stalk waste using TG-FTIR, Py-GC/MS, and artificial neural networks: Kinetics, thermodynamics, and gaseous products distribution

The physicochemical properties of abutilon stalk (ABUS) were investigated to assess its suitability for pyrolytic conversion into bioenergy and bio-based chemicals based on kinetic triplet state, thermodynamic parameters, decomposition mechanisms and product distributions. The pyrolysis behavior of ABUS was studied using a thermogravimetric analyzer at five heating rates in a nitrogen atmosphere. The Gaussian deconvolution function indicated that the pyrolysis process of ABUS can be successfully modeled as four parallel devolatilization events (R2 > 0.99), divided into the devolatilization of pseudo drying (PO-D), hemicellulose (PO-H), cellulose (PO-C), and lignin (PO-L), respectively. The average apparent activation energies were 34.89 kJ/mol for PO-D, 163.02 kJ/mol for PO-H, 169.55 kJ/mol for PO-C, and 107.51 kJ/mol for PO-L using four model-free methods (Ozawa-Flynn-Wall, Kissinger-Akahira-Sunose, Starink, and Tang). The pyrolysis process was accurately predicted using diffusional, power law, geometrical contraction, and order-based reaction mechanism models. Thermal degradation process for each devolatilization event was successfully reconstructed using combined kinetics (R2 > 0.96), providing a useful theoretical basis and mathematical model for predicting the pyrolysis behavior of ABUS. The volatile products produced by ABUS pyrolysis were characterized using an integrated TGA-FTIR and Py-GC/MS system, confirming the proportion of high-energy compounds (benzenes, phenols, ketones, alcohols, aldehydes, acids, esters, hydrocarbons and their derivatives, N-heterocyclic substances, and N-containing compounds) and gas emission levels (CO > CH4 > C-O > H2O > CO2 > CO > C-O(H) > HCN > NH3 > SO2). Artificial neural network (ANN) was used to predict the pyrolysis behaviors of ABUS, and the best ANN model was ANN (5 *11 *1) with high R2 of 0.99998. Based on the kinetics, thermodynamics, and gaseous products distribution, ABUS has great application potential as raw material for bioenergy production.

Kinetic analysis of the crystallization of Y2O3 and La2O3 doped Li2O-Al2O3-SiO2 glass.

Li2O-Al2O3-SiO2 (LAS) glass ceramics have a low coefficient of thermal expansion, high mechanical strength, and excellent chemical stability. With advancements in glass ceramics, researchers have explored using LAS glass ceramics with transition metal doping and rare earth doping. Most previous studies have studied the impact of rare earth element doping on crystallization primarily in the context of conventional nucleating agents present in glass. In this study, we aimed to investigate the impact of Y2O3 and La2O3 on LAS glasses in the presence of undoped nucleating agents. The crystallization mechanism of La2O3 and Y2O3 doped LAS glass ceramics was studied using differential scanning calorimetry. The crystallization kinetics of the glasses were analyzed using model-free and modeling methods. Moreover, the activation energy of crystallization and the indices of crystallization and growth of glass ceramics were calculated, and the crystalline phase and microstructure of the samples were characterized. All three fractions of the LAS glass showed consistent crystallization under different calculation methods. The glass doped with La2O3 and Y2O3 exhibited two- or three-dimensional growth during crystallization, promoting crystallization in the LAS glass. The Y3La0 sample demonstrated the most favorable crystallization effect. In the presence of an undoped nucleating agent, rare earth elements can enhance glass crystallization; this new idea can be utilized for the development of new materials.