Model-free Method Research Articles

A deep reinforcement learning based charging and discharging scheduling strategy for electric vehicles

Grid security is threatened by the uncontrolled access of large-scale electric vehicles (EVs) to the grid. The scheduling problem of EVs charging and discharging is described as a Markov decision process (MDP) to develop an efficient charging and discharging scheduling strategy. Furthermore, a deep reinforcement learning (DRL)-based model-free method is suggested to address such issue. The proposed method aims to enhance EVs charging and discharging profits while reducing drivers' electricity anxiety and ensuring grid security. Drivers' electricity anxiety is described by fuzzy mathematical theory, and analyze the effect of EV current power and remaining charging time on it. Variable electricity pricing is calculated by real-time residential load. A dynamic charging environment is constructed considering the stochasticity of electricity prices, driver’s behavior, and residential load. A soft actor-critic (SAC) framework is used to train the agent, which learns the optimal charging and discharging scheduling strategies by interacting with the dynamic charging environment. Finally, simulation with actual data is used to verify that the suggested approach can reduce drivers' charging costs and electricity anxiety while avoiding transformer overload.

Thermal decomposition of wheat straw pellets in a nitrogen environment: Characterization Using Thermogravimetric Analyzer

This experiment used a thermogravimetric (TG) analyzer within a nitrogen environment, investigating the thermal degradation patterns of wheat straw pellets (WSP) under temperatures ranging from 31 to 800°C and varying heating rates (5, 10, and 20 °C/min). Two pellet types were considered: T1 (100% wheat straw) and T2 (70% wheat straw, 10% sawdust, 10% bentonite clay, and 10% biochar). This study comprehensively analyzes WSP’s thermal degradation, emphasizing model selection, composition effects, heating rate, and temperature. Results highlighted higher volatile matter content and calorific value in WSP. Model-free methods were applied to analyze TG/DTG profiles, revealing three distinct zones in WSP thermal decomposition: drying, devolatilization, and carbonization. Devolatilization, especially its 1st and 2nd steps, was extensively examined, with a significant mass loss (approx. 65%) observed between 150 and 550°C. Higher heating rates induced a shift in thermogravimetric profiles to elevated temperatures. Maximum mass loss rates during devolatilization ranged from 4.41 to 16.28%/min for T1 and 4.0 to 15.9% for T2 pellets. Temperature significantly influenced mass loss and reaction rates, whereas heating rates had a negligible impact. Thermodynamic properties indicated equilibrium reactions during pyrolysis for both T1 and T2 pellets. Additionally, increasing heating rates correlated with an upward trend in the reactivity index. The findings contribute valuable knowledge for optimizing biomass utilization in combustion and pyrolysis processes.

Data-based power management control for battery supercapacitor hybrid energy storage system in solar DC-microgrid.

This paper addresses the energy management control problem of solar power generation system by using the data-driven method. The battery-supercapacitor hybrid energy storage system is considered to smooth the power fluctuation. A new model-free control method is utilized in the stand-alone photovoltaic DC-microgrid to provide the power to meet the demand load, while guaranteeing the DC bus voltage is stable. Furthermore, the proposed data-based power management control strategy only needs I/O data. Numerical simulations with real data verify the effectiveness of the proposed method.

Application of Path Planning and Tracking Control Technology in Mower Robots

Path planning and tracking is the most basic technology for mowing robots, among which the performance of algorithms has a great impact on their intelligence and efficiency. Based on the research of relevant references on mower robots, it mainly focuses on complete coverage path planning, path tracking control, and obstacle avoidance path planning. In complete coverage path planning, three methods were introduced, including simple complete coverage planning, optimal complete coverage planning, and hybrid complete coverage planning. In the path tracking control section, the control methods are divided into three types based on whether the control method depends on the robot model and the type of model, namely model free control method, kinematic model-based control method, and dynamic model-based control method. In obstacle avoidance path planning, we introduce the environment detection device and obstacle avoidance planning algorithm. Then the relevant research papers are analyzed in classification, comparing the research and validation methods adopted by the researchers in the form of charts. Finally, we pointed out the limitations of path planning technology in the application of mower robots. Meanwhile, future development trends are predicted.

Investigations on the effect of kaolin catalyst on the yield of various products obtained from pyrolysis of low-density polyethylene (LDPE) wastes and reaction kinetics.

The pyrolysis characteristics of low-density polyethylene (LDPE) wastes were investigated in a batch pyrolyzer in the temperature range of 550-650°C for a varying heating rate (5, 10, 15, 20, and 25°C/min) alone and using kaolin catalyst (10, 15, and 20% w/w). The yield distribution of various products under varying operating conditions was investigated, and the corresponding favourable conditions for maximum yield of pyrolytic oil (PO) were identified. The possible effect of the catalyst on pyrolysis kinetics was explored by extracting activation energies from the TGA experiments. The various model-free iso-conversion methods estimated the activation energy of 162-166kJ/mol during thermal degradation of LDPE, while it shifted to a lower range (138-145kJ/mol) on catalytic treatment. The catalytic pyrolysis with a 20% (by mass) catalyst in a batch pyrolyzer provided the maximum PO yield with the least solid residue at 600°C using a heating rate of 20°C/min, while the temperature of 600°C with 10% catalyst showed the least liquid product with more solid residue. The presence of a catalyst improved the oil quality with transparency and lower viscosity due to lighter hydrocarbons and higher aromatic content. The FTIR analysis detected various organic groups-alkanes, alkenes, cyclic, and aromatics in the oil. GC tests of gaseous products confirmed the gases like hydrogen, propane, butane, and propylene. The present study aims to provide technical know-how for the effective utilization of LDPE wastes towards clean energy sources and create a plastic-free green environment.

Priority-based intelligent resolution method of multi-aircraft flight conflicts

Abstract The rising demand for air traffic will inevitably result in a surge in both the number and complexity of flight conflicts, necessitating intelligent strategies for conflict resolution. This study addresses the critical challenges of scalability and real-time performance in multi-aircraft flight conflict resolution by proposing a comprehensive method that integrates a priority ranking mechanism with a conflict resolution model based on the Markov decision process (MDP). Within this framework, the proximity between aircraft in a multi-aircraft conflict set is dynamically assessed to establish a conflict resolution ranking mechanism. The problem of multi-aircraft conflict resolution is formalised through the MDP, encompassing the design of state space, discrete action space and reward function, with the transition function implemented via simulation prediction using model-free methods. To address the positional uncertainty of aircraft in real-time scenarios, the conflict detection mechanism introduces the aircraft’s positional error. A deep reinforcement learning (DRL) environment is constructed incorporating actual airspace structures and traffic densities, leveraging the Actor Critic using Kronecker-factored Trust Region (ACKTR) algorithm to determine resolution actions. The experimental results indicate that with 20–30 aircraft in the airspace, the success rate can reach 94% for the training set and 85% for the test set. Furthermore, this study analyses the impact of varying aircraft numbers on the success rate within a specific airspace scenario. The outcomes of this research provide valuable insights for the automation of flight conflict resolution.

A Model-Free Adaptive Positioning Control Method for Underactuated Unmanned Surface Vessels in Unknown Ocean Currents

Aiming to address the problem of underactuated unmanned surface vehicles (USVs) performing fixed-point operations at sea without dynamic positioning control systems, this paper introduces an original approach to positioning control: the virtual anchor control method. This method is applicable in environments with currents that change slowly and does not require prior knowledge of current information or vessel motion model parameters, thus offering convenient usability. This method comprises four steps. First, a concise linear motion model with unknown disturbances is proposed. Then, a motion planning law is designed by imitating underlying principles of ship anchoring. Next, an adaptive disturbance observer is proposed to estimate uncertainties in the motion model. In the last step, based on the observer, a sliding-mode method is used to design a heading control law, and a thrust control law is also designed by applying the Lyapunov method. Numerical simulation experiments with significant disturbances and tidal current variations are conducted, which demonstrate that the proposed method has a good control effect and is robust.

Kinetic research of red mud waste oxidative pyrolysis and comparison under different atmospheres

Red mud, as a solid waste with high alkalinity, had a detrimental impact on the environment and required urgent attention. Currently, the mass processing and consumption of red mud were typically conducted under thermal conditions, so it was essential to gain a comprehensive understanding of the oxidative pyrolysis process. The thermogravimetric experiments were conducted at multiple heating rates in air and exhibited three obvious stages. The activation energy and reaction mechanism of three oxidative pyrolysis stages were explored using model-free and model-fitting methods, revealing the activation energies of 162.2, 265.8, 214.1 kJ/mol and the most suitable reaction mechanisms of g(α)=[-ln(1-α)]³, g(α)=1-(1-α)1/⁴, g(α)=[-ln(1-α)]1/2 for each stage, respectively. Furthermore, the estimated kinetic parameters and reaction mechanisms were applied to extra heating rate to verify the accuracy. More important, the effect of air on the pyrolysis process of red mud was examined by comparing the results with those obtained from pure nitrogen pyrolysis. The obtained oxidative pyrolysis characteristics of red mud could provide valuable insights of its co-pyrolysis or combustion for resources recycling.

A model free adaptive control method based on self-adjusting PID algorithm in pH neutralization process

In this paper, a new model free adaptive control method based on self-adjusting PID Algorithm (MFAC-SA-PID) is proposed to solve the problem that the pH process with strong nonlinearity is difficult to control near the neutralization point. The MFAC-SA-PID method also solves the problem that the parameters of the model free adaptive control (MFAC) method are not easy to be adjusted and the effect is not obvious by introducing a fuzzy self-adjusting algorithm to adjust the controller parameters. Then the convergence and stability of the MFAC-SA-PID method are proved in this paper. In the simulation study, the control performance of the MFAC-SA-PID method proposed in this paper is compared with the traditional MFAC method and the improved model free adaptive control (IMFAC) method, respectively. The results show that the proposed MFAC-SA-PID method has better control effect on the pH neutralization process. The MFAC-SA-PID control performance also outperforms the traditional MFAC method and IMFAC method when step input disturbances are added, which indicates that the MFAC-SA-PID method has better robustness and stability.

Pyrolysis characteristics and kinetic analysis of coating pitch derived from ethylene tar using model-free and model-fitting methods

This work conducted the pyrolysis of coating pitch derived from the ethylene tar through thermogravimetry analysis. The thermogravimetry-mass spectrometry and solid-state 13C nuclear magnetic resonance spectroscopy were performed to correlate the structural characteristics with the pyrolysis behavior. Meanwhile, the pyrolysis kinetics were estimated with model-free and model-fitting methods. The TG-DTG results showed that the pyrolysis process was generally divided into drying, fast pyrolysis, and polycondensation stages, accompanied by the decomposition of active functional groups like the aliphatic carbons bonded to oxygen (falO), the pyrolysis of relatively stable groups like the non-protonated aromatic carbon (faH), and the condensation of high bonding-energy aromatic structure like aromatic bridgehead carbon (faB). The activation energy obtained from model-free analysis ranged from 78.12 to 150.92kJ/mol with the increasing conversion, indicating the pyrolysis process as a multi-step reaction mechanism. Furthermore, the reaction model A1/3 (gα=[−ln(1−α)]3) was identified as the most suitable model for the pyrolysis of coating pitch in the conversion range of 0.25-0.8. The modeling values matched well with the experimental data, indicating the accuracy and feasibility of the results.

Kinetic studies on the catalyzed and un-catalyzed pyrolysis of mixed HDPE and PP (75:25 wt%) plastic waste using a combination of model-fitting and model-free methods

Kinetic studies on the catalyzed and un-catalyzed pyrolysis of mixed HDPE and PP (75:25 wt%) plastic waste using a combination of model-fitting and model-free methods

Kinetic analysis of coal oxidative pyrolysis before and after immersion effect

The oxidative pyrolysis kinetics of coal before and after immersion effect was studied by thermogravimetric experiments. Non-isothermal thermogravimetric analysis data were analyzed at four heating rates of 5, 10, 20 and 30 K/min. Furthermore, temperature-programmed experiments and Fourier-transform infrared (FTIR) spectroscopy were employed to understand the oxidation activity and chemical structure of raw coal and soaked coal. After coal immersion effect, the increase in the content of active functional groups, the increase in the concentration of gaseous products, and the decrease in crossing point temperature during oxidation process all indicate that soaked coal is easier to oxidize and spontaneously combust. Lastly, four model-free methods such as Flynn-Wall-Ozawa (FWO), Kissinger-Akahira-Sunose (KAS), Friedman and Kissinger methods are employed to calculate the activation energy (Eα). Kinetic analysis reveals that Eα value obtained by the former three methods significantly depends on the conversion of coal oxidative pyrolysis process, and the average Eα value of soaked coal obtained by four methods is lower than that of raw coal. For the kinetic analysis of coal oxygen adsorption and combustion stages, it is more reliable to adopt the FWO and KAS methods in turn. This research helps to better understand the mechanism of enhanced oxidation activity of soaked coal and optimizes the calculation method of kinetic parameters in different oxidation stages of coal.

An Overview Of Deep Deterministic Policy Gradient Algorithm And Applications

This research summarizes the major developments of the Deep Deterministic Policy Gradient (DDPG) in reinforcement learning Motivated by the ideas, of Deep Q-networks, DDPG has proven capable of confronting more complex problems involving continuous action spaces. The core of DDPG lies in its actor-critic architecture, which enables the learning of highly competitive policies. By leveraging neural network function approximations, it can efficiently operate in large state and action spaces. DDPG has found practical applications across various real-world domains. However, like many model-free reinforcement learning methods, DDPG still faces the challenge of requiring a large number of training steps.

Model-Free Current Predictive Control Method for Switched Reluctance Motors

Model-Free Current Predictive Control Method for Switched Reluctance Motors

Bayesian impedance deconvolution using timescale distribution for lithium-ion battery state estimation

The distribution of relaxation times (DRT) and distribution of diffusion times (DDT) are widely recognized as effective model-free methods for deconvolving the internal properties of complex electrochemical systems using electrochemical impedance spectroscopy (EIS) data. This study proposes an integrated framework that employs a Bayesian approach to accurately estimate both DRT and DDT and machine learning techniques to enhance capacity estimation, incorporating Gaussian process regression and transformer networks. These methods utilize the inferred DRT and DDT as inputs to predict the discharge capacity. In addition, we perform a peak analysis on the estimated DRT and DDT to extract additional physically meaningful features, which have also proven to be effective inputs for capacity prediction. The applicability of the proposed framework to the EIS experimental data of lithium-ion cells is demonstrated and compared with existing capacity estimation methods. The proposed framework demonstrates a mean absolute error in capacity prediction below 3.03% when using DRT and DDT directly and 2.71% when using extracted features, outperforming existing methods by approximately one to three percentage points. Our Bayesian approach, combined with peak analysis and the integration of machine learning techniques, enables a more robust diagnosis of the internal state of lithium-ion batteries through EIS measurements.

The Lattice Strain Distribution in GexSn1-x Micro-Disks Investigated at the Sub 100-nm Scale

Experimental assessment of the strain tensor within a microstructure is challenging, especially for small mechanical deformations acting over submicron length scales. In this work, we fully characterize the spatial strain distribution within a suspended micro-disk laser made of Ge1-xSnx alloy, with fine resolution <200 nm. We employ Scanning X-ray Diffraction Microscopy, a model-free method based on synchrotron radiation, to directly obtain maps of all components of lattice strain and rotation, including the shear strains, finding them on a magnitude ~10-3. We correlate these small elastic deformations to structural defects and the relaxation of the three-dimensional microstructure, demonstrating the potential of an advanced X-ray microscopy technique for microelectronics.

A novel intelligent control of discrete-time nonlinear systems in the presence of output saturation

In this paper, a model free control method for a class of discrete time nonlinear systems is introduced. A type-3 fuzzy system estimates the unknown parameters required by the control system. The control system only uses the input and output data of the plant and therefore does not need to know its mathematical equations. On the other hand, the phenomenon of output saturation is a challenging problem for all control systems, addressed in detail in the proposed method. The convergence of the proposed method is guaranteed, and the control system is very robust in the face of changes in the dynamics of the plant. The simulation results on discrete-time nonlinear systems show that the proposed method is very accurate despite the high speed of convergence. In addition, the proposed method is robust for modeling uncertainties and has a better root mean square error and step response time compared to the other methods. Also, a comparison has been made between type-1 to type-3 fuzzy systems and control system based on trial and error, which shows firstly the importance of the presence of fuzzy system and secondly the superiority of type-3 fuzzy system compared to the other two types.

Optimal load distribution control for airport terminal chiller units based on deep reinforcement learning

The building sector consumes substantial energy, with HVAC systems accounting for nearly half of the total energy use. Optimizing chiller unit operations is crucial for reducing energy consumption. This research introduces a novel model-free control method for optimizing central chiller load distribution using deep reinforcement learning (DRL). The aim is to address the challenges posed by the high-dimensional complexity and parameter tuning in traditional meta-heuristic algorithms. The proposed method leverages deep neural networks combined with reinforcement learning to optimize chiller control based on real-world data. Case studies conducted in a large airport terminal demonstrate that this approach achieves a 7.71 % energy savings, closely matching the performance of model-based control methods with only a 0.26 % difference. The results indicate that the DRL method not only outperforms traditional expert control systems but also offers a feasible alternative for optimal control in complex, variable environments with limited data. This study contributes to the field by filling a research gap in applying DRL for optimal chiller load (OCL) control and demonstrates its potential for large-scale public building applications. The novelty of this research lies in its model-free approach, which provides a robust solution for energy optimization in dynamic and uncertain environments.

Combined kinetic analysis of thermal degradation characteristics and reaction mechanism of thin intumescent fire-retardant coating of steel structure

To determine the thermal kinetic triplets and elucidate the reaction mechanism of thin intumescent fire-retardant coatings (IFR) of steel structures, one efficient combined kinetic approach was implemented in this study. Thermogravimetric experiments were conducted in air atmosphere at four heating rates, and the whole IFR thermal degradation process was divided into two stages. The average activation energy values derived by model-free methods were 78.9 kJ/mol and 175.16 kJ/mol for Stage I (0<α < 0.2) and Stage II (0.2<α < 0.9), respectively. Furthermore, the linear Coats-Redfern (CR) and non-linear Masterplots models were applied to identify the possible reaction mechanism. It was found that the F3/2 mechanistic model was better suited to the main thermal degradation process. Then model reconstruction based on the F3/2 mechanism model was performed. The results showed that the reconstructed model had a strict linear relationship in the independence analysis and KCE analysis, along with a good agreement between the theoretical and experimental results. The current study provided new insights into the systematic thermal degradation mechanism of IFR, and the proposed kinetic model would be helpful for the thermal protection prediction for steel structure during fire.

Ultra-high-resolution mapping of myelin and g-ratio in a panel of Mbp enhancer-edited mouse strains using microstructural MRI

Non-invasive myelin water fraction (MWF) and g-ratio mapping using microstructural MRI have the potential to offer critical insights into brain microstructure and our understanding of neuroplasticity and neuroinflammation. By leveraging a unique panel of variably hypomyelinating mouse strains, we validated a high-resolution, model-free image reconstruction method for whole-brain MWF mapping. Further, by employing a bipolar gradient echo MRI sequence, we achieved high spatial resolution and robust mapping of MWF and g-ratio across the whole mouse brain. Our regional white matter-tract specific analyses demonstrated a graded decrease in MWF in white matter tracts which correlated strongly with myelin basic protein gene (Mbp) mRNA levels. Using these measures, we derived the first sensitive calibrations between MWF and Mbp mRNA in the mouse. Minimal changes in axonal density supported our hypothesis that observed MWF alterations stem from hypomyelination. Overall, our work strongly emphasizes the potential of non-invasive, MRI-derived MWF and g-ratio modeling for both preclinical model validation and ultimately translation to humans.