Parameter Prediction Method Research Articles

Model Parameter Prediction Method for Accelerating Distributed DNN Training

As the size of deep neural network (DNN) models and datasets increases, distributed training becomes popular to reduce the training time. However, a severe communication bottleneck in distributed training limits its scalability. Many methods aim to solve this communication bottleneck by reducing communication traffic, such as gradient sparsification and quantification. However, these methods either are at the expense of losing model accuracy or introducing lots of computing overhead. We observe that the data distribution between layers of neural network models is similar. Thus, we propose a model parameter prediction method (MP2) to accelerate distributed DNN training under parameter server (PS) framework, where workers push only a subset of model parameters to PS, and residual model parameters are locally predicted by an already-trained deep neural network model on PS. Some key challenges are addressed. First, we build a hierarchical parameters dataset by randomly sampling a subset of model from normal distributed trainings. Second, we build a neural network model with the structure of “convolution + channel attention + Max pooling” for predicting model parameters by using a prediction result-based evaluation method. For VGGNet, ResNet, and AlexNet models on CIFAR10 and CIFAR100 datasets, compared with Baseline, Top-k, deep gradient compression (DGC), and weight nowcaster network (WNN), MP2 can reduce traffic by up to 88.98%; and accelerates the training by up to 47.32% while not losing the model accuracy. MP2 has shown good generalization.

Coati optimization algorithm based Deep Convolutional Forest method for prediction of atmospheric and oceanic parameters

Droughts and floods are examples of extreme weather events that can result from changes in ocean temperature. Ocean temperature is a key component of the global open sea system. Currently, real-time sea surface temperature (SST) forecasts are generated by numerical models based on physics principles and influenced by boundary and initial conditions. These models generally perform better over large areas than at specific locations. To address this and improve prediction accuracy, particularly in high-precision areas, the Coati Optimization Algorithm-based Deep Convolutional Forest (COA-DCF) method is proposed. This optimization approach is utilized to train the Deep Convolutional Forest (DCF) classifier, which then applies the prediction strategy. The COA-DCF method forecasts ocean surface temperature anomalies by considering key variables such as SST, Sea Surface Height (SSH), soil moisture, and wind speed, using historical data ranging from 1 to 10 days across six different locations. The proposed method achieves improved accuracy with low Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) values, and a high Pearson’s correlation coefficient (r) of 0.493, 0.487, and 0.4733, respectively, thereby enhancing the overall performance of the deep learning model.

A real-time multiple tunneling parameter prediction method of TBM steady phase based on dual recurrent neural networks

A real-time multiple tunneling parameter prediction method of TBM steady phase based on dual recurrent neural networks

Reservoir parameters prediction based on spatially transferred long short-term memory network.

Reservoir reconstruction, where parameter prediction plays a key role, constitutes an extremely important part in oil and gas reservoir exploration. With the mature development of artificial intelligence, parameter prediction methods are gradually shifting from previous petrophysical models to deep learning models, which bring about obvious improvements in terms of accuracy and efficiency. However, it is difficult to achieve large amount of data acquisition required for deep learning due to the cost of detection, technical difficulties, and the limitations of complex geological parameters. To address the data shortage problem, a transfer learning prediction model based on long short-term memory neural networks has been proposed, and the model structure has been determined by parameter search and optimization methods in this paper. The proposed approach transfers knowledge from historical data to enhance new well prediction by sharing some parameters in the neural network structure. Moreover, the practicality and effectiveness of this method was tested by comparison based on two block datasets. The results showed that this method could significantly improve the prediction accuracy of the reservoir parameters in the event of data shortage.

Hysteretic model of seawater and sea sand concrete-filled FRP- steel composite tube columns considering the confinement effectiveness

Aiming at the problems of marine infrastructure facing corrosion and complex load environments, a seawater and sea sand concrete-filled CFRP- steel composite tube (SFSCT) column structure was proposed: CFRP sheets are pasted on the inner and outer walls of the steel tube to provide corrosion resistance and improve yield capacity, and then the environment-friendly seawater and sea sand concrete is used to fill the steel tube. Based on pseudo-static tests, a fiber model was established based on OpenSEEs. The axial compression ratio, steel tube thickness, yield strength, shear span ratio, and CFRP layer were analyzed and collected as a database. The influence rules of the designed parameters were obtained. Combining experimental data and fiber model results, a complete hysteretic model was finally proposed based on the equal ratio superposition of parameters. The proposed model includes the parameter prediction methods such as peak load, drift ratio, and stiffness of the descending section, as well as the loading and unloading criteria. A trilinear skeleton curve model was established to predict the developing trends of various parameters. The confinement effectiveness of steel and CFRP was fully taken into considerations in the proposed model. The predicted results are in good agreement with the simulation results of SFSCT.

SOC estimation of lithium battery based on the combination of electrical parameters and FBG non-electrical parameters and using NGO-BP model

Accurately estimating of the state of charge (SOC) of lithium batteries is of great significance for improving the service life and utilization efficiency of lithium batteries. In this paper, fiber Bragg grating (FBG) sensors are used to capture the offset of the FBG central wavelength caused by strain and temperature during the charging and discharging process of lithium batteries, and the current state of charge of lithium batteries is obtained through the combination of electrical parameters and FBG non-electrical parameters.A battery monitoring experimental system was established, and the comparison of wavelength offset collected by PDMS packaged FBG and bare FBG during the charging and discharging process of lithium batteries was discussed. The prediction results were compared by temperature compensating the central wavelength offset collected during the charging and discharging of lithium batteries, then introduced BP neural network and optimized NGO- BP neural network regression models for training to compare the prediction effects, the results of which point to the fact that in terms of regression model performance and improvement of FBG non-electrical parameters on model performance, the various indicators predicted by NGO-BP are stronger than those predicted by BP. After adding FBG non-electrical parameters, both the BP regression model and the NGO-BP regression model showed remarkable improvement in RMSE, MAE, and R2 compared to the purely electrical parameters, and the best prediction method for the NGO-BP non-purely electrical parameters, with RMSE of 0.0201 and MAE of 0.0154, which decreased by 77.94% and 73.12% respectively compared to the BP pure electrical parameter prediction method, and R2 of 0.9988, which increased by 5.88%, thus the joint accurate estimation of lithium battery SOC by electrical measurement characteristics and FBG non-electrical parameters is basically realized, which brings values for the design and development of battery management system.

Downhole parameter prediction method based on multi-layer water injection model and historical data-based model parameter identification

Wireless communication has become a preferred direction for the development of layered water injection tools due to its low cost and high reliability. However, the wireless system relies on the underground battery for power supply，and each communication will consume a significant amount of energy. In order to save energy consumption, the wireless system adopts the intermittent sleep communication mode, with intervals of usually more than one month. During the idle time of communication, the downhole parameters such as pressure and flowrate will change as the pressure and flowrate at the wellhead. Therefore, it is crucial to predict downhole parameters based on the wellhead pressure and flowrate. In this study, a downhole parameter prediction method based on multi-layer water injection model is proposed. A multilayer injection prediction model was established based on the hydraulic analysis of the tubing string, and the model parameters were identified and updated using the historical data uploaded each time. The pressure and flow rate measured at the wellhead were used as inputs to the model, and the recursive relationship between layers in the multilayer model was utilized to predict downhole parameters for each layer. A model parameter optimization method based on time-weighting is proposed in order to address the gradual changes in model parameters during water injection. This method assigns greater weight to more recent historical data, resulting in optimized model parameters. Experimental results show that the proposed method can effectively predict the flowrate and pressure of each layer, with a prediction deviation of less than 5% F.S., which provides technical support for the application and popularization of the wireless layered water injection system.

Heat transfer enhancement for 3D chip thermal simulation and prediction

Parameter changes in the complex internal structure of multi-layer 3D stacked chips will greatly reduce the efficiency of modeling and thermal analysis. In this work, by combining thermal simulation analysis with machine learning algorithms, we can skillfully predict the hotspot temperature changes of 3D chips up to 28 layers while changing some key parameters of 3D chips with less computational effort. K-fold Cross Validation (K-CV) algorithm and support vector regression (SVR) algorithm were developed to predict the variations in hotspot temperature of the 3D chip while changing modeling and cooling parameters. Based on the training matrix, the support vector regression (SVR) model can accurately predict the random power distribution case, the random processor core location distribution case, and the random Through Silicon Via (TSV) distribution case. The validation results show that the prediction accuracy deviation is close to 0.6 K, and the correlation coefficient R2 is close to 1. Meanwhile, the variable parameter and variable layer prediction method based on the aforementioned training model can more accurately predict the hotspot temperature of the 3D chips with higher layers (28 layers) from the modeling analysis data of 3D chips with lower layers (4–5 layers). Its prediction deviation is less than 0.2%. The predicted data match the simulated numerical data quite well, indicating that the predictive algorithm can accurately feed the sample data set.

A new coal seam seepage parameter prediction method based on gas pressure recovery curve for adjacent aquifer environment

Gas extraction is the primary approach for safe and efficient coal mining, its final effect depending on the coal seam seepage parameters (coal seam gas pressure and permeability coefficient), which generally reflect the possibility and intensity of gas protrusion. However, this method is limited by the high requirement for measurement of conditions, long duration, and large errors under complicated conditions (e.g., adjacent aquifers). Therefore, this study proposed a new method to accurately obtain the coal seam gas pressure in an adjacent aquifer and introduced an advanced tool for the rapid measurement of coal seepage parameters, which were verified by engineering. Results of the mercury intrusion porosimetry demonstrated that the effective pore percentage of the coal samples was approximately 32.67%, inducing a locally high gas pressure prone zone in the test area. The multi-physical parameter test showed substantial differences in the coal sample properties, pressure distribution, and seepage parameters in the eastern and western regions of the test site. The field engineering verification results of the prediction of gas pressure, showed that the pressure recovery curve method differed from the conventional method by 4–9.6%, but the time required was only 10–50% of the latter. In the prediction of the permeability coefficient, the pressure recovery curve method had an error between 24.63 and 43.32%, and this difference was consistent with the current method when classifying the extraction difficulty. These findings are significant for the determination of gas seepage parameters at coal mines.

Accident parameter prediction method for lead-bismuth cooled reactor based on a multivariate LSTM network coupled with an optimization algorithm

Accurate acquisition of key parameters of lead–bismuth cooled reactors under accident conditions is a prerequisite for reactor safety analyses. In this work, four optimization algorithms(Particle Swarm Algorithm, Genetic algorithm, Quantum genetic algorithm, Whale optimization algorithm) are used to improve the forecast efficiency of a long short-term memory (LSTM) neural network using hyperparameter optimization, and a method is developed to predict the parameters of a lead–bismuth cooled reactor (MARS-3) during a loss of flow accident. A comprehensive evaluation of the proposed method is performed using the TOPSIS technique based on the data samples generated using the sub-channel code SUBCHANFLOW. The results show that the prediction performance of the multivariate LSTM neural network coupled with the particle swarm optimization method is optimal, and its computational efficiency is 438 times that of SUBCHANFLOW. Overall, the study findings can help improve the prediction efficiency for key thermal parameters of lead–bismuth cooled reactors and enhance the emergency response capability of such reactors.

Deriving forest stand information from small sample plots: An evaluation of statistical methods

Most strategic and operational forest management decisions are taken based on stand-level information, and quantitative models of forest dynamics are key for developing sustainable management strategies. However, data on forest stands for the initialisation of such models that are representative at large spatial scales, e.g., countries or ecoregions, are often lacking. National Forest Inventories (NFIs) provide forest data from small sample plots at large spatial scales, yet deriving full stand information based on such data is challenging. Here, we evaluate seven methods of varying complexity for deriving quantitative stand descriptions based on sample data as provided by the Swiss NFI. We selected 271 extensively measured Swiss forests stands with unimodal diameter distributions, classified them as beech- vs. spruce-dominated in five development stages and randomly placed a small sized sample plot in each stand using the Swiss NFI sampling design (i.e., a circular plot of 500 m2). Seven modelling approaches were used to derive diameter distributions and species-specific stem numbers (i.e., tree species composition) from the sample data that are representative for a particular stand (local scale) and for stand types in general (generalised scale). The prediction performance of the modelling approaches was evaluated using 100 random samples per stand to calculate prediction errors. Generalised even-aged diameter distributions were best predicted by the simultaneous parameter prediction method (PPM), i.e. a combined three-step regression approach, with on average 1.3 to 2.5 times lower prediction errors compared to the simple pooling of diameter samples. However, uneven-aged diameter distributions were best predicted by pooling. At the local scale, the simultaneous PPM performed best for data from sample plots with fewer than 17 to 19 trees across all development stages. Prediction performance of the PPMs increased for structurally and spatially diverse local stands with positively skewed diameter distributions. A Random Forest approach was most suitable for predicting species composition at both the generalised and the local scale. Our study evaluates the strengths and weaknesses of methods to model stands based on data from small sample plots. We emphasise terminological pitfalls by consequently distinguishing local accuracy and generalised representativity of the stand descriptions. We demonstrate the feasibility of deriving locally accurate stands using data from small forest sample plots and evaluate the derivation of generalised stands representative at large regions. At both scales, our developments contribute to an improved initialisation of forest models and thus to a more realistic modelling of forest development under future boundary conditions.

Neural computing for grey Richards differential equation to forecast traffic parameters with various time granularity

The existing traffic parameter prediction methods generally adopt a single prediction model, but the fusion of different theories and methods can complement each other and improve the prediction performance of the model. Starting from the statistical distribution characteristics of traffic flow, this work introduces the Richards equation to conduct grey modeling, which is used to simulate the development trend of traffic parameters; and then fuses the abilities of error feedback adjustment and complex nonlinear fitting of neural network to estimate the parameters of the grey model and forecast the volatility of traffic flow respectively. At the same time, a dynamic prediction framework of real-time data update is built for the new model, and finally establish the dynamic grey Richards neural network model (DGR-NN).Apply the new model to different traffic parameters (Speed; Volume; Jam mileage) and data resolutions (5 min; 15 min; 1 h), the modeling effect of DGR-NN is significantly improved compared to the grey Richards model (GRM), and the training and testing errors of the model in the three forecast scenarios are reduced to varying degrees, where the testing MAPE, RMSE and STD are reduced by 1.80% ∼ 10.87%, 2.55% ∼ 7.26% and 8.08–25.56% respectively. Furthermore, the results of the new model were verified and analyzed with the other five comparison models, among which the boxplot of APE shows that the error distribution of DGR-NN prediction data is concentrated and the value level is relatively low. It can be seen that DGR-NN can accurately and stably forecast traffic parameters of different time granularities.

Parameter Prediction of the Non-Linear Nomoto Model for Different Ship Loading Conditions Using Support Vector Regression

Significant changes in the load of cargo ships make it difficult to simulate and control their motion. In this work, a parameter prediction method for a ship maneuvering motion model is developed based on parameter identification and support vector regression (SVR). First, the effects of least-squares (LS) and multi-innovation least-squares (MILS) parameter identification methods for the non-linear Nomoto model are investigated. The MILS method is then used to identify the parameters of the non-linear Nomoto model under various load conditions, and model training datasets are established. On this basis, SVR is used to predict the parameters of the non-linear Nomoto model. The results reveal that the MILS method converges faster than the LS method. The SVR method achieves lower accuracy than the MILS method, but exhibits reasonable prediction accuracy for zigzag motions, and the maneuvering motion model can be predicted as navigation conditions change.

Comparative study of data-driven and model-driven approaches in prediction of nuclear power plants operating parameters

Nuclear energy plays a crucial role in mitigating climate change as a near carbon-free and clean energy source. Predicting operating parameters is key to the digitalization and intellectualization of nuclear power plants, improving energy efficiency and reducing costs. Parameter prediction methods mainly consist of model-driven and data-driven approaches, and a comparative study is necessary to select the appropriate prediction method or combine them. In this paper, the gated recurrent unit network and the thermal–hydraulic program RELAP5 are chosen as representative data-driven and model-driven approaches to be used for predicting parameters for a specific operating condition to assess the characteristics and capabilities of each. The chosen experiment is a steam generator tube rupture accident which is important to pressurized water reactors. In this paper the modelling process and the prediction results for both approaches are given. The correlation coefficient for dome pressure, dome temperature, and primary outlet temperature of both approaches is greater than 0.993, except that the correlation coefficient of downcomer temperature is 0.83865 using RELAP5 and 0.99392 using gated recurrent unit network. Then the two approaches are compared from six perspectives of practical application, including accuracy (gated recurrent unit network and RELAP5 received scores of 8 and 6, respectively.), data requirements (7 and 5, respectively), model building algorithm (7 and 5, respectively), prediction speed (10 and 6, respectively), expertise requirements (6 and 6, respectively), and interpretability (5 and 9, respectively). It shows that model-driven and data-driven approaches both require specialist knowledge and appropriate data for modelling process. The data-driven gated recurrent unit network exhibits better accuracy and higher speed. The model-driven RELAP5 has better interpretability compared to the black-box gated recurrent unit network. It is suggested that hybrid approaches of model-driven and data-driven approaches could better deal with the prediction problems.

Review and Evaluation of the Prediction Methods for Voids in the Mineral Aggregate in Asphalt Mixtures

The percentage of voids in mineral aggregates (VMA) is an important consideration in the design of asphalt mixtures. Accurate and precise prediction of VMA is critical in pavement engineering. This study reviewed and evaluated current VMA prediction methods. From the perspective of treating aggregate gradation, the methods were classified into four categories: the overall gradation prediction method (OGPM), gradual gradation prediction method (GGPM), indirect parameter prediction method (IPPM), and theoretical derivation method (TDM). Fifty-four groups of published test data from previous studies were used to evaluate the prediction results of the models, and the precision and accuracy of the prediction results were compared. From the perspective of the completeness of the modeling parameters and the theoretical foundation, the reasons for the differences in prediction results with different methods were analyzed. It was concluded that the TMD has a solid theoretical basis, especially the Yu method in TMD exhibited the best prediction precision and accuracy. Some modifications to the current methods were suggested, such as incorporating other factors affecting VMA and optimizing the particle shape indicators.

Pre-Layout Parasitic-Aware Design Optimizing for RF Circuits Using Graph Neural Network

The performance of analog and RF circuits is widely affected by the interconnection parasitic in the circuit. With the progress of technology, interconnection parasitics plays a larger role in performance deterioration. To solve this problem, designers must repeat layout design and validation process. In order to achieve an upgrade in the design efficiency, in this paper, a Graph Neural Network (GNN)-based pre-layout parasitic parameter prediction method is proposed and applied to the design optimization of a 28 nm PLL. With the new method adopted, the frequency band overlap rate of the VCO is improved by 2.3 percents for an equal design effort. Similarly, the optimized CP is superior to the traditional method with a 15 ps mismatch time. These improvements are achieved under the premise of greatly saving the optimization iteration and verification costs.

ОТДЕЛЬНЫЕ ВОПРОСЫ ДИНАМИЧЕСКОЙ ИНВЕРСИИ ВОЛНОВОГО ПОЛЯ

Currently, reservoir parameter prediction methods based on the amplitude wave field inversion play an important role in solving geological problems in the joint interpretation of seismic and well data. From a formal point of view, well data are used to obtain additional conditions for solving inverse problems of seismic. But the formal description of solution is conditional, there is rather an adjustment of results obtained independently from seismic and well data. The paper considers the following issues: stability of the simultaneous inversion parameter estimation; calculation features of background model and coordination of background and seismic solution components; compliance with restrictions on the maximum sourcereceiver distances by kinematic muting and the range of reflection angles during AVO analysis. The AVO effect is analyzed, it displays itself in a change of the reflection coefficient sign and, correspondently, phase inversion of reflected wave.

Brain-Inspired Spike Echo State Network Dynamics for Aero-Engine Intelligent Fault Prediction

Aero-engine fault prediction aims to accurately predict the development trend of the future state of aero-engines, so as to diagnose faults in advance. Traditional aero-engine parameter prediction methods mainly use the nonlinear mapping relationship of time series data but generally ignore the adequate spatio-temporal features contained in aero-engine data. To this end, we propose a brain-inspired spike echo state network (Spike-ESN) model for aero-engine intelligent fault prediction, which is used to effectively capture the evolution process of aero-engine time series data in the framework of spatio-temporal dynamics. In the proposed approach, we design a spike input layer based on Poisson distribution inspired by the spike neural encoding mechanism of biological neurons, which can extract the useful temporal characteristics in aero-engine sequence data. Then, the temporal characteristics are input into a spike reservoir through the current calculation method of spike accumulation in neurons, which projects the data into a high-dimensional sparse space. In addition, we use the ridge regression method to read out the internal state of the spike reservoir. Finally, the experimental results of aero-engine states prediction demonstrate the superiority and potential of the proposed method.

Method of Geomechanical Parameter Determination and Volumetric Fracturing Factor Simulation under Highly Stochastic Geologic Conditions

In order to accurately predict geomechanical parameters of oil-bearing reservoirs and influencing factors of volumetric fracturing, a new method of geomechanical parameter prediction combining seismic inversion, well logging interpretation and production data is proposed in this paper. Herein, we present a structure model, petrophysical model and geomechanical model. Moreover, a three-dimensional geomechanical model of a typical reservoir was established and corrected using history matching. On this basis, a typical well model was established, 11 influencing factors of volume fracturing including formation parameters and fracturing parameters were analyzed and their impact were ranked, and the oil recovery rate and the accumulated oil production before and after optimal fracturing were compared. The results show that with respect to formation parameters, reservoir thickness is the main influencing factor; interlayer thickness and stress difference are the secondary influencing factors; and formation permeability, Young’s modulus and Poisson’s ratio are the weak influencing factors. For a pilot well of a typical reservoir, the optimized fracture increased production by 7 tons/day relative to traditional fracturing. After one year of production, the method increased production by 4 tons/day relative to traditional fracturing, showing great potential in similar oil reservoirs.

Shale gas geological “sweet spot” parameter prediction method and its application based on convolutional neural network

Parameters such as gas content (GAS), porosity (PHI) and total organic carbon (TOC) are key parameters that reveal the shale gas geological “sweet spot” of reservoirs. However, the lack of a three-dimensional high-precision prediction method is not conducive to large-scale exploration of shale gas. Although the parameter prediction accuracy based on well logging data is relatively high, it is only a single point longitudinal feature. On the basis of prestack inversion of reservoir information such as P-wave velocity and density, high-precision and large-scale “sweet spot” spatial distribution predictions can be realized. Based on the fast growing and widely used deep learning methods, a one-dimensional convolutional neural network (1D-CNN) “sweet spot” parameter prediction method is proposed in this paper. First, intersection analysis is carried out for various well logging information to determine the sensitive parameters of geological “sweet spot”. We propose a new standardized preprocessing method based on the characteristics of the well logging data. Then, a 1D-CNN framework is designed, which can meet the parameter prediction of both depth-domain well logging data and time-domain seismic data. Third, well logging data is used to train a high-precision and robust geological “sweet spot” prediction model. Finally, this method was applied to the WeiRong shale gas field in Sichuan Basin to achieve a high-precision prediction of geological “sweet spots” in the Wufeng–Longmaxi shale reservoir.