Training Data Points Research Articles

Prediction of building power consumption using transfer learning-based reference building and simulation dataset

With the advancements in data processing technologies and the increased use of renewable energy systems, the development of microgrid has gained attention. Consequently, a method for machine learning studies has increased for rational building power consumption. Extensive historical data of building power consumption are required for a high accuracy prediction; however, in practice, it is difficult to gather such data from existing or new buildings. Therefore, this study proposed a method for using transfer learning based on the simulation dataset of a reference building. The transfer learning long short-term memory (TL-LSTM) model developed in this study trained only on 24 h of office building power consumption data and predicted after 24 h. The accuracy of TL-LSTM was evaluated using various simulation and experimental data, and the factors affecting the performance of TL-LSTM (the number of training data and climate zone) were analyzed. Consequently, compared to the long short-term memory (LSTM) model, the TL-LSTM model demonstrated a higher accuracy with an average coefficient of variation of the root mean square error (CVRMSE) of 4.25% and mean bias error (MBE) of 1.70%. Furthermore, the prediction for the next 24 h was possible when at least 22 training data points were gathered. Finally, when the climate zone was the same for the target and source datasets, high accuracy was demonstrated even if the location of each building was different. Additionally, the source dataset could be replaced with a simulation dataset.

New general models for condensation heat transfer coefficient of carbon dioxide in smooth tubes by intelligent and least square fitting approaches

The heat transfer characteristics of carbon dioxide (R744) as a natural and safe refrigerant are quite different from the other refrigerants due to the CO2 unique physical properties. So, it is essential to develop accurate general models for predicting the heat transfer coefficient (HTC) of CO2 condensation inside smooth tubes. In this study, an extensive experimental databank for the CO2 refrigerant including 1,016 data samples was gathered from 13 published studies, covering a broad range of operating conditions and channel sizes. Ten earlier well-known HTC models for other refrigerants showed high deviations (average absolute relative deviations (AARDs) higher than 49%). To overcome this research gap in the literature, the machine learning methods, namely, multilayer perceptron (MLP), interpolating radial basis function (RBF), and Gaussian process regression (GPR), were used for developing the predictive models for two-phase Nusselt number. The GPR model showed the best outcomes among the studied models with AARD values of 1.5% and 7.32% for training and testing data points, respectively. In addition, a new general HTC correlation for condensation of CO2 was proposed by considering two surface tensions ranges, which showed good accuracy with an AARD of 17.61% for the entire database. It was found that the developed models are reliable for designing heat exchangers with different channel sizes and various flow patterns. Finally, a sensitivity analysis using the most accurate GPR model showed that the channel diameter is the most critical factor in controlling the HTC of CO2 when the Bond number is higher than 0.5.

Application of Machine Learning in Battery: State of Charge Estimation Using Feed Forward Neural Network for Na-Ion Batteries

Estimating accurate State-of-Charge (SOC) of a battery is important to avoid the over/under charging and protect the battery pack from low cycle life. Current methods of SOC estimation use complex equation in Extended Kalman Filter (EKF) and equivalent circuit model. In this paper, we used a Feed Forward Neural Network (FNN) to estimate SOC value accurately where battery parameters such current, voltage and charge are mapped directly to SOC value at the output. FNN has the ability to self-learn the weights with each training data points and update the model parameters like weights and bias using a combination of two gradient descent (Adam). This model comprises Dropout technique, which can have many neural network architectures by dropping the neuron/mode at each epoch/training cycle using the same weights and bias. Our FNN model was trained with data comprising different current rates and tested for different cycling data for example 5th, 10th, 20th, 50th cycle and at different cut-off voltage (4.5 V). The battery used for estimating the SOC value is Na-ion, which is highly non-linear and it was fabricated in house using Na0.67Fe0.5Mn0.5O2 (NFM) and Na metal. FNN successfully estimated the SOC value for highly non-linear nature of Na-ion battery at different current rates- 0.05C, 0.1C, 0.5C, 1C, 2C, having the R2 value of ~0.97-0.99 and ~0.99 for different cycling data and at higher cut-off voltage of -4.5 VNa+.

Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equations

We present a neural network-based method for solving linear and nonlinear partial differential equations, by combining the ideas of extreme learning machines (ELM), domain decomposition and local neural networks. The field solution on each sub-domain is represented by a local feed-forward neural network, and Ck continuity conditions are imposed on the sub-domain boundaries. Each local neural network consists of a small number of hidden layers, while its last hidden layer can be wide. The weight/bias coefficients in all the hidden layers of the local neural networks are pre-set to random values and fixed throughout the computation, and only the weight coefficients in the output layers of the local neural networks are training parameters. The overall neural network is trained by a linear or nonlinear least squares computation, not by the back-propagation type algorithms. We introduce a block time-marching scheme together with the presented method for long-time simulations of time-dependent linear/nonlinear partial differential equations. The current method exhibits a clear sense of convergence with respect to the degrees of freedom in the neural network. Its numerical errors typically decrease exponentially or nearly exponentially as the number of training parameters, or the number of training data points, or the number of sub-domains increases. Extensive numerical experiments have been performed to demonstrate the computational performance of the presented method. We also demonstrate its capability for long-time dynamic simulations with some test problems. We compare the presented method with the deep Galerkin method (DGM) and the physics-informed neural network (PINN) method in terms of the accuracy and computational cost. The current method exhibits a clear superiority, with its numerical errors and network training time considerably smaller (typically by orders of magnitude) than those of DGM and PINN. We also compare the current method with the classical finite element method (FEM). The computational performance of the current method is on par with, and often exceeds, the FEM performance in terms of the accuracy and computational cost.

Imitation Learning System Design with Small Training Data for Flexible Tool Manipulation

Population aging has become a major problem in developed countries. As the labor force declines, robot arms are expected to replace human labor for simple tasks. A robotic arm attaches a tool specialized for a task and acquires the movement through teaching by an engineer with expert knowledge. However, the number of such engineers is limited; therefore, a teaching method that can be used by non-technical personnel is necessitated. As a teaching method, deep learning can be used to imitate human behavior and tool usage. However, deep learning requires a large amount of training data for learning. In this study, the target task of the robot is to sweep multiple pieces of dirt using a broom. The proposed learning system can estimate the initial parameters for deep learning based on experience, as well as the shape and physical properties of the tools. It can reduce the number of training data points when learning a new tool. A virtual reality system is used to move the robot arm easily and safely, as well as to create training data for imitation. In this study, cleaning experiments are conducted to evaluate the effectiveness of the proposed method. The experimental results confirm that the proposed method can accelerate the learning speed of deep learning and acquire cleaning ability using a small amount of training data.

Output Temperature Predictions of the Geothermal Heat Pump System Using an Improved Grey Prediction Model

This paper presents the Improved Grey Prediction Model, also called IGM (1,1) model, to increase the prediction accuracy of the Grey Prediction Model (GM) model that performs the GHPS output temperature prediction. This was based on correcting the current predicted value by subtracting the error between the previous predicted value and the previous immediate mean of the measured value. Subsequently, the IGM (1,1) model was applied to predict the output temperature of the GHPSs at Oklahoma University, the University Politècnica de València, and Oakland University, respectively. For each GHPS, the model uses a small dataset of 24 data points (i.e., 24 h) for training to predict the output temperature eight hours in advance. The proposed model was verified using three different output temperature datasets; these datasets were also used to validate the power efficiency of the proposed model. In addition, the empirical results show that the proposed IGM (1,1) model significantly improves the simulation (in-sample) and the prediction (out-of-sample) of the output temperature of the GHPS through error reduction, thereby enhancing the GM (1,1) model’s overall accuracy. As a result, the prediction accuracies were compared, and the improved model was found to be more accurate than the GM (1,1) model in both simulation and prediction results for all datasets used.

Modelling study on phase equilibria behavior of ionic liquid-based aqueous biphasic systems

The ability to predict the phase equilibria behavior is of crucial relevance in the early design stage of biphasic liquid-liquid systems. Ionic liquid-based aqueous biphasic systems (IL-ABS) have demonstrated superior performance in many applications such as the recovery of bio-products and the recycling of hydrophilic ILs from aqueous solutions. In order to better utilize these novel biphasic liquid-liquid systems, modelling studies on phase equilibria behavior are carried out in this work. First, the IL database developed in our previous work is extended to these unconventional biphasic systems. In total, 17,449 experimental binodal data points covering 171 IL-ABS at different temperatures (278.15 K-343.15 K) are collected. Then, all involved IL-ABS are correlated using a popular three-parameter mathematical description and the optimal parameters of each IL-ABS are obtained. Afterwards, we try to build a linear group contribution (GC) model to predict the phase equilibria behavior of IL-ABS, but it fails due to the high complexity of these biphasic systems. For this reason, we finally turn to applying a well-known machine learning algorithm, i.e., artificial neural network (ANN), to build a nonlinear GC model for such a purpose. This model gives a mean absolute error (MAE) of 0.0175 and squared correlation coefficient (R2) of 0.9316 for the 13,789 training data points, and for the 3,660 test data points they are 0.0177 and 0.9195, respectively. The results indicate that the proposed nonlinear ANN-GC model, to some extent, is capable to predict the phase equilibria behavior of IL-ABS. Besides the efforts of building GC models, we also discuss some main issues that govern the phase equilibria behavior of IL- ABS, which could be a guidance in the design of IL-ABS.

Application of ANN for prediction of chloride penetration resistance and concrete compressive strength

Self-consolidating concrete is considered as one of the greatest developments in concrete industry. It has several advantages over conventional concrete such as minimized voids, ease of pour into congested formworks and complex shapes and absence of mechanical vibrations. One of the major problems with concrete structures is the corrosion of steel reinforcement due to penetration of chemicals like chloride. Most of the standard tests to determine chloride penetration levels are performed at 28-days. It is desired that some rapid means of determination of chloride ion diffusion in concrete are available for design and quality assurance purposes. In present study Artificial Neural Networks (ANNs) are developed for prediction of chloride penetration level and compressive strength of SCC mixes. ANN developed for prediction of chloride penetration used 294 data points for training and validation. This trained ANN was used to verify chloride penetration level of twenty experimentally evaluated SCC specimens of varied composition. Similarly, ANN developed for prediction of compressive strength used 1031 data points for training and validation of network. Compressive strength values predicted by ANN were compared with experimentally evaluated compressive strengths of thirty-three SCC specimens of varied composition. Both ANNs use relevant input parameters for training and validation of networks. A comparison between experimentally evaluated and ANN predicted values is presented in the paper for different combinations of learning rate and momentum of network. The trained ANNs were able to predict chloride iron penetration levels and compressive strength based on selected input parameters to an acceptable extent.

An automated approach for developing neural network interatomic potentials with FLAME

The performance of machine learning interatomic potentials relies on the quality of the training dataset. In this work, we present an approach for generating diverse and representative training data points which initiates with ab initio calculations for bulk structures. The data generation and potential construction further proceed side-by-side in a cyclic process of training the neural network and crystal structure prediction based on the developed interatomic potentials. All steps of the data generation and potential development are performed with minimal human intervention. We show the reliability of our approach by assessing the performance of neural network potentials developed for two inorganic systems.

Alternating Conditional Expectation (ACE) Algorithm for Permeability Estimation in Bahariya Formation

Summary The nonparametric transformation is a data-driven technique, which can be used to estimate optimal correlations between a dependent variable (response) and a set of independent parameters (predictors). This study introduces a systematic methodology using the nonparametric transformation concept and the alternating conditional expectation (ACE) algorithm to estimate the effective gas permeability using conventional logs and the core data. The ACE algorithm was employed in the current work using the MATLAB® (The MathWorks, Inc., Natick, Massachusetts, USA) code and the open-source GRaphical ACE (GRACE) software (Xue et al. 1997) for deriving the optimal nonparametric correlations for predicting the permeability. The methodology was applied to a heterogeneous formation [Bahariya (BAH)] in Egypt to understand its characteristics and predict its permeability more accurately. The BAH Formation is considered one of the main sources for oil production throughout the Western Desert (WD) of Egypt. The cumulative oil production from the BAH Formation is estimated to be approximately 40% of the total WD production. The reservoir characteristics of the BAH Formation range from highly permeable to tight sandstone interbedded with shale and siltstone. It usually depicts low-resistivity and low-contrast (LRLC) log behavior. Thus, regional and accurate determination of the reservoir permeability for the different rock units of the BAH Formation across the WD is a challenge. Conventional well log data from approximately 100 cored wells and corresponding 5,500 core measurements were used to provide a regional permeability correlation that can be used in a large number of reservoirs. The methodology of this work included two main steps: Applying the nonparametric transformation technique to identify the collective log responses for deriving optimal correlation Predicting the permeability profiles using the selected log responses The model was applied to many wells that address different petrophysical characteristics of the BAH Formation. The established permeability profiles showed reliable correlation coefficients relative to the measured core data. The correlation coefficient was 0.893 for the training data points (75% of the collected database) and 0.913 for the testing data points (25% of the collected database). In addition, the mean absolute percentage error (MAPE) between the predicted and the measured permeability for the training and testing data points were 5.93 and 4.14%, respectively. Permeability prediction using ACE is compared with other techniques such as k-ϕ crossplots, multiple linear regression (MLR), Coates, and Wyllie-Rose correlations. This work is considered an original contribution to present regional permeability prediction correlations using the conventional well logs for reservoir characterization and simulation applications. The ACE algorithm was successfully applied to the BAH Formation and proved its capability to identify the best predictors that are required to establish a rigorous model.

The estimation of the second virial coefficients of He and N2 based on neural network potentials with quantum mechanical calculations

• We test the accuracy of the neural network potentials against the virial coefficient of He and N2. • The second virial coefficients from the neural network potential is quite accurate for He at T > 200 K. • This study indicates that the level of quantum chemical calculation still play an important role. We test the accuracy of the neural network interaction potentials against accurate thermodynamic data of He and N 2 . We perform single point energy calculations at CCSD(T) level, construct the intermolecular interactions by using neural networks, and perform Monte Carlo simulations to calculate virial coefficients. We find that the deviation of virial coefficients from previous studies is small for He while the difference is about 3 cm 3 mol −1 for N 2 . Our results show that when trying to obtain accurate thermodynamic data from the neural network interaction potentials, not only the level of quantum chemical calculations should be an important factor, but also at least thousands of training data points would be required.

Searching Configurations in Uncertainty Space: Active Learning of High-Dimensional Neural Network Reactive Potentials.

Neural network (NN) potential energy surfaces (PESs) have been widely used in atomistic simulations with ab initio accuracy. While constructing NN PESs, their training data points are often sampled by molecular dynamics trajectories. This strategy can be however inefficient for reactive systems involving rare events. Here, we develop an uncertainty-driven active learning strategy to automatically and efficiently generate high-dimensional NN-based reactive potentials, taking a gas-surface reaction as an example. The difference between two independent NN models is used as a simple and differentiable uncertainty metric, allowing us to quickly search in the uncertainty space and place new samples at which the PES is less reliable. By interfacing this algorithm with the first-principles simulation package, we demonstrate that a globally accurate NN potential of the H2 + Ag(111) system can be constructed with merely ∼150 data points. This PES can be further refined to describe H2 dissociation on Ag(100) by adding ∼130 more configurations on this facet. The entire process is completely automatic and self-terminated once the relative error criterion is fulfilled. Impressively, data points sampled by this uncertainty-driven strategy are substantially fewer than by the traditional trajectory-based sampling. The final NN PES not only converges well the quantum dissociation probability of the molecule but also well-reproduces the phonon properties of the substrate and is capable of describing surface temperature effects. These results show the potential of this active learning approach in developing high-dimensional NN reactive potentials in gas and condensed phases.

Passive Indoor Visible Light Positioning System Using Deep Learning

A passive indoor visible light positioning system is proposed that does not require active participation from the user and is suitable for IoT sensor networks. This approach does not require a line-of-sight path and measures the impulse response (IR) between sources and receivers installed in the room. The presence of an object of interest (OI), i.e., a person to be localized, disrupts the IRs among the source–receiver pairs, which can be related to its position. A deep learning framework is developed that learns the relationship between changes in sets of IRs and the OI position through a set of training data obtained by placing the OI at random locations in the room. This approach shows that the OI can be localized using a very limited set of training data under a wide range of illumination levels. In order to represent a realistic scenario, a room with furniture is modeled in the optical system design software. The ray trace information of the modeled room is used to construct IR measurements among different source–receiver pairs that include multiorder reflections. The results show that localization performance is crucially related to the signal-to-noise ratio and the number of training data points used in the learning process. A root-mean-square error (RMSE) near 30 cm is possible in the case of high SNR and a large training set. However, even with a very limited training set and over a range of dimming levels, RMSEs of near 80 cm were obtained without the need for explicit user involvement.

Improved grey system models for predicting traffic parameters

In transportation applications such as real-time route guidance, ramp metering, congestion pricing and special events traffic management, accurate short-term traffic flow prediction is needed. For this purpose, this paper proposes several novel online Grey system models (GM): GM(1,1|cos(ωt)), GM(1,1|sin(ωt),cos(ωt)), and GM(1,1|e-at,sin(ωt),cos(ωt)). To evaluate the performance of the proposed models, they are compared against a set of benchmark models: GM(1,1) model, Grey Verhulst models with and without Fourier error corrections, linear time series model, and nonlinear time series model. The evaluation is performed using loop detector and probe vehicle data from California, Virginia, and Oregon. Among the benchmark models, the error corrected Grey Verhulst model with Fourier outperformed the GM(1,1) model, linear time series, and non-linear time series models. In turn, the three proposed models, GM(1,1|cos(ωt)), GM(1,1|sin(ωt),cos(ωt)), and GM(1,1|e-at,sin(ωt),cos(ωt)), outperformed the Grey Verhulst model in prediction by at least 65%, 16% and 11%, in terms of Root Mean Squared Error, and by 82%, 58% and 42%, in terms of Mean Absolute Percentage Error, respectively. It is observed that the proposed Grey system models are more adaptive to location (e.g., perform well for all roadway types) and traffic parameters (e.g., speed, travel time, occupancy, and volume), and they do not require as many data points for training (4 observations are found to be sufficient).

Autonomous Learning of New Environments with a Robotic Team Employing Hyper-Spectral Remote Sensing, Comprehensive In-Situ Sensing and Machine Learning.

This paper describes and demonstrates an autonomous robotic team that can rapidly learn the characteristics of environments that it has never seen before. The flexible paradigm is easily scalable to multi-robot, multi-sensor autonomous teams, and it is relevant to satellite calibration/validation and the creation of new remote sensing data products. A case study is described for the rapid characterisation of the aquatic environment, over a period of just a few minutes we acquired thousands of training data points. This training data allowed for our machine learning algorithms to rapidly learn by example and provide wide area maps of the composition of the environment. Along side these larger autonomous robots two smaller robots that can be deployed by a single individual were also deployed (a walking robot and a robotic hover-board), observing significant small scale spatial variability.

Predictions-on-chip: model-based training and automated deployment of machine learning models at runtime

The design of gas turbines is a challenging area of cyber-physical systems where complex model-based simulations across multiple disciplines (e.g., performance, aerothermal) drive the design process. As a result, a continuously increasing amount of data is derived during system design. Finding new insights in such data by exploiting various machine learning (ML) techniques is a promising industrial trend since better predictions based on real data result in substantial product quality improvements and cost reduction. This paper presents a method that generates data from multi-paradigm simulation tools, develops and trains ML models for prediction, and deploys such prediction models into an active control system operating at runtime with limited computational power. We explore the replacement of existing traditional prediction modules with ML counterparts with different architectures. We validate the effectiveness of various ML models in the context of three (real) gas turbine bearings using over 150,000 data points for training, validation, and testing. We introduce code generation techniques for automated deployment of neural network models to industrial off-the-shelf programmable logic controllers.

Application of Artificial Intelligence to Predict Enhanced Oil Recovery Using Silica Nanofluids

The feasibility of approaches to enhanced oil recovery (EOR) in harsh conditions of reservoirs should be evaluated primarily in the laboratory environment to capture possible failures that threaten the performance of an operation, although such experiments are commonly expensive and time-consuming. This work investigated the application of artificial intelligence in allaying such concerns regarding the initial screening of EOR methods. Accordingly, three machine learning algorithms, namely adaptive neuro-fuzzy inference system (ANFIS), multilayer perceptron-artificial neural network (MLP–ANN), and radial basis function-artificial neural network (RBF–ANN), were employed to predict the efficiency of a set of silica nanofluid flooding experiments in carbonate and sandstone core samples. Initially, the optimum structures of the employed models were determined. Then, their performances were compared. The strongest performance was achieved by the ANFIS model, where the results in terms of coefficient of determination and root-mean-square error for training, testing, and entire data points were 0.9954 and 0.3395, 0.9877 and 0.4793, and 0.9939 and 0.3793, respectively. The ANFIS model also has the shortest execution time and the least over-fitting problems, and thus it can be utilized for screening the efficiency of silica-EOR projects.

Deep learning from three-dimensional multiphysics simulation in operational optimization and control of polymer electrolyte membrane fuel cell for maximum power

The maximum achievable power of a polymer electrolyte membrane (PEM) fuel cell under specific operating temperature is important to its application. In this paper, we propose a method that integrates an artificial neural network (ANN) with the genetic algorithm (GA) to predict the performance of a PEM fuel cell and identify its maximum powers and corresponding conditions for operational control purpose. A validated three-dimensional (3D) multiphysics model is employed to generate total 1500 data points for training, testing, and verifying the ANN, which consists of two hidden layers with eight and four neurons on each hidden layer, respectively. After the ANN is properly trained, it is incorporated into the GA for deep learning to identify the maximum power and corresponding operating conditions, which shows that the fuel cell configuration could achieve a maximum power of about 0.78 W/cm2 at 368.8 K. Additionally, the combined ANN-GA method is employed to identify the maximum powers and their operating conditions under eight typical operation temperatures in the range of 323–373 K. The deep-learning results reflect the major physical and electrochemical processes that govern fuel cell performance and are validated against the 3D multiphysics model. The results demonstrate that the combined ANN-GA method is suitable to predicting fuel cell performance and identifying operation parameters for the maximum powers under various temperatures, which is important to practical system design and rapid control in fuel cell applications.

Using machine learning to predict concrete’s strength: learning from small datasets

Despite previous efforts to map the proportioning of a concrete to its strength, a robust knowledge-based model enabling accurate strength predictions is still lacking. As an alternative to physical or chemical-based models, data-driven machine learning methods offer a promising pathway to address this problem. Although machine learning can infer the complex, non-linear, non-additive relationship between concrete mixture proportions and strength, large datasets are needed to robustly train such models. This is a concern as reliable concrete strength data is rather limited, especially for realistic industrial concretes. Here, based on the analysis of a fairly large dataset (>10,000 observations) of measured compressive strengths from industrial concretes, we compare the ability of three selected machine learning algorithms (polynomial regression, artificial neural network, random forest) to reliably predict concrete strength as a function of the size of the training dataset. In addition, by adopting stratified sampling, we investigate the influence of the representativeness of the training datapoints on the learning capability of the models considered herein. Based on these results, we discuss the nature of the competition between how accurate a given model can eventually be (when trained on a large dataset) and how much data is actually required to train this model.

Gaussian Process Modelling of the F-16 Ground Vibration Test Benchmark: Data Selection Case Study

Abstract We observe the effects of training data sample selection in modelling of a physical system with Gaussian process nonlinear autoregressive models with exogenous input. Gaussian process modelling limits the number of training data points and we use a big nonlinear benchmark data set. The combination calls for training data sample selection. We compare a ‘smart’ method based on Euclidean distance between training data points with decimation. We use the training data samples obtained by both methods to train the models, test model predictions on a test data set, and calculate two figures of merit, eRMSt and mean standardised log loss (MSLL). The model trained on the ‘smartly’ selected training data points is better in eRMSt while the one with the decimated data is superior in MSLL. The direct conclusion is that the purpose of the model determines which training data sample selection method is better, as the relevant figure of merit depends on the model purpose. We notice that the predicted variance is more sensitive to the training data sample than the predicted mean. We warn that training data sample selection may have unexpected consequences.