Pure Neural Network Model Research Articles

Interpretable Framework of Physics‐Guided Neural Network With Attention Mechanism: Simulating Paddy Field Water Temperature Variations

AbstractWith the development of large‐scale rice cultivation management initiatives in East Asia, there is concern that a reduction in the number of human cultivators per unit area may lead to poor water management, which could result in decreased land productivity, owing to abnormally high‐ and low‐temperature damage to crops. Accurate simulation of paddy field water temperature is important for studying its impact on crops and providing timely information to aid in decision‐making for more efficient management under limited resources. We propose a neural‐network framework that considers the heat transfer by the vegetation canopy and applies physical‐theory constraints in its training. A novel tuning method is proposed to cope with the trade‐off between water temperature accuracy and physical consistency during training to ensure that the calculated water temperature variations in a paddy field enjoy high accuracy and physical consistency. In the experiments, the proposed framework outperforms physical process models and pure neural network models while maintaining high accuracy in the case of sparse data sets. Furthermore, an attention‐mechanism input layer is integrated into the model to rank feature importance, providing global interpretation to the proposed framework. We also perform sensitivity analysis on the physical process and propose models to compare their different strategies of feature ranking. The results show that the two methods have different sensitivities to different feature patterns, but they complement each other. In summary, the proposed model is credible and stable for practical applications and has the potential to guide more efficient paddy management.

Accurate Storm Surge Prediction with a Parametric Cyclone and Neural Network Hybrid Model

Storm surges are one of the most devastating coastal disasters. Numerous efforts have continuously been made to achieve better prediction of storm surge variation. In this paper, we propose a parametric cyclone and neural network hybrid model for accurate, long lead-time storm surge prediction. The model was applied to the northeastern coastal region of Taiwan, i.e., Longdong station. A total of 14 historical typhoon events were used for model training and validation, and the results and questions associated with this hybrid approach carefully discussed. Overall, the proposed method reduced the complexity of network structure while retaining the important typhoon indicators. In particular, local pressure and winds estimated from the storm parameters through physically-based parametric cyclone models allow for inferring the possible future influence of a typhoon, unlike the simple collection and direct usage of observation data from local stations in earlier works. Meanwhile, the error-tolerance capability of the neural network alleviated some discrepancy in the model inputs and enabled good surge prediction. Further, the proposed method showed better and faster convergence thanks to the retention of storm information and the reduced dimensions of the search space. The hybrid model presented excellent performance or maintained reasonable capability for short lead-time and long lead-time storm surge prediction. Compared with the pure neural network model under the same network dimensions, the present model demonstrated great improvement in accuracy as the prediction lead time increased to 8 h, e.g., 33–40% (13–21%) and 32–37% (18–29%) RMSE and CE, respectively, in the training/validation phase.

Physics-Embedded Machine Learning: Case Study with Electrochemical Micro-Machining

Although intelligent machine learning techniques have been used for input-output modeling of many different manufacturing processes, these techniques map directly from the input process parameters to the outputs and do not take into consideration any partial knowledge available about the mechanisms and physics of the process. In this paper, a new approach is presented for taking advantage of the partial knowledge available about the mechanisms of the process and embedding it into the neural network structure. To validate the proposed approach, it is used to create a forward prediction model for the process of electrochemical micro-machining (μ-ECM). The prediction accuracy of the proposed approach is compared to the prediction accuracy of pure neural structure models with different structures and the results show that the Neural Network (NN) models with embedded knowledge have better prediction accuracy over pure NN models.

Automated Knowledge-Based Neural Network Modeling for Microwave Applications

Automated model generation (AMG) method is extended from generating pure neural network (NN) models to generating knowledge-based NN models. Knowledge-based models have been demonstrated in the existing literature to use less data over pure NN models while maintaining good accuracy. The proposed method automates data generation, determination of data distribution, model structure adaptation, and model training in a systematic framework. It can further reduce the number of training data through the adaptive sampling process, shorten the model development time over existing AMG methods and existing knowledge-based modeling methods, and ensure the accuracy of the final model at the same time. The algorithm is demonstrated through microwave modeling examples.

Relevance vector-machine-based solar cell model

This paper proposes an advanced machine learning method, relevance vector machines (RVMs), to model photovoltaic (PV) cells with a few measured data, over a range of expected operating conditions. RVMs are established on a Bayesian formulation which results in usage of less number of relevance vectors leading to much more sparse representation than the support vector machine. The RVM model can be used to predict short-circuit current and open-circuit voltage and thereby maximum power point for any unknown temperature and irradiation. Coordinate translation technique is used to plot the nonlinear I–V characteristics of PV cells. The proposed method matches the measured data more accurately than the pure neural network model and the neuro-fuzzy model.

Neuro-Fuzzy-Based Solar Cell Model

This work describes a hybrid soft-computing modeling technique that facilitates the modeling of newly installed solar cells, or solar cells with few historical measured data, over a range of expected operating conditions. The technique uses neuro-fuzzy models to predict solar cell short-circuit current and open-circuit voltage, followed by coordinate translation of a measured current-voltage response. The model can be extended beyond the bounds of measured data by incorporating a priori knowledge derived from theory and manufacturer's data. The solar cell model is developed and validated against measured data. The model requires fewer data than pure neural network models, and matches measured data more accurately than conventional solar cell models.

Using hybrid neural networks in scaling up an FCC model from a pilot plant to an industrial unit

The scaling up of a pilot plant fluid catalytic cracking (FCC) model to an industrial unit with use of artificial neural networks is presented in this paper. FCC is one of the most important oil refinery processes. Due to its complexity the modeling of the FCC poses great challenge. The pilot plant model is capable of predicting the weight percent of conversion and coke yield of an FCC unit. This work is focused in determining the optimum hybrid approach, in order to improve the accuracy of the pilot plant model. Industrial data from a Greek petroleum refinery were used to develop and validate the models. The hybrid models developed are compared with the pilot plant model and a pure neural network model. The results show that the hybrid approach is able to increase the accuracy of prediction especially with data that is out of the model range. Furthermore, the hybrid models are easier to interpret and analyze.

Modeling chemical processes using prior knowledge and neural networks

AbstractWe present a method for synthesizing chemical process models that combines prior knowledge and artificial neural networks. The inclusion of prior knowledge is investigated as a means of improving the neural network predictions when trained on sparse and noisy process data. Prior knowledge enters the hybrid model as a simple process model and first principle equations. The simple model controls the extrapolation of the hybrid in the regions of input space that lack training data. The first principle equations, such as mass and component balances, enforce equality constraints. The neural network compensates for inaccuracy in the prior model. In addition, inequality constraints are imposed during parameter estimation. For illustration, the approach is applied in predicting cell biomass and secondary metabolite in a fed‐batch penicillin fermentation. Our results show that prior knowledge enhances the generalization capabilities of a pure neural network model. The approach is shown to require less data for parameter estimation, produce more accurate and consistent predictions, and provide more reliable extrapolation.