Extrapolation Ability Research Articles

Material Classification for Terahertz Images Based on Neural Networks

Terahertz time-domain spectroscopy enables the extraction of electrical properties from materials. An extraction of the complex permittivity can be carried out with measurements in transmission or reflection geometry enabling the identification of materials. To perform an exact identification, the sample thickness and the angle of incidence of the terahertz radiation must be known. However, when those parameters are unknown and additionally the materials show strong absorbances, a precise differentiation between materials is challenging. A promising approach is the use of a neural network for automated material classification of terahertz images from different materials. Here, we show that a trained neural network can differentiate between 16 3D printed dielectric materials with a high accuracy of 98 % from measurements taken in transmission mode. As the constitution of the dataset has a big impact on the accuracy, various data preparations were investigated as well as the number of traces needed for achieving a well-trained network was determined. Finally, the trained neural network was evaluated with different sample thicknesses, revealing the huge impact of the materials absorbance on the extrapolation ability. This approach can be used in security application to classify harmful substances as well as for the automated generation of material maps.

Comparing data driven and physics inspired models for hopping transport in organic field effect transistors

The past few decades have seen an uptick in the scope and range of device applications of organic semiconductors, such as organic field-effect transistors, organic photovoltaics and light-emitting diodes. Several researchers have studied electrical transport in these materials and proposed physical models to describe charge transport with different material parameters, with most disordered semiconductors exhibiting hopping transport. However, there exists a lack of a consensus among the different models to describe hopping transport accurately and uniformly. In this work, we first evaluate the efficacy of using a purely data-driven approach, i.e., symbolic regression, in unravelling the relationship between the measured field-effect mobility and the controllable inputs of temperature and gate voltage. While the regressor is able to capture the scaled mobility well with mean absolute error (MAE) ~ O(10–2), better than the traditionally used hopping transport model, it is unable to derive physically interpretable input–output relationships. We then examine a physics-inspired renormalization approach to describe the scaled mobility with respect to a scale-invariant reference temperature. We observe that the renormalization approach offers more generality and interpretability with a MAE of the ~ O(10–1), still better than the traditionally used hopping model, but less accurate as compared to the symbolic regression approach. Our work shows that physics-based approaches are powerful compared to purely data-driven modelling, providing an intuitive understanding of data with extrapolative ability.

COVID-19 cough sound symptoms classification from scalogram image representation using deep learning models

Deep Learning shows promising performance in diverse fields and has become an emerging technology in Artificial Intelligence. Recent visual recognition is based on the ranking of photographs and the finding of artefacts in those images. The aim of this research is to classify the different cough sounds of COVID-19 artefacts in the signals of altered real-life environments. The introduced model takes into consideration two major steps. The first step is the transformation phase from sound to image that is optimized by the scalogram technique. The second step involves feature extraction and classification based on six deep transfer models (GoogleNet, ResNet18, ResNet50, ResNet101, MobileNetv2, and NasNetmobile). The dataset used contains 1457 (755 of COVID-19 and 702 of healthy) wave cough sounds. Although our recognition model performs the best, its accuracy only reaches 94.9% based on SGDM optimizer. The accuracy is promising enough for a wide set of labeled cough data to test the potential for generalization. The outcomes show that ResNet18 is the most stable model to classify the cough sounds from a limited dataset with a sensitivity of 94.44% and a specificity of 95.37%. Finally, a comparison of the research with a similar analysis is made. It is observed that the proposed model is more reliable and accurate than any current models. Cough research precision is promising enough to test the ability for extrapolation and generalization.

Machine learning‐based efficient stress intensity factor calculation for aeroengine disk probabilistic risk assessment under polynomial stress fields

AbstractIn probabilistic failure risk assessment, the accuracy and efficiency of the stress intensity factor calculation are important. The universal weight function method has been widely adopted for efficiency, but this method still has some debatable parts. For accurate and efficient stress intensity factor prediction, two approaches for machine learning techniques are specially designed. Three tests are conducted for the first approach where Gaussian process regression, tree‐structure models, and artificial neural network are evaluated and compared for the ability of interpolation and extrapolation. Results show that the artificial neural network and extremely randomized trees perform better. Hybrid models in the second approach are also proposed, and results show that the accuracy of SIF calculation can be improved by 5–35% compared with the weight function. Real aeroengine disks are adopted, and the errors of machine learning methods are less than 20% in the disk life calculation.

Modeling heat capacity of liquid hydrofluorocarbons and hydrofluoroolefins

As for a heat capacity model of molecular liquids, high regression accuracy and reasonable extrapolation ability are two long-term goals. However, very few methods can meet the two requirements, which limits the accuracy of relevant engineering calculations. In this work, a semi-empirical model which can solve this problem was established to obtain the heat capacity (cp) of hydrofluorocarbons (HFCs) and hydrofluoroolefins (HFOs). Here, heat capacities of HFCs and HFOs in the whole liquid region can be obtained if the critical parameters (Tc, pc), acentric factor, ideal gas isochoric heat capacity and at least two different experimental cp data points are known. The proposed model was utilized to calculate the liquid isobaric heat capacity of 12 fluids (10 HFCs and 2 HFOs) and the average absolute deviation (AAD) of the present method calculations from experimental data is 1.01%. Furthermore, heat capacities calculated by the proposed model were compared with the calculations from other models. The results indicate that the present fitting method is superior to other models for representing heat capacities, with regard to the accuracy. Additionally, 36 data points of experimental liquid heat capacity of trans-1,3,3,3-Tetrafluoropropene (also known as HFO-1234ze(E)) in high pressure region were reported in this work and were used to check the present model’s extrapolation ability. High regression accuracy and good extrapolation performance make the proposed method a valuable tool to represent liquid heat capacity of hydrofluorocarbons and hydrofluoroolefins.

Dynamic changes in moisture content and applicability analysis of a typical litter prediction model in Yunnan Province.

BackgroundForest fire risk predictions are based on the most conservation daily predictions, and the lowest litter moisture content of each day is often used to predict the day’s fire risk. Yunnan Province is the area with the most frequent and serious forest fires in China, but there is almost no research on the dynamic changes and model predictions of the litter moisture content in this area. Therefore, to reduce the occurrence of forest fires and improve the accuracy of forest fire risk predictions, it is necessary to understand these dynamic changes and establish an appropriate prediction model for the typical litter moisture content in Yunnan Province.MethodDuring the fire prevention period, daily dynamic changes in the litter moisture content are obtained by monitoring the daily step size, and the relationships between the litter moisture content and meteorological elements are analyzed. In this study, the meteorological element regression method, moisture code method and direction estimation method are selected to establish litter moisture content prediction models, and the applicability of each model is analyzed.ResultsWe found that dynamic changes in the litter moisture content have obvious lags compared with meteorological elements, and the litter moisture content is mainly related to the air temperature, relative humidity and wind speed. With an increase in the sampling interval of meteorological elements, the significances of these correlations first increase and then decrease. The moisture content value obtained by directly using the moisture code method in the Fire Weather Index (FWI) significantly different from the measured value, so this method is not applicable. The mean absolute error (MAE) and mean relative error (MRE) values obtained with the meteorological element regression method are 2.97% and 14.06%, those from the moisture code method are 3.27% and 14.07%, and those from the direct estimation method are 2.82% and 12.76%, respectively.ConclusionsThe direct estimation method has the lowest error and the strongest extrapolation ability; this method can meet the needs of daily fire forecasting. Therefore, it is feasible to use the direct estimation method to predict litter moisture contents in Yunnan Province.

Theory-guided full convolutional neural network: An efficient surrogate model for inverse problems in subsurface contaminant transport

Identification of the location and strength of a contaminant source in an aquifer is a challenging but crucial task. Efficient surrogate models can be constructed to replace traditional time-consuming simulators while solving this inverse problem. In recent years, with the rapid development of machine learning (ML) algorithms, the artificial neural network (ANN) has been proven to be an efficient way for surrogate modeling. However, it may be difficult for ANN-based algorithms to learn the convection-dispersion equation and predict the contaminant concentration field due to their point-to-point learning scheme. Because of their strong localized features, the concentration fields can be seen as images. In contrast, the convolutional neural network (CNN) can extract spatial information better due to its convolutional structure. Herein, a theory-guided full convolutional neural network (TgFCNN) model is proposed to solve inverse problems in subsurface contaminant transport. TgFCNN can construct robust and reliable surrogate models with limited training realizations, and be further utilized for inverse modeling tasks. The loss function of TgFCNN comprises the residual of governing equations of contaminant transport, as well as data mismatch. Moreover, the iterative ensemble smoother (IES) method is employed to update the parameters while solving the inverse problems. The proposed TgFCNN model is evaluated in four scenarios. The results demonstrate that the TgFCNN model possesses strong generalization and extrapolation abilities, and satisfactory accuracy when estimating unknown contaminant source parameters, as well as the permeability field. The time consumption of the TgFCNN surrogate model for inverse tasks is also greatly reduced compared to using traditional simulators directly.

Detection and prediction of equilibrium states in kinetic plasma simulations via mode tracking using reduced-order dynamic mode decomposition

A dynamic mode decomposition (DMD) based reduced-order model (ROM) is developed for tracking, detection, and prediction of kinetic plasma behavior. DMD is applied to the high-fidelity kinetic plasma model based on the electromagnetic particle-in-cell (EMPIC) algorithm to extract the underlying dynamics and key features of the model. In particular, the ability of DMD to reconstruct the spatial pattern of the self electric field from high-fidelity data and the effect of DMD extrapolated self-fields on charged particle dynamics are investigated. An in-line sliding-window DMD method is presented for identifying the transition from transient to equilibrium state based on the loci of DMD eigenvalues in the complex plane. The in-line detection of equilibrium state combined with time extrapolation ability of DMD has the potential to effectively expedite the simulation. Case studies involving electron beams and plasma ball are presented to assess the strengths and limitations of the proposed method.

Modeling of Continuous PHA Production by a Hybrid Approach Based on First Principles and Machine Learning

Polyhydroxyalkanoates (PHA) are renewable alternatives to traditional oil-derived polymers. PHA can be produced by different microorganisms in continuous culture under specific media composition, which makes the production process both promising and challenging. In order to achieve large productivities while maintaining high yield and efficiency, the continuous culture needs to be operated in the so-called dual nutrient limitation condition, where both the nitrogen and carbon sources are kept at very low concentrations. Mathematical models can greatly assist both design and operation of the bioprocess, but are challenged by the complexity of the system, in particular by the dual nutrient-limited growth phenomenon, where the cells undergo a metabolic shift that abruptly changes their behavior. Traditional, non-structured mechanistic models based on Monod uptake kinetics can be used to describe the bioreactor operation under specific process conditions. However, in the absence of a model description of the metabolic phenomena inside the cell, the extrapolation to a broader operation domain (e.g., different feeding concentrations and dilution rates) may present mismatches between the predictions and the actual process outcomes. Such detailed models may require almost perfect knowledge of the cell metabolism and omic-level measurements, hampering their development. On the other hand, purely data-driven models that learn correlations from experimental data do not require any prior knowledge of the process and are therefore unbiased and flexible. However, many more data are required for their development and their extrapolation ability is limited to conditions that are similar to the ones used for training. An attractive alternative is the combination of the extrapolation power of first principles knowledge with the flexibility of machine learning methods. This approach results in a hybrid model for the growth and uptake rates that can be used to predict the dynamic operation of the bioreactor. Here we develop a hybrid model to describe the continuous production of PHA by Pseudomonas putida GPo1 culture. After training, the model with experimental data gained under different dilution rates and medium compositions, we demonstrate how the model can describe the process in a wide range of operating conditions, including both single and dual nutrient-limited growth.

ANN‐based modelling of peppermint flavour encapsulation process with ultrasound approach

AbstractEncapsulation has great potential for preserving the flavour and health benefits of bioactive compounds. Hence, in the previous study, an attempt was made to encapsulate peppermint flavour in a gum arabic (GA) shell. To further understand the effect of a wide range of parameters, in the present study, artificial neural networks (ANNs) are developed. To predict the effect of various parameters on the encapsulation process, networks are developed with a back‐propagation algorithm. Input parameters for the ANN are flavour concentration, GA concentration, spray dryer temperature and feed flow rate to the spray dryer. The encapsulation process is evaluated in terms of encapsulation efficiency, product yield, and particle size. To predict all outputs simultaneously, a combined model is developed. The results showed that the combined model has similar accuracy as that of the individual model and also helps to save on processing time. For the combined model, the best prediction performance is obtained with 5‐4‐3 ANN architecture exhibiting an R2 value of 0.9991, and corresponding MSE values of 0.000 54, 0.000 63, and 0.000 61 for encapsulation efficiency, product yield, and particle size, respectively. This indicates that the developed ANN model is capable of predicting the encapsulation process. The interpolation and extrapolation ability of the developed network is also evaluated.

Theory-guided Auto-Encoder for surrogate construction and inverse modeling

A Theory-guided Auto-Encoder (TgAE) framework is proposed for surrogate construction, and is further used for uncertainty quantification and inverse modeling tasks. The framework is built based on the Auto-Encoder (or Encoder–Decoder) architecture of the convolutional neural network (CNN) via a theory-guided training process. In order to incorporate physical constraints for achieving theory-guided training, the governing equations of the studied problems can be discretized by the finite difference scheme, and then be embedded into the training of the CNN. The residual of the discretized governing equations, as well as the data mismatch, constitute the loss function of the TgAE. The trained TgAE can be utilized to construct a surrogate that approximates the relationship between the model parameters and model responses with limited labeled data. Several subsurface flow cases are designed to test the performance of the TgAE. The results demonstrate that satisfactory accuracy for surrogate modeling and higher efficiency for uncertainty quantification tasks can be achieved with the TgAE. The TgAE also shows good extrapolation ability for cases with different correlation lengths and variances. Furthermore, inverse modeling tasks are also implemented with the TgAE surrogate, and satisfactory results are obtained.

Selection of Laboratory Mice for the Cognitive Task Successful Solution and for the Inability to Solve It.

Using the selected mouse strain EX as the founding population (selection for extrapolation ability) three selection generations of mice were obtained, which were selected for successful solution of object permanence test (plus-sub-strain) and for lack of such solution (minus-sub-strain). The successful solution required not only the ability to operate the object permanence rule (by J. Piajet), but the performance of complicated action (executive function) which was significantly higher in plus-substrain, and this is the unique example of successful selection for cognitive trait.

Evaluation method of continuous compaction quality of highway water stable base

This article summarizes several commonly used real-time monitoring indicators of compaction quality, proposes the methods for evaluating compaction quality of water stable bases. Two methods, the Kriging interpolation method and the geostatistical analysis method, are proposed for evaluating the spatial uniformity of compaction mass. The results show that: the Kriging interpolation method has strong extrapolation ability, and can consider the spatial correlation of data; the geostatistical analysis method can not only describe the structural variation of continuous compaction detection parameters, but also describe its random variation. It would provide reference suggestions for the compaction quality evaluation of water stable base.

A probabilistic creep model incorporating test condition, initial damage, and material property uncertainty

Uncertainty is prevalent in the creep resistance of alloys, where at elevated temperature and low pressure, rupture can range across logarithmic decades. In this study, a probabilistic continuum-damage-mechanics (CDM)-based model is derived to capture the uncertainty of creep resistance. To meet this objective, creep data for alloy 304 Stainless Steel is gathered. A constitutive model, “Sinh”, is calibrated deterministically to determine the statistical variability of the material properties. Three sources of uncertainty are injected into the model: test condition (stress and temperature), initial damage, and material properties. Probabilistic simulations are carried out by (a) calibrating probability distribution functions (pdfs) for each source of uncertainty (b) randomly sampling the pdfs using Monte Carlo methods and (c) executing simulations to replicate the uncertain creep behavior. A sensitivity analysis is performed to evaluate the relative effect of each source of uncertainty. In full probabilistic simulations, the cumulative uncertainty of creep behavior is evaluated. The probabilistic model accurately predicts the creep deformation and rupture of the available experiments. The probabilistic model is validated for interpolation but lacks extrapolation ability. Several future works are proposed to further improve the model.

Classification and Recognition Model of Water Saturation Level of Rock Based on Near-Infrared Spectroscopy

Abstract In geotechnical engineering, the water saturation of rock cannot be obtained in real time and is lossless. To solve this problem, a continuous classification representation method of water saturation of rock is proposed herein, the classification and recognition theory of the water saturation level of rock is established, and near-infrared spectrum acquisition experiments of rock under different water saturation levels are carried out. Based on the near-infrared spectrum, the partial least square (PLS) method is used to establish the recognition model, and the model is applied to the real-time identification of the saturation level in the gravel water absorption process. The results are as follows: (1) In this paper, the method of hierarchical representation of water saturation of rock is proposed, which solves the limited extrapolation ability and extrapolation precision problems of the preparation accuracy of water saturation of the rock sample and the classification-learning algorithm and thus provides the completeness and feasibility for the identification of water saturation of rock. (2) The classification recognition theory and method based on near-infrared spectroscopy to set up the water saturation level of rock have better recognition precision and can realize the water saturation level of the rock in real time and nondestructively, and (3) when the PLS method is used to establish the recognition model, the appropriate threshold parameters are selected to eliminate the abnormal samples, and multiple spectral segments are used in the modeling, which can greatly improve the recognition accuracy of the model.

Representation of Vapor-Liquid Equilibria Properties for Binary Mixtures Containing R1234ze(E) using Machine Learning Models

It is necessary to develop general and efficient models for the representation of vapor-liquid equilibria (VLE) for the binary mixtures containing R1234ze(E) which will be new alternatives to conventional refrigerants. This work investigates the applicability of four machine learning models, including K-nearest neighbor (KNN), support vector regression (SVR), random forests (RF), and multi-layer perceptron (MLP), in representing VLE for 10 binary mixtures. The accuracy, stability, computational complexity and extrapolation ability of these machine learning models are analyzed and compared with two thermodynamic models including Soave–Redlich–Kwong (SRK) and Peng–Robinson (PR). The results show the KNN, RF and MLP models are not suitable to accurately represent the experimental pressure and vapor-phase mole fraction. The SVR model is the most accurate machine learning model in representing the experimental pressure with a mean absolute relative error of 0.71%, basically as accurate as the PR and SVR models. Meanwhile, the SVR model describes the experimental vapor-phase mole fraction more accurately with a mean absolute error of 0.0049 compared to the PR and SVR models. In addition, the SVR model has a low running-time cost and a certain extent of extrapolation ability.

Cautions in weighting individual ecological niche models in ensemble forecasting

Ecological niche models are frequently used in ensembles for forecasting range shifts for species under scenarios of climate change or biological invasion. In such applications, maximizing predictive power of model transfers across temporal and spatial dimensions is crucial. Among methods used to produce ensemble models, weighted averages are most widely used, with weights usually based on metrics of interpolative performance of models. Yet model extrapolative ability is not related directly to interpolative ability. Here, we assess and evaluate this often-overlooked aspect of ensemble forecasting. We designed virtual species with six populations distributed across six continents, this allowed us to assess model transferability across global geographic spaces, as opposed to simple expansion into adjacent new environments or shifts into suitable conditions within the same general area. Individual niche models were calibrated on each continent and transferred to the other five continents for evaluation. Performance of consensus and individual models, together with the methods (mean, median, weight average, and PCAm) that were used to produce consensus models, were compared using AUC metrics and commission and omission errors across the spectrum of model thresholds. We found that consensus models reflected the central tendency of the individual model but did not outperform all individual models. Among methods used to generate consensus models, PCAm generally ranked higher than weighted averages, whereas mean and median were impacted by individual models. We highlight pitfalls in weighting individual models for ensemble models produced for model transfers. Regardless of whether models are to be transferred, we recommend using PCAm rather than weighted average for producing consensus models, as it outperformed other approaches and inherently reflects the constituent models’ central tendency sought in ensemble forecasting.

Extrapolation-enhanced model for travel decision making: An ensemble machine learning approach considering behavioral theory

Modeling individuals’ travel decision making in terms of choosing transport modes, route and departure time for daily activities is an indispensable component for transport system optimization and management. Conventional approaches of modeling travel decision making suffer from presumed model structures and parametric specifications. Emerging machine learning algorithms offer data-driven and non-parametric solutions for modeling travel decision making but encounter extrapolation issues (i.e., disability to predict scenarios beyond training samples) due to neglecting behavioral mechanisms in the framework. This study proposes an extrapolation-enhanced approach for modeling travel decision making, leveraging the complementary merits of ensemble machine learning algorithms (Random Forest in our study) and knowledge-based decision-making theory to enhance both predictive accuracy and model extrapolation. The proposed approach is examined using three datasets about travel decision making, including one estimation dataset (for cross-validation) and two test datasets (for model extrapolation tests). Especially, we use two test datasets containing extrapolated choice scenarios with features that exceed the ranges of training samples, to examine the predictive ability of proposed models in extrapolated choice scenarios, which have hardly been investigated by relevant literature. The results show that both proposed models and the direct application of Random Forest (RF) can give quite good predictive accuracy (around 80%) in the estimation dataset. However, RF has a deficient predictive ability in two test datasets with extrapolated choice scenarios. In contrast, the proposed models provide substantially superior predictive performances in the two test datasets, indicating much stronger extrapolation capacity. The model based on the proposed framework could improve the precision score by 274.93% than the direct application of RF in the first test dataset and by 21.9% in the second test dataset. The results indicate the merits of the proposed approach in terms of prediction power and extrapolation ability as compared to existing methods.

Structured machine learning tools for modelling characteristics of guided waves

The use of ultrasonic guided waves to probe the materials/structures for damage continues to increase in popularity for non-destructive evaluation (NDE) and structural health monitoring (SHM). The use of high-frequency waves such as these offers an advantage over low-frequency methods from their ability to detect damage on a smaller scale. However, in order to assess damage in a structure, and implement any NDE or SHM tool, knowledge of the behaviour of a guided wave throughout the material/structure is important (especially when designing sensor placement for SHM systems). Determining this behaviour is extremely difficult in complex materials, such as fibre–matrix composites, where unique phenomena such as continuous mode conversion takes place. This paper introduces a novel method for modelling the feature-space of guided waves in a composite material. This technique is based on a data-driven model, where prior physical knowledge can be used to create structured machine learning tools; where constraints are applied to provide said structure. The method shown makes use of Gaussian processes, a full Bayesian analysis tool, and in this paper it is shown how physical knowledge of the guided waves can be utilised in modelling using an ML tool. This paper shows that through careful consideration when applying machine learning techniques, more robust models can be generated which offer advantages such as extrapolation ability and physical interpretation.

Recurrent neural networks (RNNs) learn the constitutive law of viscoelasticity

Recurrent neural networks (RNNs) have demonstrated very impressive performances in learning sequential data, such as in language translation and music generation. Here, we show that the intrinsic computational aspect of RNNs is very similar to that of classical stress update algorithms in modeling history-dependent materials with an emphasis on viscoelasticity. Several numerical examples are designed, including 1-dimensional and 3-dimensional cases, which testify the ability of RNN model to compute the viscoelastic response when predicting on unseen test data. Additionally, it is found that the RNN model trained only on linear and step strain inputs can perform very well on prediction of completely different quadratic strain inputs, demonstrating certain level of generalization ability in extrapolation. Moreover, it is observed that the extrapolation ability depends on the types of strain inputs. The performance is better for continuous strain inputs than that for jump strain inputs. The differences in the generalization ability of RNN models in viscoelasticity and other history-dependent materials are discussed. It suggests that RNN data-driven modeling can be an alternative to the conventional viscoelasticity models.