Complex Input-output Relationships Research Articles

Bridging the gap between high-level quantum chemical methods and deep learning models

Supervised deep learning (DL) models are becoming ubiquitous in computational chemistry because they can efficiently learn complex input-output relationships and predict chemical properties at a cost significantly lower than methods based on quantum mechanics. The central challenge in many DL applications is the need to invest considerable computational resources in generating large ( N>1×105 ) training sets such that the resulting DL model can be generalized reliably to unseen systems. The lack of better alternatives has encouraged the use of low-cost and relatively inaccurate density-functional theory (DFT) methods to generate training data, leading to DL models that lack accuracy and reliability. In this article, we describe a robust and easily implemented approach based on property-specific atom-centered potentials (ACPs) that resolves this central challenge in DL model development. ACPs are one-electron potentials that are applied in combination with a computationally inexpensive but inaccurate quantum mechanical method (e.g. double-ζ DFT) and fitted against relatively few high-level data ( N≈1×103 – 1×104 ), possibly obtained from the literature. The resulting ACP-corrected methods retain the low cost of the double-ζ DFT approach, while generating high-level-quality data in unseen systems for the specific property for which they were designed. With this approach, we demonstrate that ACPs can be used as an intermediate method between high-level approaches and DL model development, enabling the calculation of large and accurate DL training sets for the chemical property of interest. We demonstrate the effectiveness of the proposed approach by predicting bond dissociation enthalpies, reaction barrier heights, and reaction energies with chemical accuracy at a computational cost lower than the DFT methods routinely used for DL training data set generation.

Machine Learning and Deep Learning Guided Assessment of Groundwater Reservoir Hydrodynamic Parameters: A Case Study of The El Haouz Aquifer

The Plio-Quaternary aquifer in the EL-Haouz-Mejjate region of Morocco is critical for water supply, necessitating accurate characterization for sustainable management. This study pioneers machine learning (ML) and deep learning (DL) techniques to elucidate the aquifer’s properties. Supervised algorithms, including random forest, regression, support vector machines, Gaussian process regression and neural networks, are trained on available hydrogeological data. Diverse features capture complex input-output relationships to predict key hydrodynamic factors like hydraulic conductivity and transmissivity fields. Aquifer architecture attributes, including substratum depth, thickness, and height, are also estimated. Model outputs are validated with field measurements, demonstrating promising accuracy. Enhanced hydrodynamic insights improve the conceptual model and groundwater flow modeling confidence. Uncertainties are reduced through this data-driven approach, enabling optimized aquifer management. Overall, this work shows how useful it is to combine ML and DL with traditional hydrogeology in order to get a better understanding of complicated aquifer systems. The techniques pioneered provide a pathway for sustainable management of this vital water resource.

Trust and specialization in complexity: Evidence from U.S. states

Industries with more complex input-output relationships are comparatively larger in U.S. states with higher levels of trust. This specialization pattern is not driven by differences in the states' and industries' other observable characteristics. Given the importance of complexity in economic growth and development, our results underscore the need to focus more on the role of social norms, particularly trust, in shaping patterns of economic specialization.

Sensitivity-dependent dynamic searching approach coupling multi-intelligent surrogates in homotopy mechanism for groundwater DNAPL-source inversion

Equifinality and premature convergence can result in considerable errors when simultaneously characterizing groundwater contamination sources and estimating contaminant transport parameters. To resolve this problem, we design a sensitivity-dependent progressive optimization system embedding ensemble-learning technique. To avoid repetitive CPU-demanding model evaluations in Sobol’ global sensitivity analysis and swarm intelligence optimization inverse modeling, Kriging, support vector regression (SVR), kernel extreme learning machine (KELM), and deep convolutional neural network (DCNN) are compared and ensembled to build an accurate surrogate of the numerical model. In addition, the sensitivities of different source characteristics and contaminant transport parameters are set as important indicators to adjust the displacement vectors of the swarm in each iteration during the optimization process to achieve a balanced identification of sensitivity-varied elements. Moreover, a homotopy-based progressive searching mechanism approach to the global optimum in large areas is developed, with the aim of preventing premature convergence for multimodal search problems. The results indicate that the ensemble learning model efficiently captures the complex input-output relationship of the numerical model with an increased determination coefficient (R2 = 0.9988), while the mean relative error is limited to 0.9314%. Although the contribution of source characteristics and contaminant transport parameters to the spatial-temporal distribution of contaminants vary dramatically, the combined application of sensitivity analysis, homotopy theory, and swarm intelligence optimization provides a more stable and accurate estimation of all the elements. The mean relative error of the identification results significantly reduced from 7.2184% to 3.2718%, whereas the maximum relative error is limited to 9.9501%.

Simulasi Penggunaan Listrik Tarif Sosial Menggunakan Algoritma ERNN

The use of electricity under the social tariff category has increased significantly each year, both purely social and commercial social use. The use of social tariff electricity is intended for public interest activities for both the lower and upper middle social strata which are oriented towards fulfilling growth and development facilities for the public interest, so that a simulation of social electricity usage is needed to map a picture of the condition of the amount of social electricity usage in the future. The research was conducted to determine the estimation of how much electricity is used by using the Elman Recurrent Neural Network (ERNN) algorithm by reducing the input dimensions. The ERNN algorithm is used to simulate network parameters formed from complex input-output relationships, so that data patterns can be found. The factors of the input dimensions of this study are demographic data, electricity usage, social customers, population, gross regional domestic product (GRDP) and industrial growth. The results showed that the ERNN algorithm is capable of simulating formed network parameters that can be used for training and validation so that the value of the network Mean Square Error (MSE) can be determined, with prediction accuracy using the Mean Absolute Percentage Error (MAPE) for forecast in sample in the forecast period of 5 years obtained an average of 0.77%, and able to know the dominant factors that influence the use of social tariff electricity.

Multi-objective Bayesian optimization for the design of nacre-inspired composites: optimizing and understanding biomimetics through AI.

The hierarchical structures found in biological materials lead to an outstanding balance of multiple material properties, and numerous research studies have been initiated to emulate the key concepts for the designing of engineering materials, the so-called bioinspired composites. However, the optimization of bioinspired composites has long been difficult as it usually falls into the category of 'black-box problem', the objective functions not being available in a functional form. Also, bioinspired composites possess multiple material properties that are in a trade-off relationship, making it impossible to reach a unique optimal design solution. As a breakthrough, we propose a data-driven material design framework which can generate bioinspired composite designs with an optimal balance of material properties. In this study, a nacre-inspired composite is chosen as the subject of study and the optimization framework is applied to determine the designs that have an optimal balance of strength, toughness, and specific volume. Gaussian process regression was adopted for the modeling of a complex input-output relationship, and the model was trained with the data generated from the crack phase-field simulation. Then, multi-objective Bayesian optimization was carried out to determine pareto-optimal composite designs. As a result, the proposed data-driven algorithm could generate a 3D pareto surface of optimal composite design solutions, from which a user can choose a design that suits his/her requirement. To validate the result, several pareto-optimal designs are built using a PolyJet 3D printer, and their tensile test results show that each of the characteristic designs is well optimized for its specific target objective.

Predictive modeling of pharmaceutical product removal by a managed aquifer recharge system: Comparison and optimization of models using ensemble learners

Pharmaceutical products (PPs) are emerging water pollutants with adverse environmental and health-related impacts, owing to their toxic, persistent, and undetectable microscopic nature. Globally, increasing scientific knowledge and advanced technologies have allowed researchers to study PP-associated problems and their removal for water reuse. Experimental modeling methods require laborious, lengthy, expensive, and environmentally hazardous lab-work to optimize the process. On the other hand, predictive machine learning (ML) models can trace the complex input-output relationship of a process using available datasets. In this study, ensemble ML techniques, including decision tree (DT), random forest (RF), and Xtreme gradient boost (XGB), were used to explore PP (diclofenac, iopromide, propranolol, and trimethoprim) removal by a managed aquifer recharge (MAR) system. The model input parameters included characteristics of reclaimed water and soil used in the columns, pH, dissolved organic carbon, operating time, nitrogen dioxide, sulfate, nitrate, electrical conductivity, manganese, and iron. The selected PP removal was the model output. Datasets were collected through a one-year experimental study of continuous MAR system operation to predict the removal of PPs. DT, RF, and XGB models were then developed for one of the selected compounds and tested for the others to check the reliability of the ML model results. The developed models were assessed using statistical performance matrices. The experimental results showed >80% removal of propranolol and trimethoprim; however, removal of diclofenac and iopromide was only ≈50% by the MAR system. The proposed DT and RF models presented higher coefficients of determination (R2 ≥ 0.92) for diclofenac, propranolol, and trimethoprim than for iopromide (R2 ≤ 0.63). In contrast, the XGB model showed better results for diclofenac, iopromide, propranolol, and trimethoprim, with R2 values of 0.92, 0.72, 0.96, and 0.97, respectively. Therefore, XGB could be the best predictive model to provide insight into the adaptation of ML models to predict PP removal by the MAR system, thereby minimizing experimental work.

A Visualization Method of the Economic Input–Output Table: Mapping Monetary Flows in the Form of Sankey Diagrams

The input–output table and input–output method have been widely used to understand complex economic structures and are often used in cross-disciplinary research between economics and other disciplines, such as analysis of embodied energy, carbon footprints, the water–food nexus, etc. However, when researchers present these results to audiences, especially policymakers, they often lack an effective visualization tool to present (1) the full picture of the input–output table; (2) the complicated upstream–downstream nexus, and (3) the input–output relationships between the economic sectors. Therefore, a better visualization method is developed to solve this problem. We propose mapping an input–output table into a Sankey diagram, a so-called monetary allocation Sankey diagram. We first designed the mapping structure of a monetary allocation Sankey diagram according to the general structure of an economic monetary input–output table to establish the correspondence nexus between the table and diagram. We used China as a case study to demonstrate the usage of the monetary allocation Sankey diagram. The purpose of the monetary allocation Sankey diagram is to help people understand the input–output table in a short time and quickly grasp the big picture of the economic system. To verify whether this goal is achieved, we presented and applied these Sankey diagrams on different occasions and obtained evaluations from scholars from different academic backgrounds. The evaluation shows that the monetary allocation Sankey diagram is not only a visualization result of the input–output table but also a miniature model of the economic system, which allows people to “truly observe” the complex input–output relationship and upstream–downstream nexus in the economic system. Researchers can quickly grasp the main features of the economic system by observing the miniature model, or they can use this miniature model as an auxiliary tool to introduce the economic system and its inherent complex relationships to the audience.

Neuromorphic metamaterial structures

Computerized structural optimization methods are often used to design the shape of structures to achieve a desired function, such as a specific compliance. In the simplest case of a structure built from a material with small deformations obeying Cauchy elasticity, the compliance is constant and the relationship between the forces applied on the structure and its deformation is linear. This linearity severely limits the types of functions which can be achieved by such materials. Here we study mechanical metamaterials made of repeating unit cells, each with specific dimensional parameters and with one-sided contact non-linearities. We show that the force and displacement equilibrium configurations of these metamaterials are mathematically equivalent to the fixed points of a recurrent artificial neural network. By exploiting this equivalence, we demonstrate mechanical metamaterials that can be designed (trained) to implement complex non-linear functions, using a gradient descent algorithm as in artificial neural networks. One of our metamaterial structures has a higher compliance when it is pressed against a pattern of raised bumps corresponding to the vowels in the Braille alphabet, than when it is pressed against patterns for six consonants. As artificial neural networks are known to be efficient models for numerous problems in machine learning, our results reveal that beneficial features of neural networks can be transferred to physical objects (mechanical structures). These features include the design of systems with complex input–output relationships by using generic methods that only rely on the repetitive processing of pairs of inputs and desired outputs, as well as remarkable generalization capabilities. We anticipate our methodology to be a starting point for the transfer of some of the breakthroughs in the rapidly advancing field of machine learning to highly functional physical devices in applications that are constrained by energy, volume, data processing or response time.

DP-GMM clustering-based ensemble learning prediction methodology for dam deformation considering spatiotemporal differentiation

The deformation behavior can effectively reflect the health status of a dam, which can suffer from dynamic and time-varying loading modes and material properties, exhibiting distinct spatiotemporal differentiation. The conventional dam health evaluations, however, focus on the displacement regularity of local monitoring points, ignoring the spatiotemporal diversity in deformation behavior, and thus cannot truly reflect the status of the entire dam. Moreover, most of the existing displacement predictions are conducted on a single-algorithm machine-learning-based model, which focuses on single-task regression and disregards the potential cross-relatedness among the deformation in adjacent areas of the dam, showing less competitiveness in complex input–output relationship learning for various spatiotemporal conditions. For more reliable deformation forecasting, an integrated displacement prediction methodology considering spatiotemporal differentiation is proposed based on the spatiotemporal clustering and machine learning algorithms, in which the minimum-density-entropy-optimized density peaks clustering method is embedded with Gaussian-mixture-model-based clustering to achieve the reliable spatiotemporal differentiation identification of dam behavior and a multioutput ensemble learning framework coupled with the extreme learning machine and support vector machine is designed to capture the complex mapping from environmental factors to deformation, considering the cross-relatedness and spatiotemporal diversity. Moreover, a synchronous optimization strategy based on improved grey wolf optimization is embedded to enhance the performance. Hence, the final deformation behavior is forecasted by the ensemble learning of various data features and mapping rules. Taking the displacement observations of an actual dam as an example, the results show that the proposed methodology achieves excellent performance in dam behavior forecasting.

Effects of Ih and TASK-like shunting current on dendritic impedance in layer 5 pyramidal-tract neurons.

Pyramidal neurons in neocortex have complex input-output relationships that depend on their morphologies, ion channel distributions, and the nature of their inputs, but which cannot be replicated by simple integrate-and-fire models. The impedance properties of their dendritic arbors, such as resonance and phase shift, shape neuronal responses to synaptic inputs and provide intraneuronal functional maps reflecting their intrinsic dynamics and excitability. Experimental studies of dendritic impedance have shown that neocortical pyramidal tract neurons exhibit distance-dependent changes in resonance and impedance phase with respect to the soma. We, therefore, investigated how well several biophysically detailed multicompartment models of neocortical layer 5 pyramidal tract neurons reproduce the location-dependent impedance profiles observed experimentally. Each model tested here exhibited location-dependent impedance profiles, but most captured either the observed impedance amplitude or phase, not both. The only model that captured features from both incorporates hyperpolarization-activated cyclic nucleotide-gated (HCN) channels and a shunting current, such as that produced by Twik-related acid-sensitive K+ (TASK) channels. TASK-like channel density in this model was proportional to local HCN channel density. We found that although this shunting current alone is insufficient to produce resonance or realistic phase response, it modulates all features of dendritic impedance, including resonance frequencies, resonance strength, synchronous frequencies, and total inductive phase. We also explored how the interaction of HCN channel current (Ih) and a TASK-like shunting current shape synaptic potentials and produce degeneracy in dendritic impedance profiles, wherein different combinations of Ih and shunting current can produce the same impedance profile.NEW & NOTEWORTHY We simulated chirp current stimulation in the apical dendrites of 5 biophysically detailed multicompartment models of neocortical pyramidal tract neurons and found that a combination of HCN channels and TASK-like channels produced the best fit to experimental measurements of dendritic impedance. We then explored how HCN and TASK-like channels can shape the dendritic impedance as well as the voltage response to synaptic currents.

Trust and Specialization in Complexity: Evidence from U.S. States

In this paper, we show that trust favors specialization in complexity. U.S. states with a high level of generalized trust tend to specialize in industries involving more complex input-output relationships. This pattern is not driven by differences in the states’ other observable characteristics or unobservable time-varying industry- or state-specific factors and does not reflect selection by export destination. Although theoretical considerations suggest that trust may be endogenous to the location of complex industries, an instrumental variable strategy that leverages the impact of racial discrimination confirms that trust is a factor in the specialization of U.S. states in more complex industries.

Comparison of neural network models for estimation of reference crop evapotranspiration

Reference evapotranspiration (ETo) is important component in calculation of crop water requirements, climatological and hydrological studies. However, it is a nonlinear dynamic and complex process. Artificial neural network (ANN) techniques are powerful to accurately map complex and nonlinear input output relationships. Penman-Monteith (P-M) model is sole standard method for estimation of ETo but it requires many kinds of weather data. Hence it was planned to test an ANN technique for estimating the ETo with limited and full data for Kolhapur region, Maharashtra, India. The five ANN models were developed by different combinations of meteorological parameters as neurons in input layer with varying neurons in middle layer and one neuron in the output layer i.e. ETo by P-M method. These are ANN1 (Pan evaporation); ANN2 (Tmax. and Tmin.); ANN3 (Tmax.,Tmin and SSH); ANN4 (Tmax., Tmin., RHmax., RHmin., and SSH) and ANN5 (Tmax., Tmin., RHmax., RHmin., SSH and WS). During training mode, it was observed that ANN5 model showed the best values of all performance measures i.e. correlation coefficient (0.997), index of agreement (0.998), efficiency coefficient (0.993),root mean square error (0.100), mean absolute error (0.059), mean absolute percentage error (2.076) due to full data. Under limited data condition, it was observed that the results of all performance measure (ANN1 to ANN4) models varied in the range as correlation coefficient (0.929 to 0.967), index of agreement (0.962 to 0.983), root mean square error (0.317 to 0.460), mean absolute error (0.194 to 0.297), mean absolute percentage error (6.027 to 9.679) and efficiency coefficient (0.863 to 0.934) in training mode. It can be seen that all limited data models demonstrate relatively very close performances based on statistical criteria. Similar kind of close differences for each performance measure were observed during validation stage of all ANN models. Results indicated that all ANN models satisfy performance criteria well in training and validation mode and can be generalized for prediction of ETo values. Overall, the performance suggests that ANN models can be an accepted for accurate prediction of ETo values for Kolhapur region as per data availability.

Performing multi-target regression via gene expression programming-based ensemble models

Multi-Target Regression problem comprises the prediction of multiple continuous variables given a common set of input features, unlike traditional regression tasks, where just one output target is available. There are two major challenges when addressing this problem, namely the exploration of the inter-target dependencies and the modeling of complex input–output relationships. This work proposes a Symbolic Regression method following the basis of Gene Expression Programming paradigm to solve the multi-target regression problem, and called GEPMTR. It evolves a population of individuals, where each one represents a complete solution to the problem by using a multi-genic chromosome, and encodes a mathematical function for each target variable involving the input attributes. The proposed model can estimate the inter-target dependencies by applying some genetic operators. Furthermore, three ensemble-based methods are developed to better exploit the inter-target and input–output relationships. The effectiveness of the proposals is analyzed through an extensive experimental study on 18 datasets. The codification schema and the process followed to ensure a diverse population in GEPMTR lead to obtain an effective proposal to solve the MTR problem. Furthermore, the EGEPMTR-B ensemble method obtained the best performance across all proposed models, being the best in 8 out of 11 cases, demonstrating that more sophisticated mechanisms were not needed for ensuring that GEPMTR method would properly model the existing inter-target dependencies. Finally, the experimental results also showed that the proposed approach attains competitive results compared to state-of-the-art, showing the possibilities that can bring this research line for effectively solving the MTR problem.

Hidden representations in deep neural networks: Part 2. Regression problems

Deep neural networks are an important class of machine learning models useful for representing complex input-output relationships. While their recent success is unparalleled, so is the inability to explain their internal representations. In this second part of a two-part series, we focus on understanding the hidden representations of deep neural networks and the underlying mechanisms for regression problems. We highlight challenges associated with deep neural networks with simple models that help us gain insight into the functioning of the hidden layers and the mechanism of the operation of the network. The article is structured in a tutorial-like fashion for the benefit of new practitioners so that they can appreciate nuances and the pitfalls involved in developing a deep neural network models for regression problems.

Implementation of the Perceptron Method for Recognizing the Patterns of Types of Intestinal Nematode Worms

The purpose of pattern recognition is do the process of classifying an object into one particular class based on the pattern it has, so it can be used to recognize patterns of intestinal nematode worm types. One of the methods used in pattern recognition is by utilizing the artificial neural network method, the artificial neural network is able to represent a complex Input-Output relationship. For that the algorithm used is the perceptron algorithm. Perceptron is one method of Artificial Neural Networks. In the introduction of types of intestinal nematode worms, a computer must be trained in advance using training data and test data, this study discusses how a computer can recognize a pattern of types of intestinal nematode worms using the perceptron method. Based on the results of testing trials with input in the form of worm image scan results, based on the results of the perceptron method testing is able to recognize the pattern recognition of the types of intestinal nematode worms and be able to analyze with the right results of 100% for pinworms patterns, hookworm patterns, and 40- 50% for roundworms, by comparing the output value and the target value entered first.

An artificial neural network based approach for prediction the thermal conductivity of nanofluids

Thermal conductivity is an important thermophysical property of nanofluids in many practical heat transfer applications. In this study, a novel approach is proposed to predict the thermal conductivity of nanofluids under multiple operating parameters. The proposed approach may be extended to be used to other thermophysical properties of nanofluids. The Kohonen’s self-organizing maps (SOM), as an unsupervised artificial neural network (ANN), is used to provide an accurate prediction tool for the problem in hand. Furthermore, SOM, similar to any ANN-based approach, can handle nonlinear and complex input–output relationships with high generalization ability. Comparison of the SOM predicted values with corresponding available theoretical results as well as experimental data implies high prediction capability of the developed approach. The proposed approach was utilized to predict thermal conductivity ratio of oxide (Al2O3, CuO and TiO2)/water nanofluids under various operating conditions (nanoparticle size, temperature, and nanoparticle volume fraction).

Performing Multi-Target Regression via a Parameter Sharing-Based Deep Network.

Multi-target regression (MTR) comprises the prediction of multiple continuous target variables from a common set of input variables. There are two major challenges when addressing the MTR problem: the exploration of the inter-target dependencies and the modeling of complex input-output relationships. This paper proposes a neural network model that is able to simultaneously address these two challenges in a flexible way. A deep architecture well suited for learning multiple continuous outputs is designed, providing some flexibility to model the inter-target relationships by sharing network parameters as well as the possibility to exploit target-specific patterns by learning a set of nonshared parameters for each target. The effectiveness of the proposal is analyzed through an extensive experimental study on 18 datasets, demonstrating the benefits of using a shared representation that exploits the commonalities between target variables. According to the experimental results, the proposed model is competitive with respect to the state-of-the-art in MTR.

Operation rule derivation of hydropower reservoir by k-means clustering method and extreme learning machine based on particle swarm optimization

In practice, the rational operation rule derived from historical information and real-time working condition can help the operators make the quasi-optimal scheduling plan of hydropower reservoirs, leading to significant improvements in the generation benefit. As an emerging artificial intelligence method, the extreme learning machine (ELM) provides a new effective tool to derivate the reservoir operation rule. However, it is difficult for the standard ELM method to avoid falling into local optima due to the random determination of both input-hidden weights and hidden bias. To enhance the ELM performance, this research develops a novel class-based evolutionary extreme learning machine (CEELM) to determine the appropriate operation rule of hydropower reservoir. In CEELM, the k-means clustering method is firstly adopted to divide all the influential factors into several disjointed sub-regions with simpler patterns; and then ELM optimized by particle swarm intelligence is applied to identify the complex input-output relationship in each cluster. The results from two reservoirs of China show that our method can obtain satisfying performance in deriving operation rules of hydropower reservoir. Thus, it can be concluded that the model’s generalization capability can be improved by isolating each subclass composed of similar dataset.

Enhancing the Productivity of Wire Electrical Discharge Machining Toward Sustainable Production by using Artificial Neural Network Modelling

Sustainability plays an important role in manufacturing industries through economically-sound processes that able to minimize negative environmental impacts while having the social benefits. In this present study, the modeling of wire electrical discharge machining (WEDM) cutting process using an artificial neural network (ANN) for prediction has been carried out with a focus on sustainable production. The objective was to develop an ANN model for prediction of two sustainable measures which were material removal rate (as an economic aspect) and surface roughness (as a social aspect) of titanium alloy with ten input parameters. By concerning environmental pollution due to its intrinsic characteristics such as liquid wastes, the water-based dielectric fluid has been used in this study which represents an environmental aspect in sustainability. For this purpose, a feed-forward backpropagation ANN was developed and trained using the minimal experimental data. The other empirical modelling techniques (statistics based) are less in flexibility and prediction accuracy. The minimal, vague data and nonlinear complex input-output relationship make this ANN model simple and perfects method in the manufacturing environment as well as in this study. The results showed good agreement with the experimental data confirming the effectiveness of the ANN approach in the modeling of material removal rate and surface roughness of this cutting process.