Robust Artificial Neural Networks Research Articles

Correlating physico-chemical properties of analytes with Hansen solubility parameters of solvents using machine learning algorithm for predicting suitable extraction solvent

Artificial neural networks (ANNs) are biologically inspired algorithms designed to simulate the way in which the human brain processes information. In sample preparation for bioanalysis, liquid–liquid extraction (LLE) represents an important step with the extraction solvent selection is the key laborious step. In the current work, a robust and reliable ANNs model for LLE solvent prediction was generated which could predict the suitable solvent for analyte extraction. The developed ANNs model takes a set of chosen descriptors for the cited analyte as an input and predicts the corresponding Hansen solubility parameters of the suitable extraction solvent as a model output. Then, from the solvent combination’s appendix, the analyst can identify the proposed extraction solvents' combination for the cited analyte easily and efficiently. For the experimental validation of the model prediction capabilities, twenty structurally diverse drugs belonging to different pharmacological classes were extracted from human plasma. The extraction process was performed using the predicted extraction solvent combination for each drug and quantitively estimated by HPLC/UV methods to assess their extraction recovery. The developed LLE solvent prediction model is in- line with the global trend towards green chemistry since it limits the consumption of organic solvents.

CFRP-RC fracture strength prediction: A novel approach using deep learning

Externally bonded carbon fiber-reinforced polymer (CFRP) composites have been widely adopted for strengthening and repairing aging concrete structures. However, premature fracture at the CFRP-concrete interface remains a significant concern, necessitating a better understanding of the interfacial bond behavior and fracture mechanisms. Traditional experimental studies and numerical simulations have limitations in accurately capturing the complex nonlinear factors influencing this fracture behavior. This paper explores the novel application of machine learning (ML), specifically artificial neural networks (ANNs), for predicting the fracture strength of CFRP-reinforced concrete (CFRP-RC) members. ANNs enable mapping the intricate relationships between input parameters (e.g., CFRP/concrete properties, geometric configurations) and fracture responses through iterative training on experimental data. Key aspects include comprehensive data collection, preprocessing techniques, feature selection methods like mean impact value (MIV) analysis, and neural network architecture design. The paper highlights successful ANN applications in predicting CFRP-RC bond strength, shear capacity, and overall fracture behavior. A systematic approach is proposed for developing robust ANN models tailored to CFRP-RC fracture prediction, encompassing data curation, input variable screening, network training/validation, and model interpretability analyses. By leveraging ML’s ability to handle multifaceted nonlinearities, this data-driven framework offers a powerful alternative to traditional methods, potentially enhancing the design, analysis, and performance evaluation of CFRP-strengthened concrete structures.

A robust neural network model for fault detection in the presence of mislabelled data

AbstractSeveral data‐driven methodologies for process monitoring and detection of faults or abnormalities have been developed for the safety of processing systems. The effectiveness of data‐based models, however, is impacted by the volume and quality of training data. This work presents a robust neural network model for addressing the mislabelled and low‐quality data in detecting faults and process abnormalities. The approach is based on harnessing data quality features along with supervisory labels in the network training. The data quality has been computed using the Mahalanobis distances and trusted centres of each class of data such as normal and faulty data. The method has been examined for detecting abnormalities in two case studies; a continuous stirred tank heater problem for detecting leaks and the Tennessee Eastman chemical process for detecting step and sticking faults. The performance of the proposed robust artificial neural networks (ANN) model is evaluated in terms of accuracy, fault detection rate, false alarm rate, and classification index at varying extents of mislabelling, namely, 1%, 5%, and 10% mislabelled data. The proposed model demonstrates higher detection performance, especially at increased labels of mislabelled data where the performance of the conventional ANN is severely impacted. The proposed methodology can be advantageous in handling mislabelled and low‐quality data issues which are crucial in the data‐driven modelling of processing systems.

Artificial neural networks-based [formula omitted]-integral prediction for cracked bodies under elasto-plastic deformation state –monotonic loading

Artificial neural networks (ANNs) integrated with a finite element (FE)-based equivalent domain integral method are developed to compute J-integral at the vicinity of crack tips through a time-efficient approach. Robust ANN models are trained to establish nonlinear relationships between FE predicted elastic and elasto-plastic stress, strain, and displacement fields of stainless steel (SS304). Subsequently, elastic–plastic J-integral can be determined by using only elastic FE analysis solution rather than computationally expensive elasto-plastic FE analysis solution. The results show that well-trained ANN models can efficiently and accurately determine J-integral around the crack tips on the basis of numerical elastic FE analysis solution.

Uncertainty Propagation Based MINLP Approach for Artificial Neural Network Structure Reduction

The performance of artificial neural networks (ANNs) is highly influenced by the selection of input variables and the architecture defined by hyper parameters such as the number of neurons in the hidden layer and connections between network variables. Although there are some black-box and trial and error based studies in the literature to deal with these issues, it is fair to state that a rigorous and systematic method providing global and unique solution is still missing. Accordingly, in this study, a mixed integer nonlinear programming (MINLP) formulation is proposed to detect the best features and connections among the neural network elements while propagating parameter and output uncertainties for regression problems. The objective of the formulation is to minimize the covariance of the estimated parameters while by (i) detecting the ideal number of neurons, (ii) synthesizing the connection configuration between those neurons, inputs and outputs, and (iii) selecting optimum input variables in a multi variable data set to design and ensure identifiable ANN architectures. As a result, suggested approach provides a robust and optimal ANN architecture with tighter prediction bounds obtained from propagation of parameter uncertainty, and higher prediction accuracy compared to the traditional fully connected approach and other benchmarks. Furthermore, such a performance is obtained after elimination of approximately 85% and 90% of the connections, for two case studies respectively, compared to traditional ANN in addition to significant reduction in the input subset.

Neuro-Scientific Analysis of Weights in Neural Networks

Deep learning is a popular topic among machine learning researchers nowadays, with great strides being made in recent years to develop robust artificial neural networks for faster convergence to a reasonable accuracy. Network architecture and hyperparameters of the model are fundamental aspects of model convergence. One such important parameter is the initial values of weights, also known as weight initialization. In this paper, we perform two research tasks concerned with the weights of neural networks. First, we develop three novel weight initialization algorithms inspired by the neuroscientific construction of the mammalian brains and then test them on benchmark datasets against other algorithms to compare and assess their performance. We call these algorithms the lognormal weight initialization, modified lognormal weight initialization, and skewed weight initialization. We observe from our results that these initialization algorithms provide state-of-the-art results on all of the benchmark datasets. Second, we analyze the influence of training an artificial neural network on its weight distribution by measuring the correlation between the quantitative metrics of skewness and kurtosis against the model accuracy using linear regression for different weight initializations. Results indicate a positive correlation between network accuracy and skewness of the weight distribution but no affirmative relation between accuracy and kurtosis. This analysis provides further insight into understanding the inner mechanism of neural network training using the shape of weight distribution. Overall, the works in this paper are the first of their kind in incorporating neuroscientific knowledge into the domain of artificial neural network weights.

Predicting Drying Performance of Osmotically Treated Heat Sensitive Products Using Artificial Intelligence

The main goal of this research is to develop and apply a robust Artificial Neural Networks (ANNs) model for predicting the characteristics of the osmotically drying treated potato and apple samples as a model heat-sensitive product in vacuum contact dryer. Concentrated salt and sugar solutions were used as the osmotic solutions at 27C. Series of experiments were performed at various temperatures of 35C, 40C, and 55C for conduction heat input under vacuum ( −760 mm Hg) condition. Some experiments were also performed in a pure vacuum without heat addition. Dimensionless moisture content (DMC), effective moisture diffusivity, and mass flux were considered as the performance parameters in this study. Results revealed that the osmotic dehydration using a concentrated sugar solution shows a higher reduction in the initial moisture loss of 19.87% compared to 5.3% in the salt solution. Furthermore, a significant enhancement of drying performance of about 27% in DMC was observed for both samples at vacuum and 40C compared to pure vacuum drying conditions. Using the experimental data, a robust artificial neural network (ANN) was proposed to describe the osmotic dehydration’s behavior on the drying process. The ANN model outputs are the dimensionless moisture contents (DMC), the diffusivity, and the mass flux. Whereas the ANN inputs were the drying time, the percent of sugar solution, and the percent of salt solution. For the ANN apple’s model, the minimum root mean square error (RMSE) values were 0.0261, 0.0349 and 0.0406, for DMC, diffusivity, and mass flux, respectively. Whereas the best correlation coefficients of the above three parameters’ determination values were 0.9909, 0.9867 and 0.9744, respectively. For the ANN potato’s model, the minimum RMSE values were 0.0124, 0.0140 and 0.0333, for DMC, diffusivity, and mass flux, respectively. And the best correlation coefficients of the parameters’ values were found 0.9969, 0.9968 and 0.9736, respectively. Accordingly, the ANN model’s prediction has a perfect agreement with the experimental dataset, which confirmed the ANN model’s accuracy.

Floating-gate photosensitive synaptic transistors with tunable functions for neuromorphic computing

Synaptic devices that merge memory and processing functions into one unit have broad application potentials in neuromorphic computing, soft robots, and human-machine interfaces. However, most previously reported synaptic devices exhibit fixed performance once been fabricated, which limits their application in diverse scenarios. Here, we report floating-gate photosensitive synaptic transistors with charge-trapping perovskite quantum dots (PQDs) and atomic layer deposited (ALD) Al2O3 tunneling layers, which exhibit typical synaptic behaviors including excitatory postsynaptic current (EPSC), pair-pulse facilitation and dynamic filtering characteristics under both electrical or optical signal stimulation. Further, the combination of the high-quality Al2O3 tuning layer and highly photosensitive PQDs charge-trapping layer provides the devices with extensively tunable synaptic performance under optical and electrical co-modulation. Applying light during electrical modulation can significantly improve both the synaptic weight changes and the non-linearity of weight updates, while the memory effect under light modulation can be obviously adjusted by the gate voltage. The pattern learning and forgetting processes for “0” and “1” with different synaptic weights and memory times are further demonstrated in the device array. Overall, this work provides synaptic devices with tunable functions for building complex and robust artificial neural networks.

Accelerated Modeling of Lithium Diffusion in Solid State Electrolytes Using Artificial Neural Networks

There is great interest in solid state lithium electrolytes to replace the flammable organic electrolyte for an all solid state battery. Previous efforts trying to understand the structure-function relationships resulting in high ionic conductivity materials have mainly relied on ab initio molecular dynamics. Such simulations, however, are computationally demanding and cannot be reasonably applied to large systems containing more than a few hundred atoms. Herein, we investigate using artificial neural networks (ANN) to accelerate the calculation of high accuracy atomic forces and energies used during molecular dynamics (MD) simulations (Figure 1A), to eliminate the need for costly ab initio force and energy evaluation methods, such as density functional theory (DFT). After carefully training a robust ANN for four and five element systems, we obtain nearly identical lithium ion diffusivities for Li10GeP2S12 (LGPS) when benchmarking the ANN-MD results with DFT-MD. We find that the ANN generated potentials scale linearly with atomic system size, compared to cubically in DFT (Figure 1B). To demonstrate the power of the outlined ANN-MD approach we apply it to a doped LGPS system to calculate the effect of concentrations of chlorine on the lithium diffusivity at a resolution that would be unrealistic to model with DFT-MD. We find that there is an optimal chlorine doping concentration which is a result of competing electronic effects of the effective dielectric constant and effective ionic radius of chlorine. Such effects could not be adequately studied with classical potentials. We find that ANN-MD simulations can provide the framework to study systems that require a large number of atoms more efficiently while maintaining high accuracy. Figure 1

Development of robust artificial neural networks for prediction of 5-year survival in bladder cancer

PurposeWhen exploring survival outcomes for patients with bladder cancer, most studies rely on conventional statistical methods such as proportional hazards models. Given the successful application of machine learning to handle big data in many disciplines outside of medicine, we sought to determine if machine learning could be used to improve our ability to predict survival in bladder cancer patients. We compare the performance of artificial neural networks (ANN), a type of machine learning algorithm, with that of multivariable Cox proportional hazards (CPH) models in the prediction of 5-year disease-specific survival (DSS) and overall survival (OS) in patients with bladder cancer. Subjects and methodsThe National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) 18 program database was queried to identify adult patients with bladder cancer diagnosed between 1995 and 2010, yielding 161,227 patients who met our inclusion criteria. ANNs were trained and tested on an 80/20 split of the dataset. Multivariable CPH models were developed in parallel. Variables used for prediction included age, sex, race, grade, SEER stage, tumor size, lymph node involvement, degree of extension, and surgery received. The primary outcomes were 5-year DSS and 5-year OS. Receiver operating characteristic curve analysis was conducted, and ANN models were tested for calibration. ResultsThe area under the curve for the ANN models was 0.81 for the OS model and 0.80 for the DSS model. Area under the curve for the CPH models was 0.70 for OS and 0.81 for DSS. The ANN OS model achieved a calibration slope of 1.03 and a calibration intercept of −0.04, while the ANN DSS model achieved a calibration slope of 0.99 and a calibration intercept of −0.04. ConclusionsMachine learning algorithms can improve our ability to predict bladder cancer prognosis. Compared to CPH models, ANNs predicted OS more accurately and DSS with similar accuracy. Given the inherent limitations of administrative datasets, machine learning may allow for optimal interpretation of the complex data they contain.

Application of radial basis function artificial neural network to quantify interfacial energies related to membrane fouling in a membrane bioreactor

Efficient quantification of interfacial energy related with membrane fouling represents the primary interest in membrane bioreactors (MBRs) as interfacial energy determines foulant layer formation. In this study, radial basis function (RBF) artificial neural networks (ANNs) with five related factors as input variables were applied to quantify interfacial energy with randomly rough membrane surface. It was found that, RBF ANNs could well capture the complex non-linear relationships between the related factors and interfacial energy. RBF ANN quantification showed high regression coefficient and accuracy, suggesting its high capacity to quantify interfacial energy. Compared to at least one-week time consumption of the advanced extensive Derjaguin-Landau-Verwey-Overbeek (XDLVO) approach, quantification by RBF ANNs only took several seconds for a same case, indicating the high efficiency of RBF ANNs. Moreover, the abilities of RBF ANNs can be further improved. The robust RBF ANNs proposed paved a new way to study membrane fouling in MBRs.

Stochastic-Based Synapse and Soft-Limiting Neuron with Spintronic Devices for Low Power and Robust Artificial Neural Networks

We propose an innovative stochastic-based computing architecture to implement low-power and robust artificial neural network (S-ANN) with both magnetic tunneling junction (MTJ) and Domain Wall (DW) devices. Our mixed-model HSPICE simulation results have shown that, for a well-known pattern recognition task, a 34-neuron S-ANN implementation achieves more than 1.5 orders of magnitude lower energy consumption and 2.5 orders of magnitude less hidden layer chip area, when compared with its deterministic-based ANN counterparts which are implemented with digital and analog CMOS circuits. We believe that our S-ANN architecture achieves such a remarkable performance gain by leveraging two key ideas. First, because all neural signals are encoded as random bit streams, the standard weighted-sum synapses can be accomplished by stochastic bit writing and reading procedure. Second, we designed and implemented a novel multiple-phase pumping circuit structure to effectively realize the soft-limiting neural transfer function that is essential to improve the overall ANN capability and reduce its network complexity.

Median-Pi artificial neural network for forecasting

Datasets with outliers can be predicted with robust learning methods or robust artificial neural networks. In robust artificial neural networks, the architectures become robust by using robust statistics as aggregation functions. Median neural network and trimmed mean neural network are two robust artificial neural networks used in the literature. In these robust artificial neural networks, median and trimmed mean statistics are used as aggregation functions. In this study, Median-Pi artificial neural network is proposed as a new robust neural network for the purpose of forecasting. In Median-Pi artificial neural network, median and multiplicative functions are used as aggregation functions. Because of using median, the proposed network can produce good results for data with outliers. The Median-Pi artificial neural network is trained by particle swarm optimization. The performance of the neural network is investigated by using datasets from the International Time Series Forecast Competition 2016 (CIF-2016). The performance of the proposed method in case of outlier is compared to some other artificial neural networks. Median neural network, trimmed mean neural network, Pi-Sigma neural network and the proposed robust network are applied to time series with outlier, and the obtained results are compared. According to application results, the proposed Median-Pi artificial neural network can produce better forecast results than the other network types.

Application of RBF neural networks for real-time pressure prediction in a Diesel engine

This study aims at building efficient and robust artificial neural networks (ANN) in order to reconstruct the in-cylinder pressure of a Diesel engine starting from the signal of a low-cost accelerometer placed on the engine block. The accelerometer is a perfect non-intrusive replacement for expensive probes and is prospectively suitable for production vehicles. In this view, the ANN is meant to be efficient in terms of response time. In addition, robustness is sought in order to provide flexibility in terms of operation parameters.The neural network considered here is based on radial basis functions (RBF). The network is trained using measurements from a single cylinder Diesel engine operating under varying conditions. Training data are composed of time series from the accelerometer and corresponding measured in-cylinder pressure signals. The network parameters, including the spread parameter of the radial basis functions and the number of neurons, are used to optimize the network quality. The RBF network is then validated.The results show good correspondence between the measured and the simulated pressure signal. The accuracy of the simulated pressure signals is analyzed in terms of mean square error and in terms of a number of parameters, such as pressure peak and mass burned fraction (MBF), and their angular locations. Robustness is sought with respect to changes in the engine parameters and in the nature of the fuel. The encouraging results indicate that the reconstruction model based on RBF neural networks can be incorporated in the design of fuel-independent real-time control of Diesel engines.

Efficient construction of robust artificial neural networks for accurate determination of superficial sample optical properties

In general, diffuse reflectance spectroscopy (DRS) systems work with photon diffusion models to determine the absorption coefficient μa and reduced scattering coefficient μs' of turbid samples. However, in some DRS measurement scenarios, such as using short source-detector separations to investigate superficial tissues with comparable μa and μs', photon diffusion models might be invalid or might not have analytical solutions. In this study, a systematic workflow of constructing a rapid, accurate photon transport model that is valid at short source-detector separations (SDSs) and at a wide range of sample albedo is revealed. To create such a model, we first employed a GPU (Graphic Processing Unit) based Monte Carlo model to calculate the reflectance at various sample optical property combinations and established a database at high speed. The database was then utilized to train an artificial neural network (ANN) for determining the sample absorption and reduced scattering coefficients from the reflectance measured at several SDSs without applying spectral constraints. The robustness of the produced ANN model was rigorously validated. We evaluated the performance of a successfully trained ANN using tissue simulating phantoms. We also determined the 500-1000 nm absorption and reduced scattering spectra of in-vivo skin using our ANN model and found that the values agree well with those reported in several independent studies.

The prediction of human intestinal absorption based on the molecular structure.

Human Intestinal Absorption (HIA) has been modeled many times by using classification models. However, regression models are scarce. Here, Artificial Neural Networks (ANNs) are implemented for this purpose. A dataset of structurally diverse chemicals with their respective experimental HIA were used to design robust, true predictive and widespread applicable ANN models. An input variables pool was made up of structural invariants calculated by using either Dragon or our software Desmol 1. The selection of best variables was performed following three steps using the entire dataset of molecules. Firstly, variables poorly correlated with the experimental data were eliminated. Secondly, input variable selection was performed by stepwise multilinear regression. Thirdly, correlation matrix in the set of selected variables was then obtained to eliminate those variables strongly intercorrelated. Backpropagation ANNs were trained for these variables finally selected as inputs, and HIA as output. The training and selection procedure to find robust models consisted of randomly partitioning the dataset into three sets: training set, with 50% of the population, test set with 25%, and validation set with the other 25%. With each partitioning, diverse numbers of hidden nodes were assayed to optimize the performance in the prediction for the three sets. Models with r(2) greater than 0.6 for the three sets were considered as robust. A randomization test following all these steps was performed, and the poor results obtained confirm the validity of the method presented in this paper to predict HIA for datasets of structurally diverse organic compounds.

GSM Speech Coder Indirect Identification Algorithm

This paper presents GSM speech coder indirect identification algorithm based on sending novel identification pilot signals through the GSM speech channel. Each GSM subsystem disturbs identification pilot, while speech coder uniquely changes the tempo-spectral characteristics of the proposed pilot signal. Speech coder identification algorithm identifies speech coder with the usage of robust linear frequency cepstral coefficient (LFCC) feature extraction procedure and fast artificial neural networks. First step of speech coder identification algorithm is the exact position detection of the identification pilot signal using normalized cross correlation approach. Next stage is time-domain windowing of the input signal to convolve each frame of the input speech signal and window spectrum. Consecutive step is a short-time Fast Fourier Transformation to produce the magnitude spectrum of each windowed frame. Further, a noise reduction with spectral subtraction based on spectral smoothing is carried out. In last steps we perform the frequency filtering and Discrete Cosine Transformation to receive 24 uncorrelated cepstral coefficients per frame as a result. Speech coder identification is completed with fast artificial neural network classification using the input feature vector of 24 LFCC coefficients, giving a result of identified speech coder. For GSM speech coder indirect identification evaluation, the standardized GSM ETSI bit-exact implementations were used. Furthermore, a set of custom tools was build. These tools were used to simulate and control various conditions in the GSM system. Final results show that proposed algorithm identifies the GSM-EFR speech coder with the accuracy of 98.85%, the GSM-FR speech coder with 98.71%, and the GSM-HR coder with 98.61%. These scores were achieved at various types of surrounding noises and even at very low SNR conditions.

Robust Artificial Neural Networks for Pricing of European Options

The option pricing ability of Robust Artificial Neural Networks optimized with the Huber function is compared against those optimized with Least Squares. Comparison is in respect to pricing European call options on the S&P 500 using daily data for the period April 1998 to August 2001. The analysis is augmented with the use of several historical and implied volatility measures. Implied volatilities are the overall average, and the average per maturity. Beyond the standard neural networks, hybrid networks that directly incorporate information from the parametric model are included in the analysis. It is shown that the artificial neural network models with the use of the Huber function outperform the ones optimized with least squares.

Incorporating Neural Network Material Models Within Finite Element Analysis for Rheological Behavior Prediction

The accuracy of a finite element model for design and analysis of a metal forging operation is limited by the incorporated material model’s ability to predict deformation behavior over a wide range of operating conditions. Current rheological models prove deficient in several respects due to the difficulty in establishing complicated relations between many parameters. More recently, artificial neural networks (ANN) have been suggested as an effective means to overcome these difficulties. To this end, a robust ANN with the ability to determine flow stresses based on strain, strain rate, and temperature is developed and linked with finite element code. Comparisons of this novel method with conventional means are carried out to demonstrate the advantages of this approach.