Neural Network Configuration Research Articles

Four-Wheeled Vehicle Sideslip Angle Estimation: A Machine Learning-Based Technique for Real-Time Virtual Sensor Development

In the last few decades, the role of vehicle dynamics control systems has become crucial. In this complex scenario, the correct real-time estimation of the vehicle’s sideslip angle is decisive. Indeed, this quantity is deeply linked to several aspects, such as traction and stability optimization, and its correct understanding leads to the possibility of reaching greater road safety, increased efficiency, and a better driving experience for both autonomous and human-controlled vehicles. This paper aims to estimate accurately the sideslip angle of the vehicle using different neural network configurations. Then, the proposed approach involves using two separate neural networks in a dual-network architecture. The first network is dedicated to estimating the longitudinal velocity, while the second network predicts the sideslip angle and takes the longitudinal velocity estimate from the first network as input. This enables the creation of a virtual sensor to replace the real one. To obtain a reliable training dataset, several test sessions were conducted on different tracks with various layouts and characteristics, using the same reference instrumented vehicle. Starting from the acquired channels, such as lateral and longitudinal acceleration, steering angle, yaw rate, and angular wheel speeds, it has been possible to estimate the sideslip angle through different neural network architectures and training strategies. The goodness of the approach was assessed by comparing the estimations with the measurements obtained from an optical sensor able to provide accurate values of the target variable. The obtained results show a robust alignment with the reference values in a great number of tested conditions. This confirms that the adoption of artificial neural networks represents a reliable strategy to develop real-time virtual sensors for onboard solutions, expanding the information available for controls.

Optimizing the performance of convolutional neural network for enhanced gesture recognition using sEMG

Deep neural networks (DNNs) have demonstrated higher performance results when compared to traditional approaches for implementing robust myoelectric control (MEC) systems. However, the delay induced by optimising a MEC remains a concern for real-time applications. As a result, an optimised DNN architecture based on fine-tuned hyperparameters is required. This study investigates the optimal configuration of convolutional neural network (CNN)-based MEC by proposing an effective data segmentation technique and a generalised set of hyperparameters. Firstly, two segmentation strategies (disjoint and overlap) and various segment and overlap sizes were studied to optimise segmentation parameters. Secondly, to address the challenge of optimising the hyperparameters of a DNN-based MEC system, the problem has been abstracted as an optimisation problem, and Bayesian optimisation has been used to solve it. From 20 healthy people, ten surface electromyography (sEMG) grasping movements abstracted from daily life were chosen as the target gesture set. With an ideal segment size of 200 ms and an overlap size of 80%, the results show that the overlap segmentation technique outperforms the disjoint segmentation technique (p-value < 0.05). In comparison to manual (12.76 ± 4.66), grid (0.10 ± 0.03), and random (0.12 ± 0.05) search hyperparameters optimisation strategies, the proposed optimisation technique resulted in a mean classification error rate (CER) of 0.08 ± 0.03 across all subjects. In addition, a generalised CNN architecture with an optimal set of hyperparameters is proposed. When tested separately on all individuals, the single generalised CNN architecture produced an overall CER of 0.09 ± 0.03. This study's significance lies in its contribution to the field of EMG signal processing by demonstrating the superiority of the overlap segmentation technique, optimizing CNN hyperparameters through Bayesian optimization, and offering practical insights for improving prosthetic control and human–computer interfaces.

A Technique for Creating and Training an Artificial Neural Network to Detect Network Traffic Anomalies

The article presents a technique for creating and training an artificial neural network to recognize network traffic anomalies using relatively small samples of collected data to generate training data. Various data sources for machine learning and approaches to network traffic analysis are considered. There are data format and the method of generating them from the collected network traffic is described, as well as the steps of the methodology in detail. Using the technique, an artificial neural network was created and trained for the task of recognizing anomalies in the network traffic of the ICMP protocol. The results of testing and comparing various artificial neural network configurations and learning conditions for a given task are presented. The artificial neural network trained according to the method was tested on real network traffic. The presented technique can be applied without requiring changes to detect anomalies of various network protocols and network traffic using a suitable parameterizer and data markup.

Ultra-short-term solar forecasting with reduced pre-acquired data considering optimal heuristic configurations of deep neural networks

&lt;abstract&gt; &lt;p&gt;Forecasting solar irradiance, particularly Global Horizontal Irradiance (GHI), has drawn much interest recently due to the rising demand for renewable energy sources. Many works have been proposed in the literature to forecast GHI by incorporating weather or environmental variables. Nevertheless, the expensive cost of the weather station hinders obtaining meteorological data, posing challenges in generating accurate forecasting models. Therefore, this work addresses this issue by developing a framework to reliably forecast the values of GHI even if meteorological data are unavailable or unreliable. It achieves this by leveraging lag observations of GHI values and applying feature extraction capabilities of the deep learning models. An ultra-short-term GHI forecast model is proposed using the Convolution Neural Network (CNN) algorithm, considering optimal heuristic configurations. In addition, to assess the efficacy of the proposed model, sensitivity analysis of different input variables of historical GHI observations is examined, and its performance is compared with other commonly used forecasting algorithm models over different forecasting horizons of 5, 15, and 30 minutes. A case study is carried out, and the model is trained and tested utilizing real GHI data from solar data located in Riyadh, Saudi Arabia. Results reveal the importance of employing historical GHI data in providing precise forecasting outcomes. The developed CNN-based model outperformed in ultra-short-term forecasting, showcasing average root mean square error results across different forecasting horizons: 2.262 W/m&lt;sup&gt;2&lt;/sup&gt; (5min), 30.569 W/m&lt;sup&gt;2&lt;/sup&gt; (15min), and 54.244 W/m&lt;sup&gt;2&lt;/sup&gt; (30min) across varied day types. Finally, the findings of this research can permit GHI to be integrated into the power grid and encourage the development of sustainable energy systems.&lt;/p&gt; &lt;/abstract&gt;

Large Scale Time-Series Representation Learning via Simultaneous Low-and High-Frequency Feature Bootstrapping.

Learning representations from unlabeled time series data is a challenging problem. Most existing self-supervised and unsupervised approaches in the time-series domain fall short in capturing low-and high-frequency features at the same time. As a result, the generalization ability of the learned representations remains limited. Furthermore, some of these methods employ large-scale models like transformers or rely on computationally expensive techniques such as contrastive learning. To tackle these problems, we propose a noncontrastive self-supervised learning (SSL) approach that efficiently captures low-and high-frequency features in a cost-effective manner. The proposed framework comprises a Siamese configuration of a deep neural network with two weight-sharing branches which are followed by low-and high-frequency feature extraction modules. The two branches of the proposed network allow bootstrapping of the latent representation by taking two different augmented views of raw time series data as input. The augmented views are created by applying random transformations sampled from a single set of augmentations. The low-and high-frequency feature extraction modules of the proposed network contain a combination of multilayer perceptron (MLP) and temporal convolutional network (TCN) heads, respectively, which capture the temporal dependencies from the raw input data at various scales due to the varying receptive fields. To demonstrate the robustness of our model, we performed extensive experiments and ablation studies on five real-world time-series datasets. Our method achieves state-of-art performance on all the considered datasets.

Statistical Treatment of Convolutional Neural Network Superresolution of Inland Surface Wind for Subgrid-Scale Variability Quantification

Abstract Machine learning models have been employed to perform either physics-free data-driven or hybrid dynamical downscaling of climate data. Most of these implementations operate over relatively small downscaling factors because of the challenge of recovering fine-scale information from coarse data. This limits their compatibility with many global climate model outputs, often available between ∼50- and 100-km resolution, to scales of interest such as cloud resolving or urban scales. This study systematically examines the capability of a type of superresolving convolutional neural network (SR-CNNs) to downscale surface wind speed data over land from different coarse resolutions (25-, 48-, and 100-km resolution) to 3 km. For each downscaling factor, we consider three convolutional neural network (CNN) configurations that generate superresolved predictions of fine-scale wind speed, which take between one and three input fields: coarse wind speed, fine-scale topography, and diurnal cycle. In addition to fine-scale wind speeds, probability density function parameters are generated through which sample wind speeds can be generated, accounting for the intrinsic stochasticity of wind speed. For assessing generalization to new data, CNN models are tested on regions with different topography and climate that are unseen during training. The evaluation of superresolved predictions focuses on subgrid-scale variability and the recovery of extremes. Models with coarse wind and fine topography as inputs exhibit the best performance when compared with other model configurations, operating across the same downscaling factor. Our diurnal cycle encoding results in lower out-of-sample generalizability when compared with other input configurations.

Use of Parallel Explanatory Models to Enhance Transparency of Neural Network Configurations for Cell Degradation Detection.

In a previous paper, we have shown that a recurrent neural network (RNN) can be used to detect cellular network radio signal degradations accurately. We unexpectedly found, though, that accuracy gains diminished as we added layers to the RNN. To investigate this, in this article, we build a parallel model to illuminate and understand the internal operation of neural networks (NNs), such as the RNN, which store their internal state to process sequential inputs. This model is widely applicable in that it can be used with any input domain where the inputs can be represented by a Gaussian mixture. By looking at RNN processing from a probability density function (pdf) perspective, we are able to show how each layer of the RNN transforms the input distributions to increase detection accuracy. At the same time we also discover a side effect acting to limit the improvement in accuracy. To demonstrate the fidelity of the model, we validate it against each stage of RNN processing and output predictions. As a result, we have been able to explain the reasons for RNN performance limits with useful insights for future designs for RNNs and similar types of NN.

Comparative analysis of feed-forward neural network and second-order polynomial regression in textile wastewater treatment efficiency

&lt;abstract&gt;&lt;p&gt;This study refines a single-layer Feed-Forward Neural Network (FFNN) for the treatment of textile dye wastewater, concentrating on percentage decolorization (%DEC) and percentage chemical oxygen demand (%COD) reduction. The optimized neural network configuration comprises four input and one output neuron, fine-tuned based on the mean squared error (MSE). The training phase demonstrates a consistent MSE decline, reaching its lowest at epoch 209 for %DEC and epoch 34 for %COD, with corresponding MSEs of $1.799 \times 10^{-5}$ and $ 1.4 \times 10^{-3} $, respectively. The maximum absolute errors for %DEC and %COD were found to be $ 4.0787 $ and $ 2.4486 $, while the mean absolute errors were $ 0.4821 $ and $ 0.7256 $, respectively. In contrast to second-degree polynomial regression, the FFNN model exhibits enhanced predictive accuracy, as indicated by higher $ R^2 $ values of $ 0.99363 $ for %DEC and $ 0.99716 $ for %COD, and reduced error metrics.&lt;/p&gt;&lt;/abstract&gt;

Optimal configuration of the artificial neural network to estimate the speed of the BLDC motor

Optimal configuration of the artificial neural network to estimate the speed of the BLDC motor

Dynamic Behavior Forecast of an Experimental Indirect Solar Dryer Using an Artificial Neural Network

This research presents the prediction of temperatures in the chamber of a solar dryer using artificial neural networks (ANN). The dryer is a forced-flow type and indirect. Climatic conditions, temperatures, airflow, and geometric parameters were considered to build the ANN model. The model was a feed-forward network trained using a backpropagation algorithm and Levenberg–Marquardt optimization. The configuration of the optimal neural network to carry out the verification and validation processes was nine neurons in the input layer, one in the output layer, and two hidden layers of thirteen and twelve neurons each (9-13-12-1). The percentage error of the predictive model was below 1%. The predictive model has been successfully tested, achieving a predictor with good capabilities. This consistency is reflected in the relative error between the predicted and experimental temperatures. The error is below 0.25% for the model’s verification and validation. Moreover, this model could be the basis for developing a powerful real-time operation optimization tool and the optimal design for indirect solar dryers to reduce cost and time in food-drying processes.

Transformer-Based Representation of Organic Molecules for Potential Modeling of Physicochemical Properties.

In this work, we study the use of three configurations of an autoencoder neural network to process organic substances with the aim of generating meaningful molecular descriptors that can be employed to develop property prediction models. A total of 18,322,500 compounds represented as SMILES strings were used to train the model, demonstrating that a latent space of 24 units is able to adequately reconstruct the data. After AE training, an analysis of the latent space properties in terms of compound similarity was carried out, indicating that this space possesses desired properties for the potential development of models for forecasting physical properties of organic compounds. As a final step, a QSPR model was developed to predict the boiling point of chemical substances based on the AE descriptors. 5276 substances were used for the regression task, and the predictive ability was compared with models available in the literature evaluated on the same database. The final AE model has an overall error of 1.40% (1.39% with augmented SMILES) in the prediction of the boiling temperature, while other models have errors between 2.0 and 3.2%. This shows that the SMILES representation is comparable and even outperforms the state-of-the-art representations widely used in the literature.

MaTPIP: A deep-learning architecture with eXplainable AI for sequence-driven, feature mixed protein-protein interaction prediction

Background and ObjectiveProtein-protein interaction (PPI) is a vital process in all living cells, controlling essential cell functions such as cell cycle regulation, signal transduction, and metabolic processes with broad applications that include antibody therapeutics, vaccines, and drug discovery. The problem of sequence-based PPI prediction has been a long-standing issue in computational biology. MethodsWe introduce MaTPIP, a cutting-edge deep-learning framework for predicting PPI. MaTPIP stands out due to its innovative design, fusing pre-trained Protein Language Model (PLM)-based features with manually curated protein sequence attributes, emphasizing the part-whole relationship by incorporating two-dimensional granular part (amino-acid) level features and one-dimensional whole-level (protein) features. What sets MaTPIP apart is its ability to integrate these features across three different input terminals seamlessly. MatPIP also includes a distinctive configuration of Convolutional Neural Network (CNN) with Transformer components for concurrent utilization of CNN and sequential characteristics in each iteration and a one-dimensional to two-dimensional converter followed by a unified embedding. The statistical significance of this classifier is validated using McNemar's test. ResultsMaTPIP outperformed the existing methods on both the Human PPI benchmark and cross-species PPI testing datasets, demonstrating its immense generalization capability for PPI prediction. We used seven diverse datasets with varying PPI target class distributions. Notably, within the novel PPI scenario, the most challenging category for Human PPI Benchmark, MaTPIP improves the existing state-of-the-art score from 74.1% to 78.6% (measured in Area under ROC Curve), from 23.2% to 32.8% (in average precision) and from 4.9% to 9.5% (in precision at 3% recall) for 50%, 10% and 0.3% target class distributions, respectively. In cross-species PPI evaluation, hybrid MaTPIP establishes a new benchmark score (measured in Area Under precision-recall curve) of 81.1% from the previous 60.9% for Mouse, 80.9% from 56.2% for Fly, 78.1% from 55.9% for Worm, 59.9% from 41.7% for Yeast, and 66.2% from 58.8% for E.coli. Our eXplainable AI-based assessment reveals an average contribution of different feature families per prediction on these datasets. ConclusionsMaTPIP mixes manually curated features with the feature extracted from the pre-trained PLM to predict sequence-based protein-protein association. Furthermore, MaTPIP demonstrates strong generalization capabilities for cross-species PPI predictions.

Advanced Analysis of Electrodermal Activity Measures to Detect the Onset of ON State in Parkinson’s Disease

Background: Electrodermal activity (EDA) serves as a prominent biosignal for assessing sympathetic activation across various scenarios. Prior research has suggested a connection between EDA and fluctuations in Parkinson’s disease (PD), but its precise utility in reliably detecting these fluctuations has remained unexplored. This study aims to evaluate the efficacy of both basic and advanced analyses of EDA changes in identifying the transition to the ON state following dopaminergic medication administration in individuals with PD. Methods: In this observational study, 19 individuals with PD were enrolled. EDA was continuously recorded using the Empatica E4 device, worn on the wrist, during the transition from the OFF state to the ON state following levodopa intake. The raw EDA signal underwent preprocessing and evaluation through three distinct approaches. A logistic regression model was constructed to assess the significance of variables predicting the ON/OFF state, and support vector machine (SVM) models along with various Neural Network (NN) configurations were developed for accurate state prediction. Results: Differences were identified between the ON and OFF states in both the time and frequency domains, as well as through the utilization of convex optimization techniques. SVM and NN models demonstrated highly promising results in effectively distinguishing between the OFF and ON states. Conclusions: Evaluating sympathetic activation changes via EDA measures holds substantial promise for detecting non-motor fluctuations in PD. The SVM algorithm, in particular, yields precise outcomes for predicting these non-motor fluctuation states.

Load-current demand forecasting in substations of urban railway lines

Load-current forecasting is important in the operation and energy management of urban railway lines. It helps control strategies to manage and distribute energy optimally, thereby saving energy and reducing voltage fluctuations. This paper presents a method to predict the traction current at the busbar of a substation using the supervised machine learning algorithm. Because the traction power load is supplied from both adjacent traction power stations, and the energy exchange process between trains also takes place during work, the input data are selected to combine the value history of busbar currents and feeder currents. Besides, the neural network configuration and the number of training cycles in the estimated model can be adjusted to achieve the desired accuracy. The proposed method was tested and adjusted based on the actual operation data at the Lang traction station on 24 June, 2022. The estimated results compared with measurement data from the supervisory control and data acquisition (SCADA) have proven that the largest absolute error is no more than 5 (A). The maximum relative error is not more than 0.005, and the mean squared error does not exceed 0.01 over the whole operating time of a day from 4h45 to 22h45.

Universal adaptive optics for microscopy through embedded neural network control

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution – one that can be readily transferred between microscope modalities – has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a “black box”, but provided physical insights on internal workings, which could influence future designs.

Retracted: Software Development, Configuration, Monitoring, and Management of Artificial Neural Networks

Retracted: Software Development, Configuration, Monitoring, and Management of Artificial Neural Networks

Optimal artificial neural network configurations for hourly solar irradiation estimation

&lt;span lang="EN-US"&gt;Solar energy is widely used in order to generate clean electric energy. However, due to its intermittent nature, this resource is only inserted in a limited way within the electrical networks. To increase the share of solar energy in the energy balance and allow better management of its production, it is necessary to know precisely the available solar potential at a fine time step to take into account all these stochastic variations. In this paper, a comparison between different artificial neural network (ANN) configurations is elaborated to estimate the hourly solar irradiation. An investigation of the optimal neurons and layers is investigated. To this end, feedforward neural network, cascade forward neural network and fitting neural network have been applied for this purpose. In this context, we have used different meteorological parameters to estimate the hourly global solar irirradiation in the region of Laghouat, Algeria. The validation process shows that choosing the cascade forward neural network two inputs gives an R2 value equal to 97.24% and an normalized root mean square error (NRMSE) equals to 0.1678 compared to the results of three inputs, which gives an R2 value equaled to 95.54% and an NRMSE equals to 0.2252. The comparison between different existing methods in literature show the goodness of the proposed models.&lt;/span&gt;

Machine learning-enabled thickness estimation of thin coatings on carbon fibre composites using microwaves

Accurate thickness measurement of thin coatings (typically 50–500 μm) on carbon fibre-reinforced polymer composites is a major challenge in the manufacturing and maintenance processes of modern aircraft. Different from the conventional material-dependent technique for prediction, a machine learning-enabled strategy with an artificial neural network configuration is used with no requirement of prior knowledge of the type of coating or substrate under test. In the test, an open microwave cavity resonator sensor is directly placed on a coated composite, and any variation of the coating material, coating thickness and conductivity of the composite alters the resonance frequency. Principal component analysis is employed in the signal pre-processing for the dimensionality reduction of the raw measurement data. In terms of the root-mean-square error, the maximum value for the calibration approach is approximately 15 μm and that for the machine learning-based approach is 12 μm. The sensor system developed enables real-time on-site assessment of coated composite structures and thus offers a new approach for non-destructive evaluation 4.0 with improved efficiency, accuracy and automation.

Short-Term Natural Gas and Carbon Price Forecasting Using Artificial Neural Networks

Methods of computational intelligence show a high potential for short-term price forecasting of the energy market as they offer the possibility to cope with the complexity, multi-parameter dependency, and non-linearity of pricing mechanisms. While there is a large number of publications applying neural networks to the prediction of electricity prices, the analysis of natural gas and carbon prices remains scarce. Identifying a best practice from the literature, this study presents an iterative approach to optimize both the input values and network configuration of neural networks. We apply the approach to the natural gas and carbon market, sequentially testing autoregressive and exogenous explanatory variables as well as different neural network architectures. We subsequently discuss the influence of architectural properties, input parameters, data preparation, and the models’ resilience to singular events. Results show that the selection of appropriate lags of gas and carbon prices to account for autoregressive properties of the respective time series leads to a high degree of forecasting accuracy. Additionally, including ambient temperature data can slightly reduce errors of natural gas price forecasting whereas carbon price predictions benefit from electricity prices as a further explanatory input. The best configurations presented in this contribution achieve a root mean square error (RMSE) of 0.64 EUR/MWh (natural gas prices) corresponding to a normalized RMSE of 0.037 and 0.33 EUR/tCO2 (carbon prices) corresponding to a normalized RMSE of 0.023.

Deep learning models for real-life human activity recognition from smartphone sensor data

Nowadays, the field of human activity recognition (HAR) is a remarkably hot topic within the scientific community. Given the low cost, ease of use and high accuracy of the sensors from different wearable devices and smartphones, more and more researchers are opting to do their bit in this area. However, until very recently, all the work carried out in this field was done in laboratory conditions, with very few similarities with our daily lives. This paper will focus on this new trend of integrating all the knowledge acquired so far into a real-life environment. Thus, a dataset already published following this philosophy was used. In this way, this work aims to be able to identify the different actions studied there. In order to perform this classification, this paper explores new designs and architectures for models inspired by the ones which have yielded the best results in the literature. More specifically, different configurations of Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) have been tested, but on real-life conditions instead of laboratory ones. It is worth mentioning that the hybrid models formed from these techniques yielded the best results, with a peak accuracy of 94.80% on the dataset used.