Simple Artificial Neural Network Research Articles

Abrupt and spontaneous strategy switches emerge in simple regularised neural networks.

Humans sometimes have an insight that leads to a sudden and drastic performance improvement on the task they are working on. Sudden strategy adaptations are often linked to insights, considered to be a unique aspect of human cognition tied to complex processes such as creativity or meta-cognitive reasoning. Here, we take a learning perspective and ask whether insight-like behaviour can occur in simple artificial neural networks, even when the models only learn to form input-output associations through gradual gradient descent. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that included a hidden regularity to solve the task more efficiently. Our results show that only some humans discover this regularity, whose behaviour was marked by a sudden and abrupt strategy switch that reflects an aha-moment. Notably, we find that simple neural networks with a gradual learning rule and a constant learning rate closely mimicked behavioural characteristics of human insight-like switches, exhibiting delay of insight, suddenness and selective occurrence in only some networks. Analyses of network architectures and learning dynamics revealed that insight-like behaviour crucially depended on a regularised gating mechanism and noise added to gradient updates, which allowed the networks to accumulate "silent knowledge" that is initially suppressed by regularised gating. This suggests that insight-like behaviour can arise from gradual learning in simple neural networks, where it reflects the combined influences of noise, gating and regularisation. These results have potential implications for more complex systems, such as the brain, and guide the way for future insight research.

Oscillations in an artificial neural network convert competing inputs into a temporal code.

The field of computer vision has long drawn inspiration from neuroscientific studies of the human and non-human primate visual system. The development of convolutional neural networks (CNNs), for example, was informed by the properties of simple and complex cells in early visual cortex. However, the computational relevance of oscillatory dynamics experimentally observed in the visual system are typically not considered in artificial neural networks (ANNs). Computational models of neocortical dynamics, on the other hand, rarely take inspiration from computer vision. Here, we combine methods from computational neuroscience and machine learning to implement multiplexing in a simple ANN using oscillatory dynamics. We first trained the network to classify individually presented letters. Post-training, we added temporal dynamics to the hidden layer, introducing refraction in the hidden units as well as pulsed inhibition mimicking neuronal alpha oscillations. Without these dynamics, the trained network correctly classified individual letters but produced a mixed output when presented with two letters simultaneously, indicating a bottleneck problem. When introducing refraction and oscillatory inhibition, the output nodes corresponding to the two stimuli activate sequentially, ordered along the phase of the inhibitory oscillations. Our model implements the idea that inhibitory oscillations segregate competing inputs in time. The results of our simulations pave the way for applications in deeper network architectures and more complicated machine learning problems.

When linear inversion fails: Neural-network optimization for sparse-ray travel-time tomography of a volcanic edifice

In this study, we present an artificial neural network (ANN)-based approach for travel-time tomography of a volcanic edifice under sparse-ray coverage. We employ ray tracing to simulate the propagation of seismic waves through the heterogeneous medium of a volcanic edifice, and an inverse modeling algorithm that uses an ANN to estimate the velocity structure from the “observed” travel-time data. The performance of the approach is evaluated through a 2-dimensional numerical study that simulates i) an active source seismic experiment with a few (explosive) sources placed on one side of the edifice and a dense line of receivers placed on the other side, and ii) earthquakes located inside the edifice with receivers placed on both sides of the edifice. The results are compared with those obtained from conventional damped linear inversion. The average Root Mean Square Error (RMSE) between the input and output models is approximately 0.03 km/s for the ANN inversions, whereas it is about 0.4 km/s for the linear inversions, demonstrating that the ANN-based approach outperforms the classical approach, particularly in situations with sparse ray coverage. Our study emphasizes the advantages of employing a relatively simple ANN architecture in conjunction with second-order optimizers to minimize the loss function. Compared to using first-order optimizers, our ANN architecture shows a ∼25% reduction in RMSE. The ANN-based approach is computationally efficient. We observed that even though the ANN is trained based on completely random velocity models, it is still capable of resolving previously unseen anomalous structures within the edifice with about 5% anomalous discrepancies, making it a potentially valuable tool for the detection of low velocity anomalies related to magmatic intrusions or mush.

Prediction of Accident Risk Levels in Traffic Accidents Using Deep Learning and Radial Basis Function Neural Networks Applied to a Dataset with Information on Driving Events

A complex AI system must be worked offline because the training and execution phases are processed separately. This process often requires different computer resources due to the high model requirements. A limitation of this approach is the convoluted training process that needs to be repeated to obtain models with new data continuously incorporated into the knowledge base. Although the environment may be not static, it is crucial to dynamically train models by integrating new information during execution. In this article, artificial neural networks (ANNs) are developed to predict risk levels in traffic accidents with relatively simpler configurations than a deep learning (DL) model, which is more computationally intensive. The objective is to demonstrate that efficient, fast, and comparable results can be obtained using simple architectures such as that offered by the Radial Basis Function neural network (RBFNN). This work led to the generation of a driving dataset, which was subsequently validated for testing ANN models. The driving dataset simulated the dynamic approach by adding new data to the training on-the-fly, given the constant changes in the drivers’ data, vehicle information, environmental conditions, and traffic accidents. This study compares the processing time and performance of a Convolutional Neural Network (CNN), Random Forest (RF), Radial Basis Function (RBF), and Multilayer Perceptron (MLP), using evaluation metrics of accuracy, Specificity, and Sensitivity-recall to recommend an appropriate, simple, and fast ANN architecture that can be implemented in a secure alert traffic system that uses encrypted data.

CRYOMOVE: Cold chain real-time management of vaccine delivery using PCM and deep learning

Vaccines are temperature-sensitive biological products that can become ineffective or even hazardous if exposed to temperatures outside the recommended range. Therefore, it is crucial to maintain the cold chain during the transportation and storage of vaccines in order to ensure that they arrive at their destination in optimal condition. Vaccines are typically transported and stored in containers lined with Phase Change Material (PCM) to maintain a specific temperature range (usually lower than the ambient temperature). Firstly, we have simulated the melting of PCM at three distinct external temperatures using ANSYS Fluent software in order to forecast the Melt Fraction (MF) vs. time curve for each of these temperatures. The external temperature consistently undergoes dynamic changes. Running a simulation each time the temperature shifts is impractical due to its time and data-intensive nature. Consequently, we adopted a more efficient approach by training a simple Artificial Neural Network (ANN) based on simulation data. This ANN is designed to learn and predict the MF versus time curve for any given external temperature. Real-time temperature data from the vaccine box is transmitted to the cloud. Subsequently, the machine learning model operates on the cloud to predict the remaining time for the PCM to melt. This predictive analysis is performed recursively every 10 min to account for dynamic fluctuations in external temperatures, providing continuous updates on the time remaining for PCM meltdown. This is then conveyed to the cold chain managers to take preventive measures if the external conditions are harsh and there is a chance for vaccines to get ruined because of the temperature. This same approach can be extended to other applications beyond vaccine delivery such as food storage and transportation, pharmaceuticals, and more.

Back Propagation Artificial Neural Network Based DC Bus Capacitance Identification Method in Three-Phase PWM Rectifier for Charging System of EVs

In order to improve the performance and reliability of the three-phase PWM rectifier in electric vehicle (EV) charging system, it is important to focus on the identification of the DC bus capacitance. The existing methods have the disadvantages such as a low identification accuracy, a high hardware dependency, and high costs. Therefore, the development of new identification methodologies could be the way out of the aforementioned disadvantages, in terms of advanced algorithms. In this paper, a DC bus capacitance identification method based on back propagation (BP) artificial neural network (ANN) algorithm is proposed. The capacitance can be accurately identified in the diode rectification stage, by training the neural network to extract the potential laws in the data, which only includes the Phase-a voltage and current, as well as the DC bus voltage ripple. The regression response is 0.99991. Finally, the experimental results show the accuracy and effectiveness of the proposed method. A simple and effective BP ANN is designed, and it can identify the capacitance at different grid voltages and DC bus loads. It has a high identification accuracy that the maximum identification error is less than 4.5%.

Artificial neural networks with uniform norm-based loss functions

We explore the potential for using a nonsmooth loss function based on the max-norm in the training of an artificial neural network without hidden layers. We hypothesise that this may lead to superior classification results in some special cases where the training data are either very small or the class size is disproportional. Our numerical experiments performed on a simple artificial neural network with no hidden layer appear to confirm our hypothesis.

A Comprehensive 3-Steps Methodology for Vibration-based Fault Detection, Diagnosis and Localization in Rotating Machines

In any industry, it is the requirement to know whether the machine is healthy or not to operate machine further. If the machine is not healthy then what is the fault in the machine and then finally its location. The paper is proposing a 3-steps methodology for the machine fault diagnosis to meet the industrial requirements to aid the maintenance activity. The Step-1 identifies whether machine is healthy or faulty, then Step-2 detect the type of defect and finally its location in Step-3. This method is extended further from the earlier study on the 2-Steps method for the rotor defects only to the 3-Steps methodology to both rotor and bearing defects. The method uses the optimised vibration parameters and a simple Artificial Neural Network (ANN)-based Machine Learning (ML) model from the earlier studies. The model is initially developed, tested and validated on an experimental rotating rig operating at a speed above 1st critical speed. The proposed method and model are then further validated at 2 different operating speeds, one below 1st critical speed and other above 2nd critical speeds. The machine dynamics are expected to be significantly different at these speeds. This highlights the robustness of the proposed 3-Steps method. Conflict of Interest Statement The authors declare no conflicts of interest.

Comparative Analysis of Artificial Neural Network (ANN) and Wavelet Integrated Artificial Neural Network (W-ANN) Approaches for Rainfall Modeling of Southern Rajasthan, India

This paper addresses the challenge of predicting erratic rainfall in Rajasthan state of India, particularly in southern regions. Reliable rainfall predictions are crucial for water resource management and agriculture planning. The research involved selecting 58 stations across seven districts of southern Rajasthan and identifying the best fit computational neural (ANN) and wavelet integrated computational neural (W-ANN) architectures based on performance metrics. Different combinations of input characters, hidden layer neurons, learning algorithms, and training cycles were tested to determine optimal models. Hybrid models, combining wavelet analysis with ANN, were explored to tackle non-stationary hydrologic signals effectively. Results showed that ANN Model C with ten input layer neurons performed best for 74% of stations, followed by Model B (21% of stations) and Model A (5% of stations). Models with increased input and hidden layer neurons performed better. Among the selected stations, 81% of stations demonstrated improved performance using W-ANN models due to effective signal decomposition and information extraction. The hybrid W-ANN models outperformed simple ANN models for rainfall prediction. Both ANN and W-ANN models accurately forecasted weekly rainfall, as observed in the comparison of actual and forecasted values.

Predictive Modeling of Light–Matter Interaction in One Dimension: A Dynamic Deep Learning Approach

The mathematical modeling and the associated numerical simulation of the light–matter interaction (LMI) process are well-known to be quite complicated, particularly for media where several electronic transitions take place under electromagnetic excitation. As a result, numerical simulations of typical LMI processes usually require a high computational cost due to the involvement of a large number of coupled differential equations modeling electron and photon behavior. In this paper, we model the general LMI process involving an electromagnetic interaction medium and optical (light) excitation in one dimension (1D) via the use of a dynamic deep learning algorithm where the neural network coefficients can precisely adapt themselves based on the past values of the coefficients of adjacent layers even under the availability of very limited data. Due to the high computational cost of LMI simulations, simulation data are usually only available for short durations. Our aim here is to implement an adaptive deep learning-based model of the LMI process in 1D based on available temporal data so that the electromagnetic features of LMI simulations can be quickly decrypted by the evolving network coefficients, facilitating self-learning. This enables accurate prediction and acceleration of LMI simulations that can run for much longer durations via the reduction in the cost of computation through the elimination of the requirement for the simultaneous computation and discretization of a large set of coupled differential equations at each simulation step. Our analyses show that the LMI process can be efficiently decrypted using dynamic deep learning with less than 1% relative error (RE), enabling the extension of LMI simulations using simple artificial neural networks.

Predicting and Reconstructing Aerosol–Cloud–Precipitation Interactions with Physics-Informed Neural Networks

Interactions between clouds, aerosol, and precipitation are crucial aspects of weather and climate. The simple Koren–Feingold conceptual model is important for providing deeper insight into the complex aerosol–cloud–precipitation system. Recently, artificial neural networks (ANNs) and physics-informed neural networks (PINNs) have been used to study multiple dynamic systems. However, the Koren–Feingold model for aerosol–cloud–precipitation interactions has not yet been studied with either ANNs or PINNs. It is challenging for pure data-driven models, such as ANNs, to accurately predict and reconstruct time series in a small data regime. The pure data-driven approach results in the ANN becoming a “black box” that limits physical interpretability. We demonstrate how these challenges can be overcome by combining a simple ANN with physical laws into a PINN model (not purely data-driven, good for the small data regime, and interpretable). This paper is the first to use PINNs to learn about the original and modified Koren–Feingold models in a small data regime, including external forcings such as wildfire-induced aerosols or the diurnal cycle of clouds. By adding external forcing, we investigate the effects of environmental phenomena on the aerosol–cloud–precipitation system. In addition to predicting the system’s future, we also use PINN to reconstruct the system’s past: a nontrivial task because of time delay. So far, most research has focused on using PINNs to predict the future of dynamic systems. We demonstrate the PINN’s ability to reconstruct the past with limited data for a dynamic system with nonlinear delayed differential equations, such as the Koren–Feingold model, which remains underexplored in the literature. The main reason that this is possible is that the model is non-diffusive. We also demonstrate for the first time that PINNs have significant advantages over traditional ANNs in predicting the future and reconstructing the past of the original and modified Koren–Feingold models containing external forcings in the small data regime. We also show that the accuracy of the PINN is not sensitive to the value of the regularization factor (λ), a key parameter for the PINN that controls the weight for the physics loss relative to the data loss, for a broad range (from λ=1×103 to λ=1×105).

Interface resistance-switching with reduced cyclic variations for reliable neuromorphic computing

As a synaptic device candidate for artificial neural networks (ANNs), memristors hold great promise for efficient neuromorphic computing. However, commonly used filamentary memristors normally exhibit large cyclic variations due to the stochastic nature of filament formation and ablation, which will inevitably degrade the computing accuracy. Here we demonstrate, in nanoscale Ag2S-based memristors that resistance-switching (RS) at the contact interface can be a promising solution to reduce cyclic variations. When the Ag2S memristor is operated with a filament-free interface RS via Schottky barrier height modification at the contact interface, it shows an ultra-small cycle-to-cycle variation of 1.4% during 104 switching cycles. This is in direct contrast to the variation of (28.9%) of the RS filament extracted from the same device. Interface RS can also emulate synaptic functions and psychological behavior. Its improved learning ability over a filament RS, with a higher saturated accuracy approaching 99.6%, is finally demonstrated in a simplified ANN.

Markov-based Neural Networks for Heart Sound Segmentation: Using domain knowledge in a principled way.

This work considers the problem of segmenting heart sounds into their fundamental components. We unify statistical and data-driven solutions by introducing Markov-based Neural Networks (MNNs), a hybrid end-to-end framework that exploits Markov models as statistical inductive biases for an Artificial Neural Network (ANN) discriminator. We show that an MNN leveraging a simple one-dimensional Convolutional ANN significantly outperforms two recent purely data-driven solutions for this task in two publicly available datasets: PhysioNet 2016 (Sensitivity: 0.947 ±0.02; Positive Predictive Value : 0.937 ±0.025) and the CirCor DigiScope 2022 (Sensitivity: 0.950 ±0.008; Positive Predictive Value: 0.943 ±0.012). We also propose a novel gradient-based unsupervised learning algorithm that effectively makes the MNN adaptive to unseen datum sampled from unknown distributions. We perform a cross dataset analysis and show that an MNN pre-trained in the CirCor DigiScope 2022 can benefit from an average improvement of 3.90% Positive Predictive Value on unseen observations from the PhysioNet 2016 dataset using this method.

Ultraviolet Erythemal Irradiance (UVER) under Different Sky Conditions in Burgos, Spain: Multilinear Regression and Artificial Neural Network Models

Different strategies for modeling Global Horizontal UltraViolet Erythemal irradiance (GHUVE) based on meteorological parameters measured in Burgos (Spain) have been developed. The experimental campaign ran from September 2020 to June 2022. The selection of relevant variables for modeling was based on Pearson’s correlation coefficient. Multilinear Regression Model (MLR) and artificial neural network (ANN) techniques were employed to model GHUVE under different sky conditions (all skies, overcast, intermediate, and clear skies), classified according to the CIE standard on a 10 min basis. ANN models of GHUVE outperform those based on MLR according to the traditional statistical indices used in this study (R2, MBE, and nRMSE). Moreover, the work proposes a simple all-sky ANN model of GHUVE based on usually recorded variables at ground meteorological stations.

How good can a simple artificial neural network predict the medium reorganization energy and the free energy gap from a steady-state fluorescence spectrum?

Medium reorganization energy, Erm, and free energy gap, ΔG, are key characteristics in determining the photoinduced charge transfer rate. They are strongly related to the width and position of the steady-state fluorescence and absorption spectra of organic dyes in solvents. Steady-state spectroscopy is one of the most standard techniques for studying organic dyes. Here, we present a simple machine learning model for cost-effective prediction the medium reorganization energy and the free energy gap from the steady-state fluorescence spectra. Fluorescence spectra with corresponding reference values of physical quantities are calculated using an explicit mathematical expression that addresses electronic-vibrational transitions, intramolecular and solvent reorganizations. An approach to the separate formation of data sets for training and validation is proposed. Gaussian noise is used in training to improve predictions. This study shows that Erm and ΔG can be predicted with the mean absolute error ∼0.024 eV and ∼0.009 eV, respectively. Robustness testing is performed on the experimental spectra of coumarin-153, coumarin-307, and coumarin-522B in a series of solvents.

Neural Network Based Approach to Recognition of Meteor Tracks in the Mini-EUSO Telescope Data

Mini-EUSO is a wide-angle fluorescence telescope that registers ultraviolet (UV) radiation in the nocturnal atmosphere of Earth from the International Space Station. Meteors are among multiple phenomena that manifest themselves not only in the visible range but also in the UV. We present two simple artificial neural networks that allow for recognizing meteor signals in the Mini-EUSO data with high accuracy in terms of a binary classification problem. We expect that similar architectures can be effectively used for signal recognition in other fluorescence telescopes, regardless of the nature of the signal. Due to their simplicity, the networks can be implemented in onboard electronics of future orbital or balloon experiments.

Quantitative and Qualitative Analyses of Mass Spectra of OEL Materials by Artificial Neural Network and Interface Evaluation: Results from a VAMAS Interlaboratory Study.

Quantitative analysis of binary mixtures of tris(2-phenylpyridinato)iridium(III) (Ir(ppy)3) and tris(8-hydroxyquinolinato)aluminum (Alq3) by using an artificial neural network (ANN) system to mass spectra was attempted based on the results of a VAMAS (Versailles Project on Advanced Materials and Standards) interlaboratory study (TW2 A31) to evaluate matrix-effect correction and to investigate interface determination. Monolayers of binary mixtures having different Ir(ppy)3 ratios (0, 0.25, 0.50, 0.75, and 1.00), and the multilayers containing these mixtures and pure samples were measured using time-of-flight secondary ion mass spectrometry (ToF-SIMS) with different primary ion beams, OrbiSIMS (SIMS with both Orbitrap and ToF mass spectrometers), laser desorption ionization (LDI), desorption/ionization induced by neutral clusters (DINeC), and X-ray photoelectron spectroscopy (XPS). The mass spectra were analyzed using a simple ANN with one hidden layer. The Ir(ppy)3 ratios of the unknown samples and the interfaces of the multilayers were predicted using the simple ANN system, even though the mass spectra of binary mixtures exhibited matrix effects. The Ir(ppy)3 ratios at the interfaces indicated by the simple ANN were consistent with the XPS results and the ToF-SIMS depth profiles. The simple ANN system not only provided quantitative information on unknown samples, but also indicated important mass peaks related to each molecule in the samples without a priori information. The important mass peaks indicated by the simple ANN depended on the ionization process. The simple ANN results of the spectra sets obtained by a softer ionization method, such as LDI and DINeC, suggested large ions such as trimers. From the first step of the investigation to build an ANN model for evaluating mixture samples influenced by matrix effects, it was indicated that the simple ANN method is useful for obtaining candidate mass peaks for identification and for assuming mixture conditions that are helpful for further analysis.

Music classification with convolutional and artificial neural network

Music genre classification focus on efficiently finding expected music with a similar genre through numerous melodies, which could better satisfy the tastes and expectations of the users when listening to music. This paper proposes a new method to classify different kinds of music with Artificial Neural networks (ANN) and Convolutional Neural Networks (CNNs). First, Mel Frequency Cepstral Coefficients (MFCC) are used to preprocess the Mel-frequency cepstrum (MFC). Then, we upgrade Anupam’s CNN model. Since the extracted features only by MFC are not suitable for CNN to learn as a small dataset like this. Multiple features are then exacted for each audio file. The two most correlated features on the datasets are adopted as the input of an ANN. To verify the proposed method’s effectiveness, we compare our method with other state-of-the-art methods on the GTZAN dataset. The experimental results show that we can get higher accuracy compared to Anupam. If using only one MFCC feature, Conv-Conv-Pool, a sub-structure that we add two convolutional layers before each max pooling layer, performs better than Conv-Pool, and Conv-Pool performs better than ANN. However, by concatenating another correlated feature, spectral centroid means, which is a measure used in digital signal processing to characterize a spectrum, a simple ANN can have much higher accuracy than the one utilizing only a single MFCC feature with an accuracy of about 94.1%.

Application of Hybrid ANN Techniques for Drought Forecasting in the Semi-Arid Region of India.

The intensity and frequency of diverse hydro-meteorological disasters viz., extreme droughts, severe floods, and cyclones have increasing trends due to unsustainable management of land and water resources, coupled with increasing industrialization, urbanization and climate change. This study focuses on the forecasting of drought using selected Artificial Neural Network (ANN)-based models to enable decision-makers to improve regional water management plans and disaster mitigation/reduction plans. Four ANN models were developed in this study, viz., one conventional ANN model and three hybrid ANN models: (a) Wavelet based-ANN (WANN), (b) Bootstrap based-ANN (BANN), and (c) Wavelet-Bootstrap based-ANN (WBANN). The Standardized Precipitation Evapotranspiration Index (SPEI), the best drought index identified for the study area, was used as a variable for drought forecasting. Three drought indices, such as SPEI-3, SPEI-6 and SPEI-12 respectively representing "short-term", "intermediate-term", and "long-term" drought conditions, were forecasted for 1-month to 3-month lead times for six weather stations over the study area. Both statistical and graphical indicators were considered to assess the performance of the developed models. For the hybrid wavelet model, the performance was evaluated for different vanishing moments of Daubechies wavelets and decomposition levels. The best-performing bootstrap-based model was further used for analysing the uncertainty associated with different drought forecasts. Among the models developed for drought forecasting for 1 to 3 months, the performances of the WANN and WBANN models are superior to the simple ANN and BANN models for the SPEI-3, SPEI-6, and SPEI-12 up to the 3-month lead time. The performance of the WANN and WBANN models is the best for SPEI-12 (MAE = 0.091-0.347, NSE = 0.873-0.982) followed by SPEI-6 (MAE = 0.258-0.593; NSE = 0.487-0.848) and SPEI-3 (MAE = 0.332-0.787, NSE = 0.196-0.825) for all the stations up to 3-month lead time. This finding is supported by the WBANN analyze uncertainties as narrower band width for SPEI-12 (0.240-0.898) as compared to SPEI-6 (0.402-1.62) and SPEI-3 (0.474-2.304). Therefore, the WBANN model is recommended for the early warning of drought events as it facilitates the uncertainty analysis of drought forecasting results.

Comparing the Graphical Features of Simple Artificial Neural Networks and Cortical Development.

Brain development is characterized by changes in connections and information processing complexity. These changes inspire the training process of artificial neural network (ANN), which requires adjusting the neuron weights and biases to enhance efficiency in performing a specific task. In this work, we found affinities in the ratio of positive and negative weights in simple ANNs during training with that of excitatory and inhibitory synapses in the cortex. Additionally, we present a graphical representation of simple ANNs formed by pruning unimportant weights and aligning neurons and connections of different layers. Our findings suggest a strong relationship between the accuracy of simple neural network and graphical representation features, with graphical features at the inflection point resembling the graphical representation of the cortex.