Artificial Neural Network Learning Research Articles

Realizing reliable linearity and forming-free property in conductive bridging random access memory synapse by alloy electrode engineering

The linearity of conductance modulation of the artificial synapse severely restricts the recognition accuracy and the convergence rate in the learning of artificial neural networks. In this work, by alloy electrode engineering, a Ti–Ag device gained the forming-free property because Ag ions were promoted to migrate into the GeTeOx layer to form a thicker conductive filament. This facilitated a uniform change in conductance with the pulse number, and the alloy synapse achieved a significant improvement in linearity (350%), which demonstrated its enhancement in recognition accuracy. To further validate its potential as a comprehensive artificial synapse, the multi-essential synaptic behaviors, including spike-timing-dependent plasticity, spike-rate-dependent plasticity, paired-pulse facilitation, post-tetanic potentiation, and excitatory post-synaptic current, were achieved successfully. This work proposes a promising approach to enhance the performance of conductive bridging random access memory synaptic devices, which benefits the hardware implementation of neuromorphic systems.

Recent Developments in Machine Learning for Mass Spectrometry.

Statistical analysis and modeling of mass spectrometry (MS) data have a long and rich history with several modern MS-based applications using statistical and chemometric methods. Recently, machine learning (ML) has experienced a renaissance due to advents in computational hardware and the development of new algorithms for artificial neural networks (ANN) and deep learning architectures. Moreover, recent successes of new ANN and deep learning architectures in several areas of science, engineering, and society have further strengthened the ML field. Importantly, modern ML methods and architectures have enabled new approaches for tasks related to MS that are now widely adopted in several popular MS-based subdisciplines, such as mass spectrometry imaging and proteomics. Herein, we aim to provide an introductory summary of the practical aspects of ML methodology relevant to MS. Additionally, we seek to provide an up-to-date review of the most recent developments in ML integration with MS-based techniques while also providing critical insights into the future direction of the field.

Alkali resistance prediction and degradation mechanism of basalt fiber: Integrated with artificial neural network machine learning model

Basalt fiber is commonly regarded as an alkaline corrosion-resistant material, capable of being transformed into various products (such as basalt fiber roving, chopped yarn and composite reinforcement). However, the main composition of the basalt fiber network skeleton consists of Si ∼ O bonds, which can still react with ∼OH in the alkaline environment. Nevertheless, there are differing viewpoints on the alkaline corrosion of basalt fibers within literatures. Therefore, this study aims to select three representative types of basalt fibers and compare them with two alkali-resistant glass fibers and one regular glass fiber. The attenuation mechanism of basalt fibers were investigated through orthogonal experiments, practical application scenarios and microscopic characterization methods. Additionally, we employed the Weibull model to analyze failure probabilities after erosion on basalt fibers while also applying a machine learning artificial neural network (ANN) model for predicting their alkali resistance. The results indicated that the tensile strength of basalt fiber tended to decrease and the alkali resistance of basalt fiber between alkali-resistant glass fiber and regular glass fiber in four alkaline solutions. The Weibull model also confirms that erosion increases failure probability of basalt fiber and glass fiber. Moreover, Si ∼ O, which forms the network reacted with OH-, resulting in a reduction in surface Si and Al elements content. It became possible to predict the alkali resistance of basalt fiber accurately by establishing an ANN model. Our study provide a theoretical foundation for predicting the lifespan of basalt fibers under corrosive conditions while offering guidance for enhancing their alkali resistance.

Spatiotemporal change analysis of LULC using remote sensing and CA-ANN approach in the Hodna basin, NE of Algeria

Spatiotemporal studies of land use/land cover change (LULCC) have become an important issue due to the rapid conversion of landscape and its impact on the environment and ecosystems. This study aims to identify the LULC change's impact on the Hodna basin (26,000 km2) from 2000 to 2020 using Landsat images data and forecast the possible change scenarios to the horizon of 2050. The Maximum Likelihood Supervised Classification (MLSC) was applied to Landsat data between 2000 and 2020 with a five-year interval. The confusion matrix of image classification indicates an overall accuracy of 85% and a kappa coefficients range between 0.82 and 0.96. To forecast the possible change scenarios of each class of LULC a transitional simulation using an artificial neural network (ANN) and machine learning algorithms. While, the cellular automata (CA) algorithm was used to simulate and predict the future LULC for the years 2030, 2040 and 2050 after validation of 2020 LULC map. The overall kappa calculated is 68.5% indicating a good accuracy of the validation process. The results show that for 2000–2020, agricultural lands decreased by 2.16% and sandy sails increased by 9.45%, indicating the expansion in the desertification phenomenon. Over the period 2000 to 2050, bare land decreased by 10% in 2015, recovered by 15% in 2020, and is projected to increase by 8% by 2050, reaching 73%; mixed and dense forest exhibited similar trends, growing from 0.7% in 2000 to 1.66% in 2050 with greater stability; built-up areas in the study zone surged by 150%, from 0.86% in 2000 to 2.5% in 2020, while clear forest remained relatively consistent, covering 4–6% of the surface; agricultural land diverged from bare land due to agricultural investment policies favoring new farms over bare land expansion. Urban sprawl is observed in the major cities like M'sila and Bordj Bou Arreridj and accelerated during the study period. Urban tissue evolved at the expense of agricultural land and focused mainly on slight slope areas and valleys.

Compact Modeling of Advanced Gate-All-Around Nanosheet FETs Using Artificial Neural Network.

As the architecture of logic devices is evolving towards gate-all-around (GAA) structure, research efforts on advanced transistors are increasingly desired. In order to rapidly perform accurate compact modeling for these ultra-scaled transistors with the capability to cover dimensional variations, neural networks are considered. In this paper, a compact model generation methodology based on artificial neural network (ANN) is developed for GAA nanosheet FETs (NSFETs) at advanced technology nodes. The DC and AC characteristics of GAA NSFETs with various physical gate lengths (Lg), nanosheet widths (Wsh) and thicknesses (Tsh), as well as different gate voltages (Vgs) and drain voltages (Vds) are obtained through TCAD simulations. Subsequently, a high-precision ANN model architecture is evaluated. A systematical study on the impacts of ANN size, activation function, learning rate, and epoch (the times of complete pass through the entire training dataset) on the accuracy of ANN models is conducted, and a shallow neural network configuration for generating optimal ANN models is proposed. The results clearly show that the optimized ANN model can reproduce the DC and AC characteristics of NSFETs very accurately with a fitting error (MSE) of 0.01.

Inverse radon transform with deep learning: an application in cardiac motion correction

Objective. This paper addresses performing inverse radon transform (IRT) with artificial neural network (ANN) or deep learning, simultaneously with cardiac motion correction (MC). The suggested application domain is cardiac image reconstruction in emission or transmission tomography where IRT is relevant. Our main contribution is in proposing an ANN architecture that is particularly suitable for this purpose. Approach. We validate our approach with two types of datasets. First, we use an abstract object that looks like a heart to simulate motion-blurred radon transform. With the known ground truth in hand, we then train our proposed ANN architecture and validate its effectiveness in MC. Second, we used human cardiac gated datasets for training and validation of our approach. The gating mechanism bins data over time using the electro-cardiogram (ECG) signals for cardiac motion correction. Main results. We have shown that trained ANNs can perform motion-corrected image reconstruction directly from a motion-corrupted sinogram. We have compared our model against two other known ANN-based approaches. Significance. Our method paves the way for eliminating any need for hardware gating in medical imaging.

A Technique for Creating and Training an Artificial Neural Network to Detect Network Traffic Anomalies

The article presents a technique for creating and training an artificial neural network to recognize network traffic anomalies using relatively small samples of collected data to generate training data. Various data sources for machine learning and approaches to network traffic analysis are considered. There are data format and the method of generating them from the collected network traffic is described, as well as the steps of the methodology in detail. Using the technique, an artificial neural network was created and trained for the task of recognizing anomalies in the network traffic of the ICMP protocol. The results of testing and comparing various artificial neural network configurations and learning conditions for a given task are presented. The artificial neural network trained according to the method was tested on real network traffic. The presented technique can be applied without requiring changes to detect anomalies of various network protocols and network traffic using a suitable parameterizer and data markup.

Recent Advances in Smart Tactile Sensory Systems with Brain‐Inspired Neural Networks

Tactile sensory systems play a vital role in various emerging fields including robotics, prosthetics, and human–machine interfaces. However, traditional tactile sensors are typically designed to detect a single stimulus through a lock‐and‐key mechanism, which poses substantial challenges in the realization of multimodal tactile sensors. To address this issue, the convergence of tactile sensory systems with artificial neural network and machine learning (ML) platforms has been utilized to enhance the capabilities of multimodal sensors and enable signal decoupling/interpretation of mixed tactile stimuli. Herein, recent progress in multimodal sensors that can simultaneously identify various stimuli such as strain, pressure, and temperature is reviewed, providing in‐depth understanding of materials, structures, and methodologies. In addition, accurate interpretation of signals from mixed tactile stimuli under complex conditions remains challenging. This review presents a comprehensive exploration of ML algorithms that mimic human neural networks, discussing their significance in advancing smart sensory systems and improving signal interpretation in complex and dynamic environments.

Many-objective coevolutionary learning algorithm with extreme learning machine auto-encoder for ensemble classifier of feedforward neural networks

In artificial neural network (ANN) learning, empirical risk can be expressed by training error, while structural risk can be expressed by network complexity. Learning from data is often seen as a tradeoff between these two risks. Additionally, balancing training error and validation error can overcome the overfitting problem to some extent. It is known that the network performance is also related to regularization term. In order to consider four factors, i.e. training error, validation error, network complexity and regularization term simultaneously in the training process of a single-hidden layer feedforword neural network (SLFN), a many-objective coevolutionary learning algorithm (MOCELA) integrated with extreme learning machine auto-encoder (ELMAE), called MOCELA-ELMAE is presented. In MOCELA, the non-dominated sorting genetic algorithm III (NSGA-III) is improved for handling the many-objective optimization model with hybrid variables, where binary coding is used for structure learning and real coding is utilized for representing input parameters, referring to all input weights and hidden biases of the AE network. Output parameters of AE, i.e. output weights are analytically calculated by the non-iterative learning rule. The network structure and connection parameters of SLFN are determined based on those of AE. A set of Pareto optimal solutions are eventually collected by the MOCELA-ELMAE, which represents multiple optimal SLFNs. To make the final decision, three best SLFNs with minimum validation errors are selected as the base classifiers for selective ensemble learning. Extensive experiments are implemented on the benchmark classification data sets of UCI machine learning repository, and obvious improvements have been observed when the proposed MOCELA-ELMAE is compared with the NSGA-III based on hybrid coding and completely non-iterative learning of SLFN respectively. The experimental results also illustrate that the MOCELA-ELMAE performs much better than other state-of-the-art learning algorithms on many data sets.

Machine learning, conventional and statistical physics modeling of 2,4-Dichlorophenoxyacetic acid (2,4-D) herbicide removal using biochar prepared from Vateria indica fruit biomass

The adsorption properties of 2,4-Dichlorophenoxyacetic acid (2,4-D) onto biochar, obtained through HCl-assisted hydrothermal carbonization process of Vateria indica fruits (VI-BC), were extensively studied using traditional and statistical physics approaches. The traditional adsorption investigations encompassed kinetics, equilibrium, and thermodynamics studies. Subsequently, the Hill statistical physics model was employed to interpret the mechanism. Also, artificial neural network (ANN) and adaptive neuro-fuzzy inference system (ANFIS) machine learning tools were successfully employed to model the adsorption data wherein both models had high prediction potential (R2 > 0.99). The outcomes demonstrated that the produced VI-BC exhibited remarkable adsorptive traits, having a considerable specific surface area (111.54 m2/g), pore size (5.89 nm), a variety of functional groups, and appropriate attributes for efficiently adsorbing 2,4-D. For 10 mg/L 2,4-D, at pH 2.0 and with 0.3 g/L dose, an impressive 91.67% adsorption efficiency was achieved within a 120-min. Pseudo-second-order model aptly depicted the kinetic behavior of 2,4-D adsorption, while the Freundlich model provided a more accurate representation of the isotherms. 2,4-D maximum adsorption capacity stood at 131.39 mg/g at 303 K. The Hill statistical physics model elucidated that the adsorption primarily occurred via physisorption mechanisms, involving electrostatic attractions, π-π conjugation, and pore filling. This conclusion was further substantiated by post-adsorption characterization of the VI-BC. Thermodynamic analysis indicated that the interactions between VI-BC and 2,4-D were favorable, spontaneous, and exothermic. The calculated low energy of adsorption (1.255 kJ/mol) and ΔH° value (−20.49 kJ/mol) further supported physisorption as the dominant mechanism. In summary, this study underscores the significant potential of the newly developed biochar as a promising alternative material for efficiently removing the 2,4-D herbicide from polluted environments.

An evolutionary crow search algorithm equipped with interactive memory mechanism to optimize artificial neural network for disease diagnosis

Artificial neural network (ANN) is an information processing paradigm that loosely models the thinking patterns of the human brain with specifications such as real-time learning, self-adaption, and self-organization. The learning process of ANNs is complex and tackles shortcomings such as a slow convergence rate, learning time-consuming, and local minimum trapping, especially when using gradient-based optimization techniques. Although many metaheuristics have been proposed to arm ANN and solve these weaknesses, ANN learning still needs solution quality. Therefore, this study proposes an evolutionary crow search algorithm (ECSA) to optimize the hyperparameters of ANNs for diagnosing chronic diseases. ECSA introduces an evolutionary search strategy, self-adaptive adaptive flight length, and an interactive memory mechanism to alleviate the canonical crow search algorithm's shortcomings. The evolutionary search strategy and self-adaptive flight length provide a meaningful search strategy in which crows effectively explore and exploit problem spaces by maintaining population diversity. During the search process, the interactive memory mechanism records the best solution obtained during optimization. The performance of ECSA was evaluated and compared with well-known metaheuristic algorithms in terms of local and global search abilities, local optima avoidance, and convergence speed towards the promising area. Then, the results were statistically analyzed. Ultimately, the effectiveness of an adaptation of ECSA named ECSA-MLP for optimizing hyperparameters of the multilayer perceptron network for diagnosing diseases including coronavirus, breast cancer, diabetes, cardiovascular, cervical cancer, Parkinson's, mammography, and acute inflammation was compared with state-of-the-art competitor algorithms. The experimental results indicated the superiority of ECSA over competitor algorithms in optimizing the network.

Synchronization of Tree Parity Machines Using Nonbinary Input Vectors.

Neural cryptography is the application of artificial neural networks (ANNs) in the subject of cryptography. The functionality of this solution is based on a tree parity machine (TPM). It uses ANNs to perform secure key exchange between network entities. This brief proposes improvements to the synchronization of two TPMs. The improvement is based on learning ANN using input vectors that have a wider range of values than binary ones. As a result, the duration of the synchronization process is reduced. Therefore, TPMs achieve common weights in a shorter time due to the reduction of necessary bit exchanges. This approach improves the security of neural cryptography.

Data and Knowledge Organization for Natural Language Processing: Searching and Identifying Better Arrangements of Texts Based on Multimodal Information Architecture

Processing texts of multiple knowledge areas is a hard task. This article presents an Information Science contribution to natural language processing based on artificial neural networks through data arrangement. An extended concept of Information architecture was used, aggregating a multimodal view of organizing data. The Multimodal Information Architecture definition served as foundations for a five-step procedure to design, analyze and transform data used for artificial neural networks training and learning methods, complementing gaps identified by authors focused on Computer Science implementations. The proposal was validated with three datasets formed by texts coming from 16 knowledge areas. Results obtained through the usage of pre-processed data and raw data where compared. In each of the three datasets, the method identified arrangements which led to better and worst results, separating which corpus samples are more susceptible for knowledge extraction.

Phishing Detection Using Machine Learning: A Model Development and Integration

This study aimed to develop a robust machine learning-based phishing detection system using algorithms such as K-nearest neighbour (KNN), artificial neural network (ANN), and random forest (RF). It utilised datasets from Ariyadasa et al. (2021) and UNB (2016) to discern patterns distinguishing legitimate from phishing websites. Furthermore, an objective was to integrate the optimal model into a Django-based web application, facilitating real-time phishing detection. A comprehensive literature review on phishing detection techniques was also undertaken. Datasets chosen underwent rigorous pre-processing to address missing values and imbalance. Feature selection was achieved manually and automatically using mutual information classification. Three machine learning algorithms, RF, KNN, and ANN, were explored. Their hyper-parameters were optimised using GridSearchCV. Performance results highlighted RF's accuracy at 99.78%, KNN's at 99.67%, and ANN's at 99.11%. While RF and KNN models perfectly identified legitimate websites, ANN showcased an impeccable detection of phishing websites. The RF model, with the highest accuracy, was integrated into a Django application, providing a user interface for real-time phishing detection. All models exhibited high accuracy rates, demonstrating their efficacy in phishing detection. While RF was integrated into the web application for this study, the choice between models depends on specific user or business requirements and priorities. Feedback mechanisms within the Django application further promise refinement in future recommendations. The study provides a foundational step toward enhancing web safety through effective phishing detection.

Определение модуля деформации бинарного композита с помощью искусственной нейронной сети

Using of existing methods of determining the characteristics of soils which are part of current regulatory documents and which are based on the hypothesis of normal character of distribution require considerable time and material costs. According to the results of conducted laboratory researches the hypothesis wasn’t confirmed. In the paper it proposes to use trained artificial neural network for determination of the deformation modulus of binary composite «sand — granules of expanded polystyrene». Thus, it has been proven efficiency proposing method using trained artificial neural network in compare classical regression equation for determination of the deformation modulus of the binary composite. With a confidence probability of P = 95 % the absolute value of the relative error is equal to 11,8 % the proposing learning artificial neural network in 11 times less than the absolute value of the relative error of classical regression equation. Also with a confidence probability of P = 95 % the coefficient of determination is equal to 0,5641 and in 6,6 times less than it of regression equation. Further research will be directed to the selection of the values of the parameters of the artificial neural network program for increase the accuracy of determining the deformation modulus of the binary composite.

Optimizing Artificial Neural Network Learning Using Improved Reinforcement Learning in Artificial Bee Colony Algorithm

Artificial neural networks (ANNs) are widely used machine learning techniques with applications in various fields. Heuristic search optimization methods are typically used to minimize the loss function in ANNs. However, these methods can lead the network to become stuck in local optima, limiting performance. To overcome this challenge, this study introduces an improved optimization approach, the improvement of reinforcement learning in the artificial bee colony (improved R‐ABC) algorithm, to enhance the optimization process for ANNs. The proposed method aims to overcome the limitations of heuristic search and improve the efficiency of weight adjustment in ANNs. This new approach enhances the discovery phase of the traditional R‐ABC by including the parameters of neighboring food sources, augmenting the search capabilities for finding the optimal solution. The performance of the improved R‐ABC was compared with ANNs utilizing backpropagation with stochastic gradient descent (SGD) and Adam optimizers, as well as other swarm intelligence (SI) methods such as particle swarm optimization (PSO) and traditional R‐ABC. The results showed that both PSO and R‐ABC continuously improved the solutions across all benchmark datasets. In the iris dataset, all SI approaches consistently achieved F1‐scores exceeding 0.94, outperforming SGD and Adam. For the other datasets, the SI approach generally outperformed the other optimization methods. The results indicate that when the improved R‐ABC is applied to ANNs, it outperforms heuristic search optimization, especially as the network size expands. Although SGD and Adam achieved faster execution times with TensorFlow, the study suggests that using PSO and improved R‐ABC can improve model accuracy and efficiency. Advanced SI methods enhance the optimization process and increase the ability of ANNs to obtain optimal solutions. Enhanced R‐ABC and PSO algorithms can significantly improve ANN training performance and efficiency, especially in complex and high‐dimensional datasets.

Tilt‐Engineered Molecular‐Scale Selector for Enhanced Learning in Artificial Neural Networks

AbstractMiniaturization of individual selectors in crossbar‐array‐based artificial neural networks is essential for the advancement of the underlying neuromorphic electronics, as it improves learning, recognition, and prediction accuracies. This study proposes a tilt‐engineered molecular‐scale selector comprising a heterostructure of biphenyl‐4‐thiol (OPT2) or 1‐octanethiol (C8) molecular layers and an n‐type two‐dimensional MoS2 monolayer (1L‐MoS2) at an approximate contact radius of 3 nm, which is evaluated via conductive atomic force microscopy under various tip‐loading forces. The molecular tilt configuration controlled by the tip‐loading force is used as a rectifying engineer for the OPT2/1L‐MoS2 and C8/1L‐MoS2 heterojunction accuracies. Rectification ratios and conductance levels are significantly influenced by the molecular backbones and tilt angle. The proposed tilt‐engineered selector can aid in controlling undesired neural signals affecting vector–matrix multiplications and adjusting the switching range compatibility of an integrated synaptic device cell, significantly influencing the pattern recognition accuracy. By controlling the tilt angle, the recognition accuracy on the MNIST dataset increases from 78.65% to 86.45% and from 7.74% to 86.09% when using the OPT2/1L‐MoS2 and C8/1L‐MoS2 selector, respectively. The proposed molecular tilt configuration can be used for developing customized molecular‐scale selectors for crossbar‐array‐based artificial neural networks to improve learning while suppressing undesired neural signals.

Machine learning application for prediction of surgical site infection after posterior cervical surgery.

Surgical site infection (SSI) is one of the most common complications of posterior cervical surgery. It is difficult to diagnose in the early stage and may lead to severe consequences such as wound dehiscence and central nervous system infection. This retrospective study included patients who underwent posterior cervical surgery at The Second Affiliated Hospital and Yuying Childrens Hospital of Wenzhou Medical University from September 2018 to June 2022. We employed several machine learning methods, such as the gradient boosting (GB), random forests (RF), artificial neural network (ANN) and other popular machine learning models. To minimize the variability introduced by random splitting, the results underwent 10-fold cross-validation repeated 10 times. Five measurements were averaged across 10 repetitions with 10-fold cross-validation, the RF model achieved the highest AUROC (0.9916), specificity (0.9890) and precision (0.9759). The GB model achieved the highest sensitivity (0.9535) and the KNN achieved the highest sensitivity (0.9958). The application of machine learning techniques facilitated the development of a precise model for predicting SSI after posterior cervical surgery. This dynamic model can be served as a valuable tool for clinicians and patients to assess SSI risk and prevent it in clinical practice.

Automated Detection of Abnormal Optical Coherence Tomography B-scans Using a Deep Learning Artificial Intelligence Neural Network Platform.

Automated Detection of Abnormal Optical Coherence Tomography B-scans Using a Deep Learning Artificial Intelligence Neural Network Platform.

Semantic speech analysis using machine learning and deep learning techniques: a comprehensive review

Human cognitive functions such as perception, attention, learning, memory, reasoning, and problem-solving are all significantly influenced by emotion. Emotion has a particularly potent impact on attention, modifying its selectivity in particular and influencing behavior and action motivation. Artificial Emotional Intelligence (AEI) technologies enable computers to understand a user's emotional state and respond appropriately. These systems enable a realistic dialogue between people and machines. The current generation of adaptive user interference technologies is built on techniques from data analytics and machine learning (ML), namely deep learning (DL) artificial neural networks (ANN) from multimodal data, such as videos of facial expressions, stance, and gesture, voice, and bio-physiological data (such as eye movement, ECG, respiration, EEG, FMRT, EMG, eye tracking). In this study, we reviewed existing literature based on ML and data analytics techniques being used to detect emotions in speech. The efficacy of data analytics and ML techniques in this unique area of multimodal data processing and extracting emotions from speech. This study analyzes how emotional chatbots, facial expressions, images, and social media texts can be effective in detecting emotions. PRISMA methodology is used to review the existing survey. Support Vector Machines (SVM), Naïve Bayes (NB), Random Forests (RF), Recurrent Neural Networks (RNN), Logistic Regression (LR), etc., are commonly used ML techniques for emotion extraction purposes. This study provides a new taxonomy about the application of ML in SER. The result shows that Long-Short Term Memory (LSTM) and Convolutional Neural Networks (CNN) are found to be the most useful methodology for this purpose.