Neurons In Layer Research Articles

Neurons as Autoencoders.

This letter presents the idea that neural backpropagation is exploiting dendritic processing to enable individual neurons to perform autoencoding. Using a very simple connection weight search heuristic and artificial neural network model, the effects of interleaving autoencoding for each neuron in a hidden layer of a feedforward network are explored. This is contrasted with the equivalent standard layered approach to autoencoding. It is shown that such individualized processing is not detrimental and can improve network learning.

Design of multi-hidden-layer Bayesian neural networks for model updating

Structural health monitoring (SHM) based on artificial neural networks (ANN) has received a lot of attention in recent decades, especially the Bayesian neural network (BNN) approach, which is more robust to noise for training and generalization. Since building a high-quality BNN architecture suitable for a specific task depends highly on user experience, the existing literature generally studies BNNs with simple single-hidden-layer architectures designed by empirical formulas or very few tailor-made algorithms. Multiple hidden layer BNN (MBNN) has a more complex architecture than single hidden layer BNN (SBNN) and is expected to achieve better generalization performance for complex problems. But the corresponding architectural design problem is much more complex, and there is no previous experience in the literature. To achieve this for MBNN, this paper proposes two feasible algorithms to optimize the number of neurons in each hidden layer simultaneously, where a logarithmic evidence metric is introduced to quantitatively characterize the performance of MBNN for a given multi-hidden-layer architecture. The feasibility and effectiveness of the proposed method are verified by the finite element (FE) model updating of both a planar truss model and a real-life pedestrian bridge. The verification results show that the second algorithm is much more computationally efficient for the optimal design of the MBNN architecture and has the potential for practical application.

Comparative analysis of RNN versus IIR digital filtering to optimize resilience to dynamic perturbations in pH sensing for vertical farming

AbstractVertical farming (VF) refers to systems of agriculture where crops are grown in trays stacked vertically by exposing them to artificial light and using sensing technology to improve product quality and yield. In this work, we propose an advanced filtering scheme based on recurrent neural networks (RNNs) and deep learning to enable efficient control strategies for VF applications. We demonstrate that the best RNN model incorporates five neuron layers, with the first and second containing 90 long short‐term memory neurons. The third layer implements one gated recurrent units neuron. The fourth segment incorporates one RNN network, while the output layer is designed by using a single neuron exhibiting a rectified linear activation function. By utilizing this RNN digital filter, we introduce two variations: (1) a scaled RNN model to tune the filter to the signal of interest, and (2) a moving average filter to eliminate harmonic oscillations of the output waveforms. The RNN models are contrasted with conventional digital Butterworth, Chebyshev I, Chebyshev II, and elliptic infinite impulse response (IIR) configurations. The RNN digital filtering schemes avoid introducing unwanted oscillations, which makes them more suitable for VF than their IIR counterparts. Finally, by utilizing the advanced features of scaling of the RNN model, we demonstrate that the RNN digital filter can be pH selective, as opposed to conventional IIR filters.

MODEL OF MAXIMAL WEIGHTS INVERSE CHAINS FOR THE ANALYSIS OF THE INFLUENCE FACTORS OF THE SOFTWARE COMPLEXES SUPPORT

Context. The problem of identification, formation and restoration of the boundaries of influencing factors, lost as a result of the implementation of multi-layer perceptron models into the models of subjective perception of the object of software complexes support, as well as the applied practical problem of primary monitoring of the frequency manifestation of a given influencing factor in the post-real-time mode, is considered. The object of research is the influencing factors of support of software complexes. Objective – the goal of the work is to develop a model of inverse chains of maximum weights for the analysis of influencing factors of the software complexes support. Method. A model of maximal weights inverse chains for the analysis of the influence factors of the software complexes support was developed for the analysis of the influencing factors of the software complexes support. The developed model provides possibility to identify and form feedback chains of maximum weights for the identification and further analysis of influencing factors that are reflected into the results of the object perception (the supported software complex or its support processes), by the relevant subjects of interaction which directly or indirectly interact with it. Results. Results of the resolved applied practical problem of primary monitoring of the frequency manifestation of a given influencing factor in the post-real-time mode have been provided as an example of the applied practical use of the developed model. The output results of the developed models functioning – are the reverse chains of maximum weights. In the future, the results obtained by the developed model are used to solve the applied-scientific problem of identification, formation and restoration of the boundaries of influencing factors, lost as a result of the implementation of the appropriate models of multilayer perceptron inside the models of subjective perception of the software complexes support. So the developed model of maximal weights inverse chains for the analysis of the influence factors of the software complexes support resolves this applied-scientific problem, initially caused by the implementation of the corresponding multilayer perceptron models inside the model of the subjective perception of the object of software complexessupport. The developed model provides the possibility of carrying out a qualitative analysis of the transformation of the input characteristics of the object of support into the output resulting characteristics of its subjective perception. Conclusions. Developed model allows to resolve the described problems. At the same time, the developed model improves the classical understanding of multilayer perceptron artificial neural networks, as it introduces an additional value to the neurons of hidden layers, which (starting from now) are able to perform a completely new role of influencing factors markers, while in the classical understanding of multilayer perceptron artificial neural networks they did not perform any functions other than arithmetic to ensure the possibility of correct learning and functioning of a multilayer perceptron artificial neural networks.

Representational drift in barrel cortex is receptive field dependent.

Cortical populations often exhibit changes in activity even when behavior is stable. How behavioral stability is maintained in the face of such "representational drift" remains unclear. One possibility is that some neurons are more stable than others. We examined whisker touch responses in layers 2-4 of the primary vibrissal somatosensory cortex (vS1) over several weeks in mice stably performing an object detection task with two whiskers. Although the number of touch neurons remained constant, individual neurons changed with time. Touch-responsive neurons with broad receptive fields were more stable than narrowly tuned neurons. Transitions between functional types were non-random: before becoming broadly tuned, unresponsive neurons first passed through a period of narrower tuning. Broadly tuned neurons in layers 2 and 3 with higher pairwise correlations to other touch neurons were more stable than neurons with lower correlations. Thus, a small population of broadly tuned and synchronously active touch neurons exhibits elevated stability and may be particularly important for behavior.

A novel Taguchi-based approach for optimizing neural network architectures: application to elastic short fiber composites

This study presents an innovative application of the Taguchi design of experiment method to optimize the structure of an Artificial Neural Network (ANN) model for the prediction of elastic properties of short fiber reinforced composites. The main goal is to minimize the computational effort required for hyperparameter optimization while enhancing the prediction accuracy. By utilizing a robust experimental design framework, the structure of an ANN model is optimized. This approach involves identifying a combination of hyperparameters that provides optimal predictive accuracy with the fewest algorithmic runs, thereby significantly reducing the required computational effort. The results suggested that the Taguchi-based developed ANN model with three hidden layers, 20 neurons in each hidden layer, elu activation function, Adam optimizer, and a learning rate of 0.001 can predict the anisotropic elastic properties of short fiber reinforced composites with a prediction accuracy of 97.71%. Then, external validation of the proposed ANN model was done using experimental data, and differences of less than 10% were obtained, indicating an appropriate predictive performance of the proposed ANN algorithm. Our findings demonstrate that the Taguchi method not only streamlines the hyperparameter tuning process but also substantially improves the algorithm's performance. These results highlight the potential of the Taguchi method as a powerful tool for optimizing machine learning algorithms, especially in scenarios where computational resources are limited. The implications of this study are far-reaching, offering insights for future research in the optimization of different algorithms for improved accuracies and computational efficiencies.

Jaynes machine: The universal microstructure of deep neural networks

Despite the recent stunning progress in large-scale deep neural network applications, our understanding of their microstructure, ‘energy’ functions, and optimal design remains incomplete. Here, we present a new game-theoretic framework, called statistical teleodynamics, that reveals important insights into these key properties. The optimally robust design of such networks inherently involves computational benefit-cost trade-offs that physics-inspired models do not adequately capture. These trade-offs occur as neurons and connections compete to increase their effective utilities under resource constraints during training. In a fully trained network, this results in a state of arbitrage equilibrium, where all neurons in a given layer have the same effective utility, and all connections to a given layer have the same effective utility. The equilibrium is characterized by the emergence of two lognormal distributions of connection weights and neuronal output as the universal microstructure of large deep neural networks. We call such a network the Jaynes Machine. Our theoretical predictions are shown to be supported by empirical data from seven large-scale deep neural networks. We also show that the Hopfield network and the Boltzmann Machine are the same special case of the Jaynes Machine.

Improving Workplace Safety and Health Through a Rapid Ergonomic Risk Assessment Methodology Enhanced by an Artificial Intelligence System

The comfort of a worker while performing any activity is extremely important. If that activity extends beyond a person’s capacity to withstand physical and psychological stress, the worker may suffer from both physical and mental ailments. Over time, if the stress persists, these conditions can become chronic diseases and can even be the cause of workplace accidents. In this research, a methodology was developed for the rapid assessment of ergonomic risks and for calculating the level of ergonomic comfort in the workplace. This methodology uses artificial intelligence through a specific algorithm and takes into account a number of factors that, when combined, can have a significant impact on workers. To achieve a more accurate simulation of a work situation or to evaluate an ongoing work situation, and to significantly correlate these parameters, we used logarithmic calculation formulas. To streamline the process, we developed software that performs these calculations, conducts a rapid assessment of ergonomic risks, estimates a comfort level, and proposes possible measures to mitigate the risks and effects on workers. To assist in diagnosing the work situation, we used a neural network with five neurons in the input layer, one hidden layer, and two neurons in the output layer. As a result, most work situations, in any industrial field, can be quickly analyzed and evaluated using this methodology. The use of this new analysis and diagnosis tool, implemented through this new research technology, is beneficial for employers and workers. Moreover, through further developments of this methodology, achieved by increasing the number of relevant input parameters for ergonomics and integrating advanced artificial intelligence systems, we aim to provide high precision in assessing ergonomic risk and calculating the level of ergonomic comfort.

Internal Combustion Engine Fault Detection Based on Random Convolutional Neural Networks

The internal combustion engine plays a very important role in many fields, and when the internal combustion engine fails, if it is not found in time, it may cause continuous damage to the internal combustion engine, and further affect the life of the entire mechanical system. To solve the above problems, this paper proposes fault detection of internal combustion engines based on a random convolutional neural network. The cylinder burst noise signal of the internal combustion engine is decomposed by a stationary wavelet, and the inclusion matrix is composed of partial decomposition coefficients. Using singular value theory, the singular value containing matrix is extracted as the characteristic of cylinder burst noise signal. The singular value is used as the input of a random convolutional neural network to train and identify faults. The AdaBound optimizer was then applied to adapt the learning rate to changes, thereby accelerating the weight update of the model. At the same time, the neurons in the structure are randomly deactivated by dropout technology to prevent complex collaborative responses to the training data, and the diagnosis results of each network model are integrated by Dempster synthesis rules. The experimental results show that the minimum mean square error of 0.00165 is achieved when there are 15 neurons in the hidden layer, and it gradually increases as the number of neurons increases. Therefore, it is determined that there should be 15 neurons in the hidden layer, and the trainLM algorithm is used. The decision factors for training, validation, cross-validation, and overall data are 0.98947, 0.98597, 0.97738, and 0.9801, respectively, all reaching the control accuracy of 0.95, which indicates that the random convolutional neural network proposed has a high accuracy for internal combustion engine fault detection. For the main bearing Y-directional force, as the gap increases, its fluctuation trend strengthens; the main bearing force changes within a small range at first, then increases sharply when the gap reaches 0.2[Formula: see text]mm, and some main bearings even experience significant impact loads.

F-BAR proteins CIP4 and FBP17 function in cortical neuron radial migration and process outgrowth.

During embryonic development, radial migration of newly born cortical neurons is a complex process that underlies the proper formation of the neocortex, the outermost layers of neurons in the brain. Disruptions in radial migration results in profound effects on cognitive function and can lead to devastating developmental disabilities. To better understand this critical process in brain development we examined two members of the F-BAR family of membrane bending proteins, CIP4 and FBP17, which are present in the developing brain. We demonstrate that intracellular concentrations of these proteins must be tightly regulated. Increasing or decreasing levels of either protein has profound effects on neuronal morphology and proper radial migration, suggesting they are key players in cortical development.

Contrastive signal-dependent plasticity: Self-supervised learning in spiking neural circuits.

Brain-inspired machine intelligence research seeks to develop computational models that emulate the information processing and adaptability that distinguishes biological systems of neurons. This has led to the development of spiking neural networks, a class of models that promisingly addresses the biological implausibility and the lack of energy efficiency inherent to modern-day deep neural networks. In this work, we address the challenge of designing neurobiologically motivated schemes for adjusting the synapses of spiking networks and propose contrastive signal-dependent plasticity, a process which generalizes ideas behind self-supervised learning to facilitate local adaptation in architectures of event-based neuronal layers that operate in parallel. Our experimental simulations demonstrate a consistent advantage over other biologically plausible approaches when training recurrent spiking networks, crucially side-stepping the need for extra structure such as feedback synapses.

Layer-specific dynamics of local field potentials in monkey V1 during electrical stimulation.

The mammalian neocortex, organized into six cellular layers or laminae, forms a cortical network within layers. Layer specific computations are crucial for sensory processing of visual stimuli within primary visual cortex. Laminar recordings of local field potentials (LFPs) are a powerful tool to study neural activity within cortical layers. Electric brain stimulation is widely used in basic neuroscience and in a large range of clinical applications. However, the layer-specific effects of electric stimulation on LFPs remain unclear. To address this gap, we conducted laminar LFP recordings of the primary visual cortex in monkeys while presenting a flash visual stimulus. Simultaneously, we applied a low frequency sinusoidal current to the occipital lobe with offset frequency to the flash stimulus repetition rate. We analyzed the modulation of visual-evoked potentials with respect to the applied phase of the electric stimulation. Our results reveal that only the deeper layers, but not the superficial layers, show phase-dependent changes in LFP components with respect to the applied current. Employing a cortical column model, we demonstrate that these in vivo observations can be explained by phase-dependent changes in the driving force within neurons of deeper layers. Our findings offer crucial insight into the selective modulation of cortical layers through electric stimulation, thus advancing approaches for more targeted neuromodulation.

Performance Comparison of Bio-Inspired Algorithms for Optimizing an ANN-Based MPPT Forecast for PV Systems.

This study compares bio-inspired optimization algorithms for enhancing an ANN-based Maximum Power Point Tracking (MPPT) forecast system under partial shading conditions in photovoltaic systems. Four algorithms-grey wolf optimizer (GWO), particle swarm optimization (PSO), squirrel search algorithm (SSA), and cuckoo search (CS)-were evaluated, with the dataset augmented by perturbations to simulate shading. The standard ANN performed poorly, with 64 neurons in Layer 1 and 32 in Layer 2 (MSE of 159.9437, MAE of 8.0781). Among the optimized approaches, GWO, with 66 neurons in Layer 1 and 100 in Layer 2, achieved the best prediction accuracy (MSE of 11.9487, MAE of 2.4552) and was computationally efficient (execution time of 1198.99 s). PSO, using 98 neurons in Layer 1 and 100 in Layer 2, minimized MAE (2.1679) but had a slightly longer execution time (1417.80 s). SSA, with the same neuron count as GWO, also performed well (MSE 12.1500, MAE 2.7003) and was the fastest (987.45 s). CS, with 84 neurons in Layer 1 and 74 in Layer 2, was less reliable (MSE 33.7767, MAE 3.8547) and slower (1904.01 s). GWO proved to be the best overall, balancing accuracy and speed. Future real-world applications of this methodology include improving energy efficiency in solar farms under variable weather conditions and optimizing the performance of residential solar panels to reduce energy costs. Further optimization developments could address more complex and larger-scale datasets in real-time, such as integrating renewable energy sources into smart grid systems for better energy distribution.

Artificial Neural Network Modeling of the Removal of Methylene Blue Dye Using Magnetic Clays: An Environmentally Friendly Approach

In this study, the effectiveness of Fe3O4-based clay as a cost-effective material for removing methylene blue (MB) dye from aqueous solutions was evaluated. The structural properties of the clay and Fe3O4-based clay were analyzed using SEM, XRF, BET, XRD, FTIR, and TGA techniques. In this research, the effects of various aspects, such as adsorbent amount, contact time, solution pH, adsorption temperature, and initial dye concentration, on the adsorption of Fe3O4-based clay are investigated. The experiments aimed at understanding the adsorption mechanism of Fe3O4-based clay have shown that the adsorption kinetics are accurately described by the pseudo-second order kinetic model, while the equilibrium data are well represented by the Langmuir isotherm model. The maximum adsorption capacity (qm) was calculated as 52.63 mg/g at 25 °C, 53.48 mg/g at 30 °C, and 54.64 mg/g at 35 °C. All variables affecting the MB adsorption process were systematically optimized in a controlled experimental framework. The effectiveness of the artificial neural network (ANN) model was refined by modifying variables such as the quantity of neurons in the latent layer, the number of inputs, and the learning rate. The model’s accuracy was assessed using the mean absolute percentage error (MAPE) for the removal and adsorption percentage output parameters. The coefficient of determination (R2) values for the dyestuff training, validation, and test sets were found to be 99.40%, 92.25%, and 96.30%, respectively. The ANN model demonstrated a mean squared error (MSE) of 0.614565 for the training data. For the validation dataset, the model recorded MSE values of 0.99406 for the training data, 0.92255 for the validation set, and 0.96302 for the test data. In conclusion, the examined Fe3O4-based clays offer potential as effective and cost-efficient adsorbents for purifying water containing MB dye in various industrial settings.

Modelling and optimizing secondary metabolites production in Spirodela polyrhiza using machine learning

Spirodela. polyrhiza as one of the potential feedstocks for industrial natural dye and pharmaceutical products, efficient process to produce from it were scarce. Here, Artificial Neural Network (ANN) and Genetic Algorithm (GA) were employed for modelling and optimization respectively. Current study focus on establishing useable models and anticipating the optimal concentration of nitrate (NO3−), phosphate (PO43−) and ammonium (NH4+) to give highest relative growth rate (RGR) and produce maximum amount of anthocyanins, chlorophylls, phenolic compounds and antioxidants. In short, 12 and 11 neurons in hidden layer were found to be most suitable neuron numbers used for the modelling of growth and secondary metabolites respectively. All the models exhibited a high degree of robustness and performance with a high r2 value (higher than 0.75) and a low mean squared error (lower than 0.05). GA also anticipated the best concentration of macronutrients to achieve the maximum growth and metabolites production with significant low errors (below 7 %). The employment of machine learning was useful for such applications.

A Cortical Site that Encodes Vocal Expression and Reception.

Socially effective vocal communication requires brain regions that encode expressive and receptive aspects of vocal communication in a social context-dependent manner. Here, we combined a novel behavioral assay with microendoscopy to interrogate neuronal activity in the posterior insula (pIns) in socially interacting mice as they switched rapidly between states of vocal expression and reception. We found that distinct but spatially intermingled subsets of pIns neurons were active during vocal expression and reception. Notably, pIns activity during vocal expression increased prior to vocal onset and was also detected in congenitally deaf mice, pointing to a motor signal. Furthermore, receptive pIns activity depended strongly on social cues, including female odorants. Lastly, tracing experiments reveal that deep layer neurons in the pIns directly bridge the auditory thalamus to a midbrain vocal gating region. Therefore, the pIns is a site that encodes vocal expression and reception in a manner that depends on social context.

Modeling of carbon dioxide absorption into aqueous alkanolamines using machine learning and response surface methodology

This research focuses on modeling CO2 absorption into alkanolamine solvents using multilayer perceptron (MLP), radial basis function network (RBF), Support Vector Machine (SVM), networks, and response surface methodology (RSM). The parameters, including solvent density, mass fraction, temperature, liquid phase equilibrium constant, CO2 loading, and partial pressure of CO2, were used as input factors in the models. In addition, the value of CO2 mass flux was considered as output in the models. Trainlm, trainbr, and trainscg algorithms trained the networks. The results showed that the best number of neurons for MLP with one layer is 16; with two layers, 5 neurons in the first layer and 12 neurons in the second layer; and with three layers, 9 neurons in the first layer, 5 neurons in the second layer, and 1 neuron in the third layer. The best spread in RBF was found to be 2.202 for optimal network performance. Furthermore, statistical data analysis revealed that the trainlm function performs best. The coefficients of determination for RSM, MLP, RBF, and SVM for optimized structures are obtained at 0.9802, 0.9996, 0.9940, and 0.8946, respectively. The results demonstrate that MLP and RBF networks can model CO2 absorption using the trainlm, trainbr, and trainscg algorithms.

Deprivation of visual input alters specific subset of inhibitory neurons and affect thalamic afferent terminals in V1 of rd1 mouse.

Retinitis Pigmentosa (RP) is a heterogenous group of inherited disorder, and its progression not only affects the retina but also the primary visual cortex. This manifests imbalances in the excitatory and inhibitory neurotransmission. Here, we investigated if changes in cortical functioning is linked to alterations in GABAergic population of neurons and its two important subsets, somatostatin (SST) and parvalbumin (PV) neuron in rd1 model of retinal degeneration (RD). We demonstrate marked decrease in the proportion of SST neurons in different layers of cortex whereas PV neurons were less affected. Moreover, we found reduced expression of glutamatergic thalamic afferents (VGLUT2) due to lack of visual activity. These results suggest PV neurons are likely recruited by the cortical circuitry to increase the inhibitory drive and compensate the disrupted inhibition-excitation balance. However, reduced SST expression perhaps results in weakening of stimulus selectivity. Delineating their functional role during RD will provide insights for acquisition of high-resolution vision thereby improving current state of vision restoration.

Analytical Neural Network System for the Helicopter Turboshaft Engines Operating Modes Classification

A novel neural network-based analytical system has been developed for classifying helicopter turboshaft engine operating modes during flight operations. This system utilizes a neural network architecture comprising an input layer with three neurons, two fully connected hidden layers, and an output layer with two neurons. The proposed approach demonstrates an exceptional recognition accuracy of 0.997 (99.7%) across steady-state, unsteady, and transient operating modes of helicopter turboshaft engines. A new method for training this neural network has been introduced, employing forward propagation, loss calculation, backpropagation, and weight updates, enhanced by an adaptive learning rate and the cross-entropy function as the loss criterion. The method also incorporates a novel modified Smooth ReLU activation function for hidden layer neurons. This innovation led to a near-perfect accuracy in network training and reduced the loss to 0.025 (2.5%), highlighting the high quality and reliability of the neural network in classifying engine operating modes during flight. Furthermore, it has been empirically shown that the application of this neural network significantly reduces type I errors by a factor of 2.09 to 2.14 and type II errors by 2.05 to 2.21 times compared to traditional classifiers based on ART-1 and BAM networks. This advancement marks a substantial improvement in classification accuracy and error minimization for helicopter turboshaft engine operating modes.

A neural network model for predicting the temperature of insulated overhead lines

In this work, we develop a neural network model based on experimental heating and cooling curves of insulated wires of overhead lines under changes in wind speed and its direction relative to the wire axis. Insulated SIP-3 wire used in overhead lines was investigated. A multilayer perceptron neural network was employed to model the process of heating and cooling of insulated wire under changes in wind speed and its direction. The average absolute error was taken as a criterion for evaluating the prediction accuracy of wire core and insulation temperatures. The main parameters of the developed neural network model include the number of hidden layers and neurons in each hidden layer, the degree of regulation, and the regulatory rigidity of the model. The modelled heating and cooling curves of insulated wire were compared with those obtained experimentally. The average absolute error was equal to 1.74 and –4.08°C for the predicted core and insulation temperatures, respectively. The difference between the heating curves at low wind speeds was found to range within 9°C. It was shown that an increase in wind speed leads to a decrease in the difference between the curves. Our analysis showed that neural network models used for predicting variations in the temperature of insulated overhead lines should be trained using a larger number of input parameters. This is the main prerequisite for high prediction accuracy of such models, when the difference between the simulated and experimental data does not exceed 5%.