Feed-forward Network Model Research Articles

A spatiotemporal energy model based on spiking neurons for human motion perception.

Inspired by the motion processing pathway, this paper proposes a bio-inspired feedforward spiking network model based on Hodgkin-Huxley neurons for human motion perception. The proposed network mimics the mechanisms of direction selectivity found in simple and complex cells of the primary visual cortex. Simple cells' receptive fields are modeled using Gabor energy filters, while complex cells' receptive fields are constructed by integrating the responses of simple cells in an energy model. To generate the motion map, the spiking output of the network integrates motion information encoded by the responses of complex cells with various preferred directions. Simulation results demonstrate that the spiking neuron-based network effectively replicates the directional selectivity operation of the visual cortex when presented with a sequence of time-varying images. We evaluate the proposed model against state-of-the-art spiking neuron-based motion detection models using publicly available datasets. The results highlight the model's capability to extract motion energy from diverse video sequences, akin to human visual motion perception models. Additionally, we showcase the application of the proposed model in motion segmentation tasks and compare its performance with state-of-the-art motion-based segmentation models using challenging video segmentation benchmarks. The results indicate competitive performance. The motion maps generated by the proposed model can be utilized for action recognition in input videos.

Residual networks for text-independent speaker identification: Unleashing the power of residual learning

The human voice, a dynamic signal, conveys valuable information for speaker identification, encompassing gender, age, emotions, and language. In the biometrics industry, identifying voices in real-time amidst diverse accents, tones, and noisy backgrounds is a challenging task. Voice biometry, a complex aspect of speaker identification, is gaining importance in various applications, such as user authentication, attendance systems, forensics, and banking operations, as it eliminates the need for traditional credentials like cards or passwords. Recent advancements in Human–Computer Interaction technology have made conversational tasks technically feasible. Deep Neural Learning approaches, especially Convolutional Deep Neural Networks (CDNN), have emerged as a powerful tool in the field of speech processing, surpassing traditional Speaker Identification methods. This paper introduces a novel approach using 1-Dimensional Convolutional Residual Blocks for audio classification and Speaker Identification, specifically focusing on speaker recognition from spoken Hindi language. The proposed Residual architecture significantly enhances speaker identification, even in low Signal Noise Ratio environments, achieving an impressive accuracy rate of 86.02%. This outperforms traditional Gaussian Mixture Model (GMM) and Feed Forward Back-propagation Network (FFBN) model for the same set of speakers. Future research directions may explore the classification of audio and speaker identification using various acoustic features derived from speech signals.

Dynamic Temperature Compensation of Pressure Sensors in Migratory Bird Biologging Applications

This article proposes an improved dynamic quantum particle swarm optimization (DQPSO) algorithm to optimize a radial basis function (RBF) neural network for temperature compensation of pressure sensors used in tracking and monitoring wild migratory birds. The algorithm incorporates a temperature-pressure fitting model that includes temperature rate of change and gradient reference terms. It also includes a loss function that considers fitting accuracy and complexity, thereby improving the robustness of the sensor for complex temperature variations. The calibration experiments revealed that after implementation, the average absolute error of the pressure sensor output during dynamic temperature changes was reduced from 145.3 Pa to 20.2 Pa. This reduction represents an 86% improvement over the commercial polynomial compensation method, and the DQPSO approach significantly outperformed traditional feedforward network models. Finally, the algorithm was deployed and verified in an embedded environment for low-power, high-precision, real-time pressure compensation during the tracking and monitoring of wild migratory birds.

Rule-Based Non-Intrusive Load Monitoring Using Steady-State Current Waveform Features.

Monitoring electricity energy usage can help to reduce power consumption considerably. Among load monitoring techniques, non-intrusive load monitoring (NILM) provides a cost-efficient solution to identify individual load consumption details from the aggregate voltage and current measurements. Existing load monitoring techniques often require large datasets or use complex algorithms to obtain acceptable performance. In this paper, a NILM technique using six non-redundant current waveform features with rule-based set theory (CRuST) is proposed. The architecture consists of an event detection stage followed by preprocessing and framing of the current signal, feature extraction, and finally, the load identification stage. During the event detection stage, a change in connected loads is ascertained using current waveform features. Once an event is detected, the aggregate current is processed and framed to obtain the event-causing load current. From the obtained load current, the six features are extracted. Furthermore, the load identification stage determines the event-causing load, utilizing the features extracted and the appliance model. The results of the CRuST NILM are evaluated using performance metrics for different scenarios, and it is observed to provide more than 96% accuracy for all test cases. The CRuST NILM is also observed to have superior performance compared to the feed-forward back-propagation network model and a few other existing NILM techniques.

Reliable composite fault diagnosis of hydraulic systems based on linear discriminant analysis and multi-output hybrid kernel extreme learning machine

With increasingly stringent in requirements on the reliability and safety of hydraulic systems, data-driven fault diagnosis has emerged as a popular area of research. Hydraulic systems may have multiple failure modes, and accurately diagnosing compound failures in multi-component systems is a daunting task. In this paper, a method of multi-output classification by combining linear discriminant analysis (LDA) with the hybrid kernel extreme learning machine (HKELM) is proposed to diagnose compound faults in hydraulic systems. Data selection based on LDA is used in place of expert knowledge to screen out sensitive channels of each component from multi-channel signals. The multi-output strategy is embedded into the HKELM, which can simultaneously output the fault status of multiple components to diagnose the health of the system. An improved Hamming loss is also proposed to evaluate the total error in the multi-output classification because it has greater applicative relevance than classification accuracy. The results of experiments show that the proposed method can diagnose composite faults in multi-component systems with an accuracy higher than 99.5% and an error of only 0.20% on a dataset of hydraulic systems. As a shallow feed-forward network model, it can be used for real-time fault diagnosis due to its efficiency.

Multidimensional cerebellar computations for flexible kinematic control of movements

Both the environment and our body keep changing dynamically. Hence, ensuring movement precision requires adaptation to multiple demands occurring simultaneously. Here we show that the cerebellum performs the necessary multi-dimensional computations for the flexible control of different movement parameters depending on the prevailing context. This conclusion is based on the identification of a manifold-like activity in both mossy fibers (MFs, network input) and Purkinje cells (PCs, output), recorded from monkeys performing a saccade task. Unlike MFs, the PC manifolds developed selective representations of individual movement parameters. Error feedback-driven climbing fiber input modulated the PC manifolds to predict specific, error type-dependent changes in subsequent actions. Furthermore, a feed-forward network model that simulated MF-to-PC transformations revealed that amplification and restructuring of the lesser variability in the MF activity is a pivotal circuit mechanism. Therefore, the flexible control of movements by the cerebellum crucially depends on its capacity for multi-dimensional computations.

Sparse RNNs can support high-capacity classification

Feedforward network models performing classification tasks rely on highly convergent output units that collect the information passed on by preceding layers. Although convergent output-unit like neurons may exist in some biological neural circuits, notably the cerebellar cortex, neocortical circuits do not exhibit any obvious candidates for this role; instead they are highly recurrent. We investigate whether a sparsely connected recurrent neural network (RNN) can perform classification in a distributed manner without ever bringing all of the relevant information to a single convergence site. Our model is based on a sparse RNN that performs classification dynamically. Specifically, the interconnections of the RNN are trained to resonantly amplify the magnitude of responses to some external inputs but not others. The amplified and non-amplified responses then form the basis for binary classification. Furthermore, the network acts as an evidence accumulator and maintains its decision even after the input is turned off. Despite highly sparse connectivity, learned recurrent connections allow input information to flow to every neuron of the RNN, providing the basis for distributed computation. In this arrangement, the minimum number of synapses per neuron required to reach maximum memory capacity scales only logarithmically with network size. The model is robust to various types of noise, works with different activation and loss functions and with both backpropagation- and Hebbian-based learning rules. The RNN can also be constructed with a split excitation-inhibition architecture with little reduction in performance.

Construction Safety Risk Model with Construction Accident Network: A Graph Convolutional Network Approach

Construction risk assessment (RA) based on expert knowledge and experience incorporates uncertainties that reduce its accuracy and effectiveness in implementing countermeasures. To support the construction of RA procedures and enhance associated decision-making processes, machine learning (ML) approaches have recently been investigated in the literature. Most ML approaches have difficulty processing dependency information from real-life construction datasets. This study developed a novel RA model that incorporates a graph convolutional network (GCN) to account for dependency information between construction accidents. For this purpose, the construction accident dataset was restructured into an accident network, wherein the accidents were connected based on the shared project type. The GCN decodes the construction accident network information to predict each construction activity’s severity outcome, resulting in a prediction accuracy of 94%. Compared with the benchmark feedforward network (FFN) model, the GCN demonstrated a higher prediction accuracy and better generalization ability. The developed GCN severity predictor allows construction professionals to identify high-risk construction accident scenarios while considering dependency based on the shared project type. Ultimately, understanding the relational information between construction accidents increases the representativeness of RA severity predictors, enriches ML models’ comprehension, and results in a more reliable safety model for construction professionals.

Characteristics of wideband tympanometry in patients with Ménière's disease based on neutral network

Objective:To construct a prediction model for Ménière's disease based on neural network and evaluate its prediction ability. Methods:Sixty-four patients with Ménière's disease underwent gadolinium enhanced magnetic resonance imaging of inner ear which showed endolymphatic hydrops. Meanwhile, 40 healthy adults were enrolled as controls. The database of wideband tympanometry of patients and control subjects was analyzed, and the neural network model was established by MATLAB 2021a software. The prediction ability of the model was evaluated by accuracy, positive predictive value, negative predictive value, the Youden index, sensitivity, specificity, receiver operating characteristic curve and area under curve （AUC）. Results:A feedforward network model was built with a single hidden layer to predict Ménière's disease with wideband tympanometry. There were 104 features in the input layer, 13 neuron nodes in the hidden layer and 1 output neuron in the output layer. The accuracy of the model was 83.2%, the positive predictive value was 80.7%, the negative predictive value was 84.3%, the sensitivity was 76.5%, the specificity was 83.7%, the Youden index was 0.602, and the AUC was 0.855. Conclusion:Based on neural network, the prediction model of Ménière's disease with high accuracy was constructed according to the results of wideband tympanometry, which provided reference for the diagnose of Ménière's disease.

Artificial Neural Networking (ANN) Model for Drag Coefficient Optimization for Various Obstacles

For various obstacles in the path of a flowing liquid stream, an artificial neural networking (ANN) model is constructed to study the hydrodynamic force depending on the object. The multilayer perceptron (MLP), back propagation (BP), and feed-forward (FF) network models were employed to create the ANN model, which has a high prediction accuracy and a strong structure. To be more specific, circular-, octagon-, hexagon-, square-, and triangular-shaped cylinders are installed in a rectangular channel. The fluid is flowing from the left wall of the channel by following two velocity profiles explicitly linear velocity and parabolic velocity. The no-slip condition is maintained on the channel upper and bottom walls. The Neumann condition is applied to the outlet. The entire physical design is mathematically regulated using flow equations. The result is presented using the finite element approach, with the LBB-stable finite element pair and a hybrid meshing scheme. The drag coefficient values are calculated by doing line integration around installed obstructions for both linear and parabolic profiles. The values of the drag coefficient are predicted with high accuracy by developing an ANN model toward various obstacles.

Transfer Learning Approach to Multitarget Temperature-Dependent Reaction Rate Prediction.

Accurate prediction of temperature-dependent reaction rate constants of organic compounds is of great importance to both atmospheric chemistry and combustion science. Extensive work has been done on developing automated mechanism generation systems but the lack of quality reaction rate data remains a huge bottleneck in the application of highly detailed mechanisms. Machine learning prediction models have been recently adopted to alleviate the data gap in thermochemistry and have great potential to do the same for kinetic data with the recent release of quality reaction rate data compilations. The ultimate goal is to formulate easily accessible, general-purpose, temperature-dependent, and multitarget models for the prediction of reaction rates. To that end, we propose a model that depends on the well-known Morgan fingerprints as well as learned representations transferred from the QM9 data set. We propose the use of an Arrhenius-based loss where predictions of the three modified-Arrhenius parameters (A, n, and B = Ea/R) are given instead of the direct prediction of reaction rate constants. Our model is >35% more accurate compared to a baseline model of feed forward network (FFN) on Morgan fingerprints.

Neural Network Prediction Model to Explore Complex Nonlinear Behavior in Dynamic Biological Network

Organism network systems provide a biological data with high complex level. Besides, these data reflect the complex activities in organisms that identifies nonlinear behavior as well. Hence, mathematical modelling methods such as Ordinary Differential Equations model (ODE's) are becoming significant tools to predict, and expose implied knowledge and data. Unfortunately, the aforementioned approaches face some of cons such as the scarcity and the vagueness in the biological knowledge to expect the protein concentrations measurements. So, the main object of this research presents a computational model such as a neural Feed Forward Network model using Back Propagation algorithm to engage with imprecise and missing biological knowledge to provide more insight about biological systems in organisms. Therefore, the model predicts protein concentration and illustrates the nonlinear behavior for the biological dynamic behavior in precise form. Also, the desired results are matched with recent ODE's model and it provides precise results in simpler form than ODEs.

A Hybrid ANN-GA Model for an Automated Rapid Vulnerability Assessment of Existing RC Buildings

Determining the risk priorities for the building stock in highly seismic-prone regions and making the final decisions about the buildings is one of the essential precautionary measures that needs to be taken before the earthquake. This study aims to develop an Artificial Neural Network (ANN)-based model to predict risk priorities for reinforced-concrete (RC) buildings that constitute a large part of the existing building stock. For this purpose, the network parameters in the network structure have been optimized by establishing a hybrid structure with the Genetic Algorithm (GA). As a result, the ANN model can make accurate predictions with maximum efficiency. The suggested ANN model is a feedforward back-propagation network model. It aims to predict the risk priorities for 329 RC buildings in the most successful way, for which the performance score was calculated using the Turkey Rapid Evaluation Method (2013). In this paper, a GA-ANN hybrid model was implemented in which the ANN, using the most successful gene revealed by the model, produced successful results in calculating the performance score. In addition, the required input parameters for obtaining more efficient results in solving such a problem and the parameters that need to be used in establishing such an ANN network structure have been optimized. With the help of such a model, the operation process will be eliminated. The created hybrid model was 98% successful in determining the risk priority in RC buildings.

Simple yet effective 3D ego-pose lift-up based on vector and distance for a mounted omnidirectional camera

Following the advances in convolutional neural networks and synthetic data generation, 3D egocentric body pose estimations from a mounted fisheye camera have been developed. Previous works estimated 3D joint positions from raw image pixels and intermediate supervision during the process. The mounted fisheye camera captures notably different images that are affected by the optical properties of the lens, angle of views, and setup positions. Therefore, 3D ego-pose estimation from a mounted fisheye camera must be trained for each set of camera optics and setup. We propose a 3D ego-pose estimation from a single mounted omnidirectional camera that captures the entire circumference by back-to-back dual fisheye cameras. The omnidirectional camera can capture the user’s body in the 360∘ field of view under a wide variety of motions. We also propose a simple feed-forward network model to estimate 3D joint positions from 2D joint locations. The lift-up model can be used in real time yet obtains accuracy comparable to those of previous works on our new dataset. Moreover, our model is trainable with the ground truth 3D joint positions and the unit vectors toward the 3D joint positions, which are easily generated from existing publicly available 3D mocap datasets. This advantage alleviates the data collection and training burden due to changes in the camera optics and setups, although it is limited to the effect after the 2D joint location estimation.

Analysis of the effect of arrhenius activation energy and temperature dependent viscosity on non-newtonian maxwell nanofluid bio-convective flow with partial slip by artificial intelligence approach

This study focused on the analysis of partial slip effect of Arrhenius activation energy and temperature dependent viscosity on non-Newtonian Maxwell nanofluid bio-convective flow using artificial intelligence approach. Local Nusselt number, local Sherwood number and local density number values, which are dimensionless flow parameters, have been used to examine the said effect. Three different artificial neural network models have been developed using the numerically obtained data sets. Each of the feed forward back propagation multilayer perceptron network models has been developed with different input parameters. 80% of the data set has been used for training the model and 20% for the testing phase. The estimation performance of the network models developed with the Bayesian regularization training algorithm has been extensively analyzed, and the compatibility between the estimation values and the target data has been examined. The findings have shown that artificial neural network models have been developed to make predictions with high accuracy. In addition, artificial neural networks have also proven to be an ideal engineering tool that can be used to analyze the partial slip effect of non-Newtonian Maxwell nanofluids on bio-convective flow.

Metaheuristic Optimization in Neural Network Model for Seasonal Data

This research was focused on the use of metaheuristic optimization in neural network for time series modeling. The three optimization methods were used as experiments, i.e Genetic Algorithm, Particle Swarm Optimization, and Modified Bee Colony. Neural network architecture chosen in this research was Feed Forward Neural Network. The weaknesses and limitations of gradient-based methods inspired many researchers to try to use other methods. Non-gradient based method is a reasonable choice considering the learning algorithm. Neural network is inspired by the characteristics of creatures, so that optimization methods that also resemble the patterns of life in nature will be appropriate. In this paper, the three metaheuristic optimization methods were tried with various scenarios for getting the best one. The proposed procedure was applied to the rainfall data. The experimental study showed that Genetic Algorithm and Particle Swarm Optimization were recommended as optimization methods at Feed Forward Neural Network (FFNN) model for the rainfall data.

On Phase Information for Deep Neural Networks to Solve Full-Wave Nonlinear Inverse Scattering Problems

The phase information's role in deep neural networks (DNNs) to solve the electromagnetic inverse scattering problems is investigated. The feedforward neutral network model with complex-valued (CV) data stream and the corresponding CV backpropagation training algorithm are proposed to realize CV convolutional neural networks. Numerical examples are carried out to demonstrate the phase information's role in DNNs in terms of generalization capability as well as the convergence speed in the training stage.

Investigating deep feedforward neural networks for classification of transposon-derived piRNAs

PIWI-interacting RNAs (piRNAS) form an important class of non-coding RNAs that play a key role in gene expression regulation and genome integrity by silencing transposable elements. However, despite the importance of piRNAs and the large application of deep learning in computational biology, there are few studies of deep learning for piRNAs prediction. Still, current methods focus on using advanced architectures like CNN and variations. This paper presents an investigation on deep feedforward network models for classification of human transposon-derived piRNAs. We developed a lightweight predictor (when compared to other deep learning methods) and we show by practical evidence that simple neural networks can perform as well as better than complex neural networks when using the appropriate hyperparameters. For that, we train, analyze and compare the results of a multilayer perceptron with different hyperparameter choices, such as numbers of hidden layers, activation functions and optimizers, clarifying the advantages and disadvantages of each choice. Our proposed predictor reached a F-score of 0.872, outperforming other state-of-the-art methods for human transposon-derived piRNAs classification. In addition, to better access the generalization of our proposal, we also showed it achieved competitive results when classifying piRNAs of other species.

Machine learning models for the secondary Bjerknes force between two insonated bubbles

The secondary Bjerknes force plays a significant role in the evolution of bubble clusters. However, due to the complex dependence of the force on multiple parameters, it is highly non-trivial to include its effects in the simulations of bubble clusters. In this paper, machine learning is used to develop a data-driven model for the secondary Bjerknes force between two insonated bubbles as a function of the equilibrium radii of the bubbles, the distance between the bubbles, the amplitude and the center frequency of the ultrasound wave. The sign of the force may change with the phase difference between the oscillating bubbles. Meanwhile, the magnitude of the force varies over several orders of magnitude, which poses a serious challenge for the usual machine learning models. To overcome this difficulty, the magnitudes and the signs of the force are separated and modelled separately. A nonlinear regression is obtained with a feed-forward network model for the logarithm of the magnitude, whereas the sign is modelled by a support-vector machine model. The principle, the practical aspects related to the training and validation of the machine models are introduced. The predictions from the models are checked against the values computed from the Keller–Miksis equations. The results show that the models are extremely efficient while providing accurate estimate of the force. The models make it computationally feasible for the future simulations of the bubble clusters to include the effects of the secondary Bjerknes force.Graphic abstract

Adsorption of neutral organic compounds on polar and nonpolar microplastics: Prediction and insight into mechanisms based on pp-LFERs

Adsorption of 18 neutral organic compounds (OCs) on polar (polybutylene succinate (PBS) and polycaprolactone (PCL)) and nonpolar (low-density polyethylene (LDPE) and polystyrene (PS)) microplastics (MPs) were investigated. The adsorption coefficients (Kd) varied with ranges of 130–42,002, 124–27,768, 6.40–10,713, and 1.52–10,332 L kg−1 for adsorption on PCL, PBS, LDPE, and PS MPs, respectively. The polar MPs showed greater adsorption capacities than nonpolar MPs. Non-specific interaction, i.e. hydrophobic partition played a crucial role in the adsorption of OCs on all MPs, while polar interactions also contributed significantly to the greater adsorption on polar MPs. Poly-parameter linear free energy relationships (pp-LFERs) with multiple linear regression (MLR) and feedforward network (FN) were then employed to model the adsorption of OCs on MPs so as to obtain deep insights into adsorption mechanisms. The MLR models achieved Radj2 of 0.90–0.97 and root mean square error (RMSE) of 0.13–0.38 log units, while the FN models achieved Radj2 of 0.85–0.90 and RMSE of 0.21–0.60 log units. The MLR models are more accurate under selected equilibrium concentrations while FN models are capable of making predictions under varying equilibrium concentrations. Lastly, both MLR and FN models showed good prediction on literature adsorption data on nonpolar MPs.