Feedforward Network Research Articles

Reverse engineering of feedforward cortical-Hippocampal microcircuits for modelling neural network function and dysfunction

Engineered biological neural networks are indispensable models for investigation of neural function and dysfunction from the subcellular to the network level. Notably, advanced neuroengineering approaches are of significant interest for their potential to replicate the topological and functional organization of brain networks. In this study, we reverse engineered feedforward neural networks of primary cortical and hippocampal neurons, using a custom-designed multinodal microfluidic device with Tesla valve inspired microtunnels. By interfacing this device with nanoporous microelectrodes, we show that the reverse engineered multinodal neural networks exhibit capacity for both segregated and integrated functional activity, mimicking brain network dynamics. To advocate the broader applicability of our model system, we induced localized perturbations with amyloid beta to study the impact of pathology on network functionality. Additionally, we demonstrate long-term culturing of subregion- and layer specific neurons extracted from the entorhinal cortex and hippocampus of adult Alzheimer’s-model mice and rats. Our results thus highlight the potential of our approach for reverse engineering of anatomically relevant multinodal neural networks to study dynamic structure-function relationships in both healthy and pathological conditions.

Transfer learning for probabilistic localization of hidden cracks in concrete structures

AbstractThe utility of discriminative supervised learning models built using multiple training-data sources is investigated for hidden crack localization in concrete. Feed-forward neural network (FFNN) is chosen as the model architecture, and transfer learning is used to assimilate the information obtained from different sources (computational physics simulations and laboratory experiments). The labeled training data consists of values of a damage index and the known locations of hidden cracks. The classification models need to learn how the presence of damage (hidden cracks) affects the damage index at different sensors for different test conditions. To this end, diagnostic FFNN models are built by sequentially adding and training new hidden layers to assimilate labeled information from computer models (different model geometries, test conditions, crack lengths, crack locations) and laboratory experiments on a plain cement slab. These transfer learning-based models are then used to localize damage in concrete specimens that reflect real-world conditions (i.e., specimens with steel reinforcement and randomly distributed aggregate). The actual damage state in these specimens is determined by extracting cores and performing petrographic studies on the extracted cores. The damage probability estimated by transfer learning-based models is compared with the petrographic damage rating index (DRI) to identify the most suitable approach to train the diagnostic models. The transfer learning-based diagnostic methodology shows promise and could be used in various structural health monitoring applications, where sufficient labeled data are typically not available from a single data source.

Competitive plasticity to reduce the energetic costs of learning.

The brain is not only constrained by energy needed to fuel computation, but it is also constrained by energy needed to form memories. Experiments have shown that learning simple conditioning tasks which might require only a few synaptic updates, already carries a significant metabolic cost. Yet, learning a task like MNIST to 95% accuracy appears to require at least 108 synaptic updates. Therefore the brain has likely evolved to be able to learn using as little energy as possible. We explored the energy required for learning in feedforward neural networks. Based on a parsimonious energy model, we propose two plasticity restricting algorithms that save energy: 1) only modify synapses with large updates, and 2) restrict plasticity to subsets of synapses that form a path through the network. In biology networks are often much larger than the task requires, yet vanilla backprop prescribes to update all synapses. In particular in this case, large savings can be achieved while only incurring a slightly worse learning time. Thus competitively restricting plasticity helps to save metabolic energy associated to synaptic plasticity. The results might lead to a better understanding of biological plasticity and a better match between artificial and biological learning. Moreover, the algorithms might benefit hardware because also electronic memory storage is energetically costly.

A software tool for fabricating phantoms mimicking human tissues with designated dielectric properties and frequency.

Dielectric materials play a crucial role in assessing and refining the measurement performance of dielectric properties for specific tasks. The availability ofviable and standardized dielectric materials could greatly enhance medical applications related to dielectric properties. However, obtaining reliable phantoms with designated dielectric properties across a specified frequency range remains challenging. In this study, we propose software to easily determine the components of dielectric materials in the frequency range of 16 MHz to 3 GHz. A total of 184 phantoms were fabricated and measured using open-ended coaxial probe method. The relationship among dielectric properties, frequency, and thecomponents of dielectric materials was fitted through feedforward neural networks. Software was developed to quickly calculate the composition of dielectric materials. We performed validation experiments including blood, muscle, skin, and lung tissue phantoms at 128 MHz, 298 MHz, 915 MHz, and 2.45 GHz. Compared with literature values, the relative errors of dielectric properties are less than 15 %. This study establishes a reliable method for fabricating dielectric materials with designated dielectric properties and frequency through the development of the software. This research holds significant importance in enhancing medical research and applications that rely on tissue simulation using dielectric phantoms.

A Novel Technique in Determining Mud Cake Permeability in SiO2 Nanoparticles and KCl Salt Water Based Drilling Fluid using Deep Learning Algorithm

The permeability of the mud cake formed at the formation-wellbore interface is an important factor in the designing of water-based drilling fluids. This study presents a novel approach to utilizing experimental thixotropic and rheological parameters of polymeric water-based drilling fluids having varying concentrations of SiO2 nanoparticles and KCl salt. A fully connected feed-forward multi-layered neural network, more commonly known as a Multilayer Perceptron (MLP) was developed to predict the mud cake permeability using input parameters such as SiO2 & KCl concentration, differential pressure, temperature, mud cake thickness, API LPLT and HPHT filter loss volume and spurt loss volume. The results suggested that the developed Multilayer Perceptron model effectively determined the mud cake permeability based on the input parameters of the WBDF mentioned above. The model converged on the global minima, minimizing the loss function using the Gradient descent algorithm. A higher Coefficient of Determination (R2) value i.e., 0.8781, and a lesser Root Mean Square Error (RMSE) value i.e., 0.04378 indicates the higher accuracy of the model. Pearson’s Coefficient of Correlation obtained via the heatmap indicates that mud cake permeability is strongly influenced by the differential pressure followed by filter loss volume, spurt loss volume, mud cake thickness, and temperature. Previous similar studies have focused on using machine learning algorithms, this study utilized a robust deep learning algorithm i.e., Multilayer Perceptron (MLP) neural network to simultaneously model the combined effects of SiO2 nanoparticles and KCl salt concentrations on mud cake permeability, offering an unprecedented level of accuracy in predicting key WBDF performance parameters

3D reconstruction from focus for lensless imaging

The lensless camera is an ultra-thin imaging system that utilizes encoding elements instead of lenses to perceive the light field and reconstruct it through computational methods. Early studies have demonstrated that lensless cameras can encode 3D scenes at various depths in caustic patterns with varying sizes, known as point spread functions (PSFs). By deconvolving measurements with these PSFs, the reconstruction exhibits distinct focusing effects: objects in the focal plane appear sharp, while objects in other planes become blurred. Building upon this feature, we propose a feedforward network based on depth from focus to generate the depth map and the all-in-focus image by reconstructing the focal stack and deriving the probability of pixel clarity. Using our optimization framework, we present superior and more stable depth estimation than previous methods in both simulated data and real measurements captured by our lensless camera.

Design of Input Signal for System Identification of a Generic Fighter Configuration

This article investigates the design of time-accurate input signals in the angle-of-attack and pitch rate space to identify the aerodynamic characteristics of a generic triple-delta wing configuration at subsonic speeds. Regression models were created from the time history of signal simulations in DoD HPCMP CREATETM-AV/Kestrel software. The input signals included chirp, Schroeder, pseudorandom binary sequence (PRBS), random, and sinusoidal signals. Although similar in structure, the coefficients of these regression models were estimated based on the specific input signals. The signals covered a wide range of angle-of-attack and pitch rate space, resulting in varying regression coefficients for each signal. After creating and validating the models, they were used to predict static aerodynamic data at a wide range of angles of attack but with zero pitch rate. Next, slope coefficients and dynamic derivatives in the pitch direction were estimated from each signal. These predictions were compared with each other as well as with the ONERA wind tunnel data and some CFD calculations from the DLR TAU code provided by the NATO Science and Technology Organization research task group AVT-351. Subsequently, the models were used to predict different pitch oscillations at various mean angles of attack with given amplitudes and frequencies. Again, the model predictions were compared with wind tunnel data. Final predictions involved responses to new signals from different models. A feed-forward neural network was then used to model pressure coefficients on the upper surface of the vehicle at different spanwise sections for each signal and the validated models were used to predict pressure data at different angles of attack. Overall, the models predict similar integrated forces and moments, with the main discrepancies appearing at higher angles of attack. All models failed to predict the stall behavior observed in the measurements and CFD data. Regarding the pressure data, the PRBS signal provided the best accuracy among all the models.

Air quality index prediction with optimisation enabled deep learning model in IoT application

ABSTRACT The development of industrial and urban places caused air pollution, which has resulted in a variety of effects on individuals and the atmosphere over the years. The measurement of the air quality index (AQI) depends on various environmental situations, such as emissions, dispersions, and chemical reactions. This paper developed the Internet of Things (IoT)-based Deep Learning (DL) technique for predicting air quality. Initially, the IoT simulation is performed, where the nodes receive input data. The routing technique is used to identify the best route toward the Base station (BS). The proposed Tangent Two-Stage Algorithm (TTSA) is used in the routing mechanism. For AQI prediction, the time series data is transmitted to the BS. The Z-score normalisation is employed to neglect the unessential data. Furthermore, feature indicator extraction is employed to extract the relevant feature indicators. The Deep Feedforward Neural Network (DFNN) is used to predict air quality. Furthermore, the proposed Fractional Tangent Two-Stage Optimisation (FTTSA) is employed for the training process of DFNN. Moreover, metrics such as energy, time, and distance are used to evaluate the routing process, and superior results such as 0.979J, 0.025s and 0.196 m are obtained. Furthermore, the AQI is predicted by metrics like root mean square error (RMSE), R-squared (R2), mean square error (MSE), and mean absolute percentage error (MAPE), whereas the superior values such as 0.602, 0.598, 0.362, and 0.456 are attained.

PAFPT: Progressive aggregator with feature prompted transformer for underwater image enhancement

The optical characteristics of underwater environments often result in distorted underwater images, which makes it difficult to meet the needs of complex underwater environment perception. The demand for obtaining high-quality underwater images presents challenges for existing underwater image enhancement methods. In this paper, a progressive aggregator with feature prompted transformer for underwater image enhancement is presented. First, a feature prompted transformer module consisting of a feature convolutional transposed attention and a prompted feedforward neural network is proposed. It integrates feature attention mechanisms and adjusts input features through prompt weights to better focus on global information while enhancing local details, enriching texture, removing blurring, and correcting color deviation. Concurrently, a progressive aggregator consisting of four stages of up-sampling aggregation blocks is constructed to transfer features effectively and facilitate the aggregation of features of different scales at different stages to realize comprehensive attention to image details and semantic information. Besides, a region reconstruction unit is designed, and a reconstruction attention mechanism is introduced to pay attention to the bottleneck localized region to suppress noise. The performance of our method is thoroughly evaluated on public underwater image datasets: EUVP, UIEB, RUIE and SQUID. The quantitative and qualitative results indicate that our method outperforms nine state-of-the-art methods in both subjective perception and evaluation metrics, demonstrating excellent learning and generalization abilities. Additionally, the excellent results reflect the significant performance gains it brings to downstream visual applications.

Improved Early Detection of Tube Leaks Faults in Pulverised Coal-fired Boiler Using Deep Feed Forward Neural Network

Boiler tube leaks significantly reduce the operational availability of power units, yet their early detection and prediction have not been fully realised in the industry. This paper introduces a novel approach employing deep feedforward neural networks for early detection of boiler tube leaks in pulverised coal-fired boilers. Early detection enhances repair planning, minimising downtime and production losses. It also improves monitoring and control of boiler tube failures, thereby optimising power plant operations and revenue. Diverse deep neural network models were developed and rigorously tested by leveraging 9 years of operational data (2012–2020). Exhaustive hyper-parameter optimisation, involving seven parameters, substantially improved predictive accuracy. By achieving training and testing accuracies of 82.8% to 99.3%, the study assessed their ability to detect boiler tube leaks over the same 9-year span, providing insights into leak detection capabilities. The models notably predicted all 12 identified tube leak events, averaging a 14-day lead time before boiler shutdown. In addition to leak prediction, a leak detection matrix was devised to analyse residual behaviour and reduce the likelihood of false alarms. However, the models’ predictive performance was observed to be limited to the following year, with satisfactory results for 2021 only. Incorporating the 2021 data into retraining significantly improved the predictions for 2022. The study concludes that while the models demonstrate robust short-term prediction capabilities, they require continuous retraining to maintain accuracy and relevance. This ongoing refinement is essential for keeping the models up-to-date and reliable in predicting future boiler tube leaks.

Trajectory error compensation method for grinding robots based on kinematic calibration and joint variable prediction

Trajectory accuracy, a crucial metric in assessing the dynamic performance of grinding robots, is influenced by the uncertain movement of the tool center point, directly impacting the surface quality of processed workpieces. This article introduces an innovative method for compensating trajectory errors. Initially, a strategy for error compensation is derived using differential kinematics theory. Subsequently, a robot kinematic calibration method utilizing ring particle swarm optimization (RPSO) is proposed to address static errors in the grinding robot. Simultaneously, a method for predicting robot joint variables based on a dual-channel feedforward neural network (DCFNN) is designed to mitigate dynamic errors. Finally, a simulation platform is developed to validate the proposed method. Simulation analysis using extensive data demonstrates an 89.3% improvement in absolute position accuracy and a 74.2% reduction in error fluctuation range, outperforming sparrow search algorithm (SSA), improved mayfly algorithm (IMA), multi-representation integrated predictive neural network (MRIPNN), etc. Algorithmic comparison reveals that kinematic calibration significantly reduces the average trajectory error, while joint variable prediction notably minimizes error fluctuation. Validation through trajectory straightness testing and a 3D printing propeller grinding experiment achieves a trajectory straightness of 0.2425 mm. Implementing this method enables achieving 86.1% surface machining allowance within tolerance, making it an optimal solution for grinding robots.

Dissecting the Herbicidal Mechanism of Microbial Natural Product Lydicamycins Using a Deep Learning-Based Nonlinear Regression Model.

The plant microbiome significantly influences plant-microbe interactions, but the mechanisms are often complex and nonlinear. Here we show the nonlinear regulatory effects of Streptomyces ginsengnesis G7 on Arabidopsis thaliana growth. We focused on lydicamycin, a molecule from this bacterium that interferes with auxin polar transport. Using a deep learning approach with a feedforward neural network, we integrated multiomics data to elucidate the mechanism of lydicamycin on plant growth and development. We also examined the impact of flavonol metabolites, particularly isorhamnetin from A. thaliana, on the PIN protein family's role in auxin transport. Our findings indicate that lydicamycin regulates auxin transport by inducing flavonol overaccumulation in A. thaliana, affecting plant development. This study identifies potential molecular targets for crop enhancement and improved agricultural productivity.

Research on a Non-Stationary Groundwater Level Prediction Model Based on VMD-iTransformer and Its Application in Sustainable Water Resource Management of Ecological Reserves

The precise forecasting of groundwater levels significantly influences plant growth and the sustainable management of ecosystems. Nonetheless, the non-stationary characteristics of groundwater level data often hinder the current deep learning algorithms from precisely capturing variations in groundwater levels. We used Variational Mode Decomposition (VMD) and an enhanced Transformer model to address this issue. Our objective was to develop a deep learning model called VMD-iTransformer, which aims to forecast variations in the groundwater level. This research used nine groundwater level monitoring stations located in Hangjinqi Ecological Reserve in Kubuqi Desert, China, as case studies to forecast the groundwater level over four months. To enhance the predictive performance of VMD-iTransformer, we introduced a novel approach to model the fluctuations in groundwater levels in the Kubuqi Desert region. This technique aims to achieve precise predictions of the non-stationary groundwater level conditions. Compared with the classic Transformer model, our deep learning model more effectively captured the non-stationarity of groundwater level variations and enhanced the prediction accuracy by 70% in the test set. The novelty of this deep learning model lies in its initial decomposition of multimodal signals using an adaptive approach, followed by the reconfiguration of the conventional Transformer model’s structure (via self-attention and inversion of a feed-forward neural network (FNN)) to effectively address the challenge of multivariate time prediction. Through the evaluation of the prediction results, we determined that the method had a mean absolute error (MAE) of 0.0251, a root mean square error (RMSE) of 0.0262, a mean absolute percentage error (MAPE) of 1.2811%, and a coefficient of determination (R2) of 0.9287. This study validated VMD and the iTransformer deep learning model, offering a novel modeling approach for precisely predicting fluctuations in groundwater levels in a non-stationary context, thereby aiding sustainable water resource management in ecological reserves. The VMD-iTransformer model enhances projections of the water level, facilitating the reasonable distribution of water resources and the long-term preservation of ecosystems, providing technical assistance for ecosystems’ vitality and sustainable regional development.

Hysteresis Compensation and Trajectory Tracking Control Model for Pneumatic Artificial Muscles

The optimum performance position control of pneumatic artificial muscles (PAM) is restricted by their in-built hysteresis and nonlinearity. The hysteresis is usually depicted by a phenomenological model, while the model mentioned above always only describes the hysteresis phenomenon under certain conditions. Thus, the universality of the compensator is due to its weakness in handling disparate outside conditions. Our research employs the FN–QUPI (feedforward neural network–quadratic unparallel Prandtl–Ishlinskii) model to depict the phenomenon of pressure-displacement hysteresis in PAMs. This model has high-precision expression and generalization ability for the PAM hysteresis phenomenon. According to this, an inverse model of the QUPI operator is established as a feedforward control while combining with the feedback control of incremental PID-type iterative learning. The results show that due to the hysteresis of PAM, the compound control of feedforward control and iterative learning has better tracking performance than the ordinary PID compound control in terms of convergence rate and stability. According to the mean absolute error (MAE) and root mean square error (RMSE) of the tracking process, it can be seen that the control model can achieve accurate nonlinear compensation, and the control system shows excellent robustness to different input signals.

Hypergraph convolutional networks with multi-ordering relations for cross-document event coreference resolution

Recognizing the coreference relationship between different event mentions in the text (i.e., event coreference resolution) is an important task in natural language processing. It helps to understand the association between various events in the text, and plays an important role in information extraction, question answering systems, and reading comprehension. Existing research has made progress in improving the performance of event coreference resolution, but there are also some shortcomings. For example, most of the existing methods analyze the event data in the document in a serial processing mode, without considering the complex relationship between events, and it is difficult to mine the deep semantics of events. To solve these problems, this paper proposes a cross-document event co-reference resolution method (HGCN-ECR) based on hypergraph convolutional neural networks. Firstly, the BiLSTM-CRF model was used to label the semantic role of the events extracted from a number of documents. According to the labeling results, the trigger words and non-trigger words of the event were determined, and the multi-document event hypergraph was constructed around the event trigger words. Then hypergraph convolutional neural networks are used to learn higher-order semantic information in multi-document event hypergraphs, and multi-head attention mechanisms are introduced to understand the hidden features of different event relationship types by treating each event relationship as a set of separate attention mechanisms. Finally, the feed-forward neural network and the average link clustering method are used to calculate the coreference score of events and complete the coreference event clustering, and the cross-document event coreference resolution is realized. The experimental results show that the cross-document event co-reference resolution method is superior to the baseline model.

An Enhanced Edge Computing Technique for Detection of Voltage Fluctuation in Grid-tied Renewable Energy

Renewable energy sources (RES) such as solar photovoltaic and wind are becoming the most attractive power generation options in many nations. Even while high penetration seems likely, power quality anomalies such as voltage fluctuation, harmonics, and frequency fluctuation associated with RES hinder seamless integration. The variability and unpredictability of these sources create the most oddities. In grid-tied renewable energy, monitoring power quality efficiently is crucial. Power grid monitoring solutions in related literature use sensor-based cloud and edge computing techniques. The existing systems struggle with excessive latency when delivering large amounts of generated data to the cloud. To fill this gap, a new approach for the detection and localization of voltage fluctuation is proposed in this study. The approach integrated three techniques namely; feed-forward neural network (FFNN), Stockwell transform, and anomaly-aware edge computing to detect and locate voltage fluctuation in a GtRE. Using MATLAB/Simulink, virtual emulation of a modified IEEE 33 Bus and a GtRE representing a section of Ado Ekiti (in Nigeria) low-voltage distribution grid are carried out for data generation and system evaluation. Feature extraction was carried out in a Python IDE using Stockwell transform. The voltage fluctuation events are detected and localized based on the extracted features using the trained FFNN model deployed and evaluated within three microcontroller-based computing devices. The proposed approach integrated anomaly-aware with edge computing to send only voltage data that are considered abnormal to a dedicated data center for visualization and storage. Performance evaluation of the proposed technique on the simulated GtRE demonstrates a significant decrease of 98% and 90% in latency when compared to cloud computing and conventional edge computing respectively. Comparison of the proposed approach to two closely related solutions in literature also demonstrates a 50% and 92.5 % reduction in latency. The contribution of the study is the reduced latency and minimal bandwidth utilization achieved by the implementation of the developed technique.

Resource optimization in smart electronic health systems using IoT for heart disease prediction via feedforward neural networks

Resource optimization in smart electronic health systems using IoT for heart disease prediction via feedforward neural networks

Estimating Interplanetary Magnetic Field Conditions at Mercury's Orbit From MESSENGER Magnetosheath Observations Using a Feedforward Neural Network

AbstractMercury's small magnetosphere is embedded in the dynamic and intense solar wind environment characteristic of the inner heliosphere. Both the magnitude and orientation of the interplanetary magnetic field (IMF) significantly influence the solar wind‐magnetospheric interaction at Mercury, driving phenomena such as magnetic reconnection. The MErcury Surface, Space Environment, Geochemistry and Ranging (MESSENGER) spacecraft provided in‐situ magnetic field measurements of the solar wind, the magnetosheath, and the magnetosphere along each orbit. However, it is a challenge to directly assess the IMF's impact on Mercury's plasma environment due to the temporal separation between observations within the solar wind and the magnetosphere, especially in the absence of an upstream monitor. Here, we present a feedforward neural network (FNN) trained on a subset of magnetosheath observations to estimate the strength and orientation of the IMF upstream of the bow shock. Utilizing magnetosheath magnetic field, cylindrical spatial coordinates, and heliocentric distance measurements, the FNN predicts upstream IMF conditions with an score of 0.70 and mean averaged error of 5.3 nT, thereby greatly decreasing the temporal separation between IMF estimates and magnetospheric measurements throughout the MESSENGER mission. This approach yields IMF estimates for all magnetosheath data measured by MESSENGER, providing a useful tool for future investigations of the IMF impact on Mercury's magnetosphere. This method will be integrable with the dual‐spacecraft BepiColombo magnetosheath measurements, providing useful estimates of upstream IMF conditions particularly during the extended periods in which neither spacecraft sample the solar wind. Our results demonstrate the utility of machine learning techniques on advancing space science research.

Application of feedforward and recurrent neural networks for model-based control systems

AbstractIn this paper, a new study concerning the usage of artificial neural networks in the control application is given. It is shown, that the data gathered during proper operation of a given control plant can be used in the learning process to fully embrace the control pattern. Interestingly, the instances driven by neural networks have the ability to outperform the original analytically driven scenarios. Three different control schemes, namely perfect, linear-quadratic, and generalized predictive controllers were used in the theoretical study. In addition, the nonlinear recurrent neural network-based generalized predictive controller with the radial basis function-originated predictor was obtained to exemplify the main results of the paper regarding the real-world application.

Hysteresis compensation and decoupling control of an XYΘ-type flexure-based mechanism via inverse hysteresis-coupling hybrid modeling

Planar positioning systems are widely utilized in micro and nano applications. The challenges in modeling and control of XYΘ flexure-based mechanisms include hysteresis of the piezoelectric actuators, couplings among the input axes, and coupled linear and angular motions of the end effector. This paper presents an inverse hysteresis-coupling hybrid model to account for such hysteresis and couplings. First, a specially designed kinematic chain is adopted to transfer the pose of the end effector into the linear motions at three prismatic joints. Second, an inverse hysteresis-coupling hybrid model is developed to linearize and decouple the system via a multilayer feedforward neural network. A fractional-order PID controller is also integrated to improve the motion accuracy of the overall system. Experimental results demonstrate that the proposed method can accurately control the motion of the end effector with improved accuracy and robustness.