Feedforward Mode Research Articles

Sequential Recommendation Model Based on Deep Residual Recurrent Neural Network

In order to solve the problem that the sequence recommendation model based on RNN (Recurrent Neural Network) is prone to gradient vanishing or exploding when processing long sequences, which leads to the instability of the recommendation model training process, residual connection, layer normalization and feedforward neural network modules are introduced on the basis of the traditional gated recurrent unit (GRU), and a sequence recommendation model DeepGRU based on deep residual recurrent neural network is proposed. It is verified on 3 public datasets. The experimental results show that the DeepGRU has obvious advantages over the most advanced sequence recommendation methods (the average recommendation accuracy is improved 8.68%). The ablation experiment verifies the effectiveness of the introduced residual connection and other modules under the DeepGRU framework. In addition, the DeepGRU effectively alleviates the problem of unstable training process when processing long sequences.

Energy Efficient Data Aggregation in Wireless Sensor Networks Using Meta Heuristic Based Feed Forward Back Propagation Neural Network Approach

Sensor nodes are low-cost, low-power, tiny devices that make up the majority of WSNs, or distributed, self-organizing systems. These sensor nodes are able to exchange, perceive, and interpret data. The sensor nodes are equipped with a wide variety of sensors, such as chemical, touch, motion, temperature, and weather sensors. Because of its adaptability, sensors are used in a variety of applications such as automation, tracking, monitoring, and surveillance. Despite the enormous number of sensor applications, WSNs continue to suffer from common challenges like as low memory, slow processing speed, and short network lifetime. The feed forward back propagation neural network mode (FFBPNN) based on meta heuristics aims to create many paths for effective data aggregation in wireless sensor networks. This model handled the process of identifying and selecting the optimum route path. The distributed sensor nodes are utilized to create the various route paths. In this research paper, data aggregation is done using meta-heuristic firefly algorithm that helped in identifying an optimal route from among the found routes. After selecting the operative ideal route choice, the data aggregation procedure practices a rank-based approach to accomplish lower latency and a better packet delivery ratio(PDR). In addition to throughput, simulation was done to improve and measure performance in terms of packet delivery ratio, energy consumption, and end-to-end latency.

FEFN: Feature Enhancement Feedforward Network for Lightweight Object Detection in Remote Sensing Images

Object detection in remote sensing images has become a crucial component of computer vision. It has been employed in multiple domains, including military surveillance, maritime rescue, and military operations. However, the high density of small objects in remote sensing images makes it challenging for existing networks to accurately distinguish objects from shallow image features. These factors contribute to many object detection networks that produce missed detections and false alarms, particularly for densely arranged objects and small objects. To address the above problems, this paper proposes a feature enhancement feedforward network (FEFN), based on a lightweight channel feedforward module (LCFM) and a feature enhancement module (FEM). First, the FEFN captures shallow spatial information in images through a lightweight channel feedforward module that can extract the edge information of small objects such as ships. Next, it enhances the feature interaction and representation by utilizing a feature enhancement module that can achieve more accurate detection results for densely arranged objects and small objects. Finally, comparative experiments on two publicly challenging remote sensing datasets demonstrate the effectiveness of the proposed method.

LVP-net: A deep network of learning visual pathway for edge detection

Deep learning-based edge detectors typically consist of the encoder and the decoder. To integrate multi-scale features into a global edge map effectively, researchers utilize classification networks such as VGG16 as the encoder and focus on the decoder architecture. In contrast to existing approaches, we propose a novel deep network for edge detection called learning-visual-pathway network (LVP-Net), in which an enhancer-encoder-decoder architecture is designed inspired by the biological visual pathway: the retina/lateral geniculate nucleus→the primary visual cortex (V1) → V2 → V4 → the inferior temporal cortex (IT). To simulate the visual mechanisms along this pathway, we design a feature enhancer network (FENet) that boosts the feature representation capability of the encoder. FENet is combined with VGG16 based on the hierarchical structure of the pathway. Furthermore, inspired by the integration ability of multiple features in IT, we introduce a feedforward fusion module (FFM). Finally, we evaluate LVP-Net on three benchmark datasets, i.e., BSDS500, NYUDv2, and Multicue. Experimental results demonstrate that our method achieves competitive performance compared with most state-of-the-art approaches.

Water resource management and flood mitigation: hybrid decomposition EMD-ANN model study under climate change

The growing population and the rise in urbanization have made managing water a critical concern around the world in recent years. Globally, flooding is one of the most devastating natural disasters. Flood risk mitigation relies heavily on accurate and consistent streamflow forecasts. Pakistan Upper Indus Basin (UIB) is most vulnerable to flooding. Floods have become more frequent in recent decades. UIB can be divided into sub-regions due to its landscape variability, and its collective impact is most prominent in the Massam region. UIB hydrological and meteorological station observations have been used to study seasonal hydro-meteorological variations. To predict flooding, this study proposes a hybrid model combining artificial neural networks as multi-layer perceptron (MLPs) in feed-forward mode, along with empirical mode decomposition (EMD). Data collected by the surface-water hydrology project and Pakistan Meteorological Department from 1960 to 2012, 1969 to 2012, and 1972 to 2012 have been utilized from 17 locations. Statistical parameters and Nash–Sutcliffe Efficiency were measured to analyze the model’s prowess. As a result, decomposition-based models perform better than AI-based models when it comes to prediction accuracy. MLPQTP-EMD performed exceptionally better than competing AI models. The results are further validated by performing a peak value analysis during the flooding season (June–September) achieving a remarkable 91.3% score adding a 5.6% increase by EMD for input data achieving 39.3–32.3% statistical indices scores.

A low-power sigma-delta modulator based on high-order op-amp sharing technique for speech communication

In this paper, a novel idea of sigma-delta modulator (SDM) is proposed based on our high-order (greater than or equal to 4th order) op-amp sharing technology. The operation timing, adder, integrator structure and the switched-capacitor op-amps common-mode negative-feedback of the conventional single-loop cascade-of-integrators with feedforward (CIFF) modulator are improved, so that the nth-order integration function can be accomplished by using one op-amp. Finally, a high-precision and low-power sigma-delta modulator for speech communication is realized. The proposed 4th-order CIFF sigma-delta modulator based on our high-order op-amp sharing is utilized in 0.18 µm standard CMOS process and occupies an area of 0.218 mm2. Simulation results show that the SNDR is 103.91 dB and the ENOB is 16.97 bits with a sampling rate of 2.56 MHz, an oversampling ratio of 256, and signal bandwidth of 5 kHz. Proposed modulator has a supply voltage of 5 V and costs a power consumption of 2.29mW. The power consumption of proposed op-amp sharing idea is 29.67 % less than that of the conventional case.

Study of a High-Precision Read-Out Integrated Circuit for Bridge Sensors.

Bridge sensors are widely used in military and civilian fields, and their demand gradually increases each year. Digital sensors are widely used in the military and civilian fields. High-precision and low-power analog-to-digital converters (ADCs) as sensor read-out circuits are a research hotspot. Sigma-delta ADC circuits based on switched-capacitor topology have the advantages of high signal-to-noise ratio (SNR), good linearity, and better compatibility with CMOS processes. In this work, a fourth-order feed-forward sigma-delta modulator and a digital decimation filter are designed and implemented with a correlated double sampling technique (CDS) to suppress pre-integrator low-frequency noise. This work used an active pre-compensator circuit for deep phase compensation to improve the system's stability in the sigma-delta modulator. The modulator's local feedback factor is designed to be adjustable off-chip to eliminate the effect of process errors. A three-stage cascade structure was chosen for the post-stage digital filter, significantly reducing the number of operations and the required memory cells in the digital circuit. Finally, the layout design and engineering circuit were fabricated by a standard 0.35 μm CMOS process from Shanghai Hua Hong with a chip area of 9 mm2. At a 5 V voltage supply and sampling frequency of 6.144 MHz, the modulator power consumption is 13 mW, the maximum input signal amplitude is -3 dBFs, the 1 Hz dynamic range is about 118 dB, the modulator signal-to-noise ratio can reach 110.5 dB when the signal bandwidth is 24 kHz, the practical bit is about 18.05 bits, and the harmonic distortion is about -113 dB, which meets the design requirements. The output bit stream is 24 bits.

FSTT: Flow-Guided Spatial Temporal Transformer for Deep Video Inpainting

Video inpainting aims to complete the missing regions with content that is consistent both spatially and temporally. How to effectively utilize the spatio-temporal information in videos is critical for video inpainting. Recent advances in video inpainting methods combine both optical flow and transformers to capture spatio-temporal information. However, these methods fail to fully explore the potential of optical flow within the transformer. Furthermore, the designed transformer block cannot effectively integrate spatio-temporal information across frames. To address the above problems, we propose a novel video inpainting model, named Flow-Guided Spatial Temporal Transformer (FSTT), which effectively establishes correspondences between missing regions and valid regions in both spatial and temporal dimensions under the guidance of completed optical flow. Specifically, a Flow-Guided Fusion Feed-Forward module is developed to enhance features with the assistance of optical flow, mitigating the inaccuracies caused by hole pixels when performing MHSA. Additionally, a decomposed spatio-temporal MHSA module is proposed to effectively capture spatio-temporal dependencies in videos. To improve the efficiency of the model, a Global–Local Temporal MHSA module is further designed based on the window partition strategy. Extensive quantitative and qualitative experiments on the DAVIS and YouTube-VOS datasets demonstrate the superiority of our proposed method.

Remote heart rate estimation via convolutional neural networks with transformers

Remote photoplethysmography (rPPG) uses subtle color changes in facial videos to estimate heart rate(HR). However, recent methods face challenges in solving remote heart rate estimation tasks, because the color changes in facial skin are very subtle and the pseudo-periodicity of rPPG requires long-distance temporal detection. To address these issues of the rPPG estimation task, we propose a convolution neural network with transformers for rPPG estimation which takes the advantages of convolutions in the locality and transformers in long-range dependencies. Specifically, we first proposed a local feed-forward module following the multi-head self-attention to compensate for the difficulties of transformers in capturing neighboring feature information. Then we add the relative and absolute position encoding to obtain the ordering of the tokens which is the key to capturing the pseudo-periodicity of rPPG. Furthermore, the temporal multi-scale module is proposed to learn the temporal information from different scales. Extensive experimental results demonstrate that our method is superior to the state-of-the-art results on the COHFACE, UBFC-rPPG, PURE, and VIPL-HR databases compared with traditional methods and deep learning-based methods. We also evaluated our method under different conditions on the VIPL-HR database, and the results show that our method is robust to various conditions.

Influence of feedforward control-based health education intervention on compliance, visual function and self-perceived burden among patients with diabetic retinopathy.

To assess the influence of feedforward control-based health education intervention on the compliance, visual function and self-perceived burden (SPB) among patients with diabetic retinopathy (DR). Eighty-six DR patients were divided into feedforward control and control groups (n=43). The control group was given routine nursing intervention, based on which the feedforward control group received feedforward control-based health education intervention. The health behavior indices were compared after intervention. The correlations of QOL score with SPB score and health behavior indices were analysed using Pearson's and Spearman's coefficients. After intervention, the total QOL score and scores of symptoms and visual function, physical function, social activity, and mentality and psychology were significantly improved compared with those before intervention, which were significantly higher in the feedforward control group (P<0.05). SPB score was significantly lower in the two groups after intervention than that before intervention, particularly in the feedforward control group (P<0.05). The QOL score of DR patients was significantly negatively correlated with SPB score but positively correlated with health behavior indices (P<0.05). The feedforward control-based health education intervention mode is beneficial for guiding DR patients to promote visual function recovery and to reduce SPB.

Top-Down Number Reading: Language Affects the Visual Identification of Digit Strings.

Reading numbers aloud involves visual processes that analyze the digit string and verbal processes that produce the number words. Cognitive models of number reading assume that information flows from the visual input to the verbal production processes-a feed-forward processing mode in which the verbal production depends on the visual input but not vice versa. Here, I show that information flows also in the opposite direction, from verbal production to the visual input processes. Participants read aloud briefly presented multi-digit strings in Hebrew, in which the order of words is congruent with the order of digits (21 = twenty-and-one), and in Arabic, in which the ones word precedes the tens word (one-and-twenty). The error-by-digit-position curve was affected by language: relative to Hebrew, in Arabic the error rate was slightly lower for the unit digit and slightly higher for the decade digit, indicating that in Arabic the unit digit was processed earlier and the decade digit later, in accord with the Arabic word order. This language-dependent processing order originated in the visual level and was not a verbal confound, because it persisted even when I controlled for the serial position of the decade/unit word in the verbal number by using numbers with 0 (two hundred three/two hundred thirty). I conclude that the visual analyzer's digit scanning order, decade-first or unit-first, is not fixed but affected by the language in which the number is produced-a top-down, verbal-to-visual information flow.

Smart robots’ virus defense using data mining technology

Abstract In order to realize online detection and control of network viruses in robots, the authors propose a data mining-based anti-virus solution for smart robots. First, using internet of things (IoT) intrusion prevention system design method based on network intrusion signal detection and feedforward modulation filtering design, the overall design description and function analysis are carried out, and then the intrusion signal detection algorithm is designed, and finally, the hardware design and software development for a breach protection solution for the IoT are completed, and the integrated design of the system is realized. The findings demonstrated that based on the mean value of 10,000 tests, the IoT’s average packet loss rate is 0. Conclusion: This system has high accuracy, good performance, and strong compatibility and friendliness.

Uncovering the Early Events Associated with Oligomeric Aβ-Induced Src Activation.

Soluble Aβ1-42 oligomers (AβO) are formed in the early stages of Alzheimer's disease (AD) and were previously shown to trigger enhanced Ca2+ levels and mitochondrial dysfunction via the activation of N-methyl-D-aspartate receptors (NMDAR). Src kinase is a ubiquitous redox-sensitive non-receptor tyrosine kinase involved in the regulation of several cellular processes, which was demonstrated to have a reciprocal interaction towards NMDAR activation. However, little is known about the early-stage mechanisms associated with AβO-induced neurodysfunction involving Src. Thus, in this work, we analysed the influence of brief exposure to oligomeric Aβ1-42 on Src activation and related mechanisms involving mitochondria and redox changes in mature primary rat hippocampal neurons. Data show that brief exposure to AβO induce H2O2-dependent Src activation involving different cellular events, including NMDAR activation and mediated intracellular Ca2+ rise, enhanced cytosolic and subsequent mitochondrial H2O2 levels, accompanied by mild mitochondrial fragmentation. Interestingly, these effects were prevented by Src inhibition, suggesting a feedforward modulation. The current study supports a relevant role for Src kinase activation in promoting the loss of postsynaptic glutamatergic synapse homeostasis involving cytosolic and mitochondrial ROS generation after brief exposure to AβO. Therefore, restoring Src activity can constitute a protective strategy for mitochondria and related hippocampal glutamatergic synapses.

Active disturbance rejection control for the reheated steam temperature of a double-reheat boiler

As a state-of-the-art thermal power generation technology with high efficiency and low emissions, the double-reheat boiler is facing many control challenges on the reheated steam temperature (RST), such as nonlinearity, coupling, and large inertia lag. However, the widely used Proportional–Integral–Derivative (PID) may not efficiently handle these challenges. To this end, a control scheme based on active disturbance rejection control (ADRC) is synthesized, after analyzing the nonlinearity of the system and the decoupling of a control structure for the mean and deviation of temperature (MDT) in the frequency domain. The distinctive features of the proposed scheme are (i) the MDT structure is utilized to decouple; (ii) Modified active disturbance rejection control (MADRC) is used to handle large inertia lag; (iii) the compensation part of MADRC is adaptive to load, and the temperature setpoints and feedforward modules are optimized to reduce nonlinearity. The superiority of the proposed control system is examined by simulation experiments. Lastly, ADRC was successfully applied to the practical RST system, showing better temperature control quality than the initial manual mode. The analysis and practice in this paper demonstrate the feasibility of ADRC for the RST system and offer guidance for further study on RST control.

PF-BERxiT: Early exiting for BERT with parameter-efficient fine-tuning and flexible early exiting strategy

The industrial usage of huge pre-training language models such as BERT and ALBERT are limited by the computational probability problem in the fine-tuning process and overthinking problem in the inference process. PF-BERxiT has been proposed to optimize the pre-trained languages with a novel parameter-efficient fine-tuning method and a flexible early exiting strategy. Significantly, the new parameter-efficient fine-tuning model integrates a bottleneck adapter architecture parallel to the transformer architecture, and only the adapter's parameters are adjusted. In addition, we integrate an extra sub-learning module to learn the samples' characteristics, improving the accuracy and efficiency simultaneously. The flexible exiting strategy allows the model to exit early if the similarity score of adjacent layers is less than the threshold for pre-defined times. It is more flexible than previous early exiting methods, for it can simultaneously adjust the similarity score thresholds and patience parameters according to the request traffic. Extensive experiments are conducted on the GLUE benchmark, demonstrating that: (1) PF-BERxiT outperforms conventional training and parameter-efficient strategies with only a few parameters fine-tuned. (2) PF-BERxiT strikes a better balance between model performances and speedup ratios than previous state-of-the-art (SOTA) early exiting methods such as PABEE and BERxiT. (3) Ablation studies in the fine-tuning process demonstrate that the best bottleneck dimension r of the adapters is 32, and the adapters placed parallel to the feed-forward module are more efficient. (4) Ablation studies in the inference process demonstrate that for variants of PF-BERxiT with different similarity scores, PF-BERxiT-kl and PF-BERxiT-bikl attain better speedup-accuracy trade-offs than PF-BERxiT-rekl. Our PF-BERxiT helps attain a better trade-off between performance and efficiency, providing a reference for the efficient application of neural computing.

Enhancing projection based iterative learning control: A set-membership approach

Lithographic machine tools require both high motion accuracy and high motion flexibility. Projection based iterative learning control (P-ILC) is appealing for wafer stages to achieve two goals, simultaneously. P-ILC contains a nonparametric feedforward controller based on ILC, and a parametric feedforward controller with a projection step for feedforward tuning. In this paper, a set-membership based frequency-domain ILC algorithm (SM-F-ILC) is employed in the enhancing P-ILC scheme to improve the performance in the nonparametric feedforward control mode. SM-F-ILC can effectively compensate for repetitive errors, attenuate the nonrepetitive error accumulation and achieve fast convergence speed with model uncertainties. These superiorities also facilitate to improve the performance of P-ILC in the parametric feedforward control mode. The validity of the enhancing P-ILC scheme is demonstrated by experimental results.

Research of Continuous Delta-Sigma Analog-to-Digital Converter for Bio-Electricity Signal Acquisition

In this paper, a continuous delta-sigma analog-to-digital converter with a low power, single-ring, third-order mixed integrator for bio-electricity signal acquisition is proposed. For the trade-off between low power consumption and high resolution, the Active-RC integrator is used in the first-order of the third-order feedforward modulator and the Gm-C integrator with improved linearity is adopted for the second and third order integrators. Because of the infinite resistance provided by the Gm-C integrator, the first-order integrator can replace the traditional second-order operational transconductance amplifier (OTA) with a single-stage OTA operating in the weak invert region to achieve low power consumption and ensure the output swing of the first-order integrator. The structure of cascade-of-integrators with feedforward (CIFF) can scale the output voltage values of three integrators and reduce the requirement of linearity of the Gm-C integrator, to reduce the design difficulty of the Gm-C integrator. In addition, the linearity of the second and third-order Gm-C integrator is improved by using the auxiliary difference pair OTA with source-level negative feedback. Therefore, the matching between the circuit structure and the model coefficients is improved significantly.

Scenario Adaptive Cuffless Blood Pressure Estimation by Integrating Cardiovascular Coupling Effects.

Adding cuffless blood pressure (BP) measurement function to wearable devices is of great value in the fight against hypertension. The widely used arterial pulse transit time (PTT)-based method for BP monitoring relies primarily on vascular status-determined BP models and typically exhibits degraded performance over time and is sensitive to measurement procedures. Developing alternative methods with improved accuracy and adaptability to various application scenarios is highly desired for cuffless BP measurement. In this work, we proposed a pattern-fusion (PF) method that incorporates cardiovascular coupling effects in the vascular model by combining three calculation modules - cardiac parameter extraction module, cardiac parameter-to-BP mapping module, and BP regulation module. Specifically, the first module combines feedforward, feedback, and propagation modes to model different modulation functions of a cardiovascular system and is responsible for extracting BP-related features from electrocardiography (ECG) and photoplethysmography (PPG) signals; the cardiac parameter-to-BP mapping module is used to map cardiac parameters into mean blood pressure (MBP) by fusing different features; finally, the BP regulation module recovers accurate systolic BP (SBP) and diastolic BP (DBP) from given MBP. With the concerted use of these three modules, the pattern fusion method consistently demonstrates excellent BP prediction accuracy in a variety of measurement scenarios and durations, exhibiting SBP/DBP mean absolute error (MAE) of 3.65/4.56 mmHg for the short-term (<10 mins) continuous measurement dataset, SBP/DBP MAE of 6.84/3.81 mmHg for the medium-term (avg. > 20 hours) continuous measurement dataset, and SBP/DBP MAE of 6.24/3.65 mmHg for the long-term (> 1 month) intermittent measurement dataset.

Large-tunable microwave photonics frequency multiplier overcoming the fixed filter bandwidth limitation on improved feed-forward modulation technology

Improved feed-forward modulation (IFFM) technology is a two-stage modulation photonics design that achieves coherent frequency up-conversion based on incoherent optical carriers. For the scheme, at least two optical fixed filters are required; the first is used to extract desired optical sidebands for the second-stage modulation’s driving signal generation, and the second filter is to suppress unwanted optical harmonics for high-quality heterodyne detection. This is for a fixed-frequency signal generation. To accommodate tunable frequency signal generation, at least two tunable optical filers are needed. In this paper, a wide frequency tunable method for an IFFM-based frequency multiplier with only a fixed bandwidth optical filter is proposed. A proof-of-concept experiment successfully demonstrates the tuning mechanism of the scheme. The scheme exploits the two pairs of coherent optical sidebands generated from two incoherent light sources to realize coherent addition of the two beat signals from each pair to attain a high-power frequency-multiplied signal for enhanced transmission power budget. From the simulation, a frequency multiplication factor of 16 and a 160-GHz signal with a stable frequency and a power level enhanced by 19 dB are achieved by employing two incoherent lasers. By adjusting the output frequency of the local oscillator, we show that multi-band signals generation, including microwave, millimeter- and terahertz-wave, can be realized, and the performance of the generated signals is also investigated.

Design of an Improved Active Disturbance Rejection Control Method for a Direct-Drive Gearshift System Equipped with Electromagnetic Linear Actuators in a Motor-Transmission Coupled Drive System

In this study, a type of direct-drive gearshift system integrated into a motor-transmission coupled drive system is introduced. It used two electromagnetic linear actuators (ELAs) to perform gearshift events. The adoption of ELAs simplifies the architecture of the gearshift system and has the potential to further optimize gearshift performance. However, a number of nonlinearities in the gearshift system should be investigated in order to enhance the performance of the direct-drive gearshift system. An active disturbance rejection control (ADRC) method was selected as the principal shifting control method due to the simple methodology and strong reliability. The nonlinear characteristics of the electromagnetic force produced by the ELA were subsequently reduced using the inverse system method (ISM) technique. The ADRC approach also incorporated an acceleration feedforward module to enhance the precision of displacement control. The extended state observer (ESO) module used a nonlinear function in place of the original function to improve the ability to reject disturbances. Comparative simulations and experiments were carried out between the ADRC method and improved ADRC (IADRC) method. The outcomes demonstrate the effectiveness of the designed control method. The shift force fluctuates less, and the shift jerk decreases noticeably during the synchronization procedure. In conclusion, combined with the optimized IADRC method, the direct-drive gearshift system equipped with ELAs shows remarkable gearshift performance, and it has the potential to be widely used in motor−transmission coupled drive systems for EVs.