Recursive Weight Research Articles

A Neural Network Weights Initialization Approach for Diagnosing Real Aircraft Engine Inter-Shaft Bearing Faults

The deep learning diagnosis of aircraft engine-bearing faults enables cost-effective predictive maintenance while playing an important role in increasing the safety, reliability, and efficiency of aircraft operations. Because of highly dynamic and harsh operating conditions of this system, such modeling is challenging due to data complexity and drift, making it difficult to reveal failure patterns. As a result, the objective of this study is dual. To begin, a highly structured data preprocessing strategy ranging from extraction, denoising, outlier removal, scaling, and balancing is provided to solve data complexity that resides specifically in outliers, noise, and data imbalance problems. Gap statistics under k-means clustering are used to evaluate preprocessing results, providing a quantitative estimate of the ideal number of clusters and thereby enhancing data representations. This is the first time, to the best of authors’ knowledge, that such a criterion has been employed for an important step in a preliminary ground truth validation in supervised learning. Furthermore, to tackle data drift issues, long-short term memory (LSTM) adaptive learning features are used and subjected to a learning parameter improvement method utilizing recursive weights initialization (RWI) across several rounds. The strength of such methodology can be seen by application to realistic, extremely new, complex, and dynamic data collected from a real test-bench. Cross validation of a single LSTM layer model with only 10 neurons shows its ability to enhance classification performance by 7.7508% over state-of-the-art results, obtaining a classification accuracy of 92.03 ± 0.0849%, which is an exceptional performance in such a benchmark.

An incremental approach to forecasting and classification of taxi demand based on evolving fuzzy systems

Private transport has become a viable and increasingly popular alternative to urban transportation. However, with this growth, an old and recurring problem becomes more latent: the relationship between passenger demands and taxi supply. This problem suggests the creation and use of techniques which make it possible to reduce the gap between the demand for taxi passengers and the effective contingent of vehicles needed to meet this demand. This work introduces a new approach to forecasting and classifying taxi passengers’ demands. The proposed approach uses historical data from taxi rides and meteorological data. The Kruskal-Wallis method identifies the most relevant variables, and an evolving fuzzy system performs demand forecasting/classification. Five evolving systems are evaluated with our approach: Autonomous Learning Multi-Model (ALMMo), evolving Multivariable Gaussian Fuzzy System (eMG), evolving Fuzzy with Multivariable Gaussian Participatory Learning and Recursive Maximum Correntropy (eFCE), evolving Fuzzy with Multivariable Gaussian Participatory Learning and Multi-Innovations Recursive Weighted Least Squares (eFMI), and evolving Neo-Fuzzy Neuron (eNFN). In addition, computational experiments using real-world data were conducted to evaluate and compare the performance of the proposed approach. The results revealed that it obtained performance superior or comparable to state-of-the-art ones. Therefore, the experimental results suggest that the proposed approach is promising as an alternative for forecasting and classifying taxi passenger demand.

Design of Decision Based Recursive Weighted Median Filter With Exponential Weights

The prescribed algorithm for removing impulse noise effectively even under high noise densities without causing any loss of image details. Hence a cascaded section of median filters that, involves an Decision-based Median Filter followed by a Recursive Weighted Median (RWM) Filter employing exponential weights are used. The median controlled algorithm is employed to calculate the exponential weights. In the algorithms that where proposed in earlier which involves a cascaded section of the median with the RWM filters provided lesser Peak Signal/Noise Ratio (PSNR) and greater Mean Square Error(MSE) values. Hence the output appeared to be distorted for higher noise levels. These drawbacks have been eliminated in this proposed algorithm.

Motion-Robust Multimodal Heart Rate Estimation Using BCG Fused Remote-PPG With Deep Facial ROI Tracker and Pose Constrained Kalman Filter

The heart rate (HR) signal is so weak in remote photoplethysmography (rPPG) and ballistocardiogram (BCG) that HR estimation is very sensitive to face and body motion disturbance caused by spontaneous head and body movements as well as facial expressions of subjects in conversation. This article proposed a novel multimodal quasi-contactless HR sensor to ensure the robustness and accuracy of HR estimation under extreme facial poses, large-motion disturbances, and multiple faces in a video for computer-aided police interrogation. Specifically, we propose a novel landmark-based approach for a deep facial region of interest (ROI) tracker and face pose constrained Kalman filter to continuously and robustly track target facial ROIs for estimating HR from face and head motion disturbances in rPPG. This motion-disturbed rPPG signal is further fused with a minimally disturbed BCG signal by the face and head movements via a bank of notch filters with a recursive weighting scheme to obtain the dominant HR frequency for final accurate HR estimation. To facilitate reproducible HR estimation research, we synchronously acquire and publicly share a multimodal data set that contains 20 sets of ECG and BCG signals as well as uncompressed, rPPG-dedicated videos from ten subjects in a stable state and large-motion state (MS) without and with large face and body movements in a sitting position. We demonstrate through experimental comparisons that the proposed multimodal HR sensor is more robust and accurate than the state-of-the-art single-modal HR sensor solely with rPPG- or BCG-based methods. The mean absolute error (MAE) of HR estimation is 7.13 BPM lower than the BCG algorithm and 3.12 BPM lower than the model-based plane-orthogonal-to-skin (POS) algorithm in the MS.

An 8.93 TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity for On-Device Speech Recognition

Long short-term memory (LSTM) is a type of recurrent neural networks (RNNs), which is widely used for time-series data and speech applications, due to its high accuracy on such tasks. However, LSTMs pose difficulties for efficient hardware implementation because they require a large amount of weight storage and exhibit computation complexity. Prior works have proposed compression techniques to alleviate the storage/computation requirements of LSTMs but elementwise sparsity schemes incur sizable index memory overhead and structured compression techniques report limited compression ratios. In this article, we present an energy-efficient LSTM RNN accelerator, featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by the HCGS-based blockwise recursive weight compression, we demonstrate LSTM networks with up to 16 $\times $ fewer weights while achieving minimal error rate degradation. The prototype chip fabricated in 65-nm LP CMOS achieves up to 8.93 TOPS/W for real-time speech recognition using compressed LSTMs based on HCGS. HCGS-based LSTMs have demonstrated energy-efficient speech recognition with low error rates for TIMIT, TED-LIUM, and LibriSpeech data sets.

Lyapunov stability-Dynamic Back Propagation-based comparative study of different types of functional link neural networks for the identification of nonlinear systems

In this paper, the performance comparison of various types of functional link neural networks (FLNNs) has been done for the nonlinear system identification. The FLNNs being compared in the present study are: trigonometry FLNN, Legendre FLNN (LeFLNN), Chebyshev FLNN, power series FLNN (PSFLNN) and Hermite FLNN. The recursive weights adjustment equations are derived using the combination of Lyapunov stability criterion and dynamic back propagation algorithm. In the simulation study, a total of three nonlinear systems (both static and dynamic systems) are considered for testing and comparing the approximation ability and computational complexity of the above-mentioned FLNNs. From the simulation results, it is observed that the LeFLNN has given better approximation accuracy and PSFLNN offered least computational load as compared to the rest models.

A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity With All Parameters Stored On-Chip

Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator, featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression, we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

Fast Non-Local Mean Filter Algorithm Based on Recursive Calculation of Similarity Weights

Abstract—A theoretically derived technique for acceleration of the original non-local means image denoising algorithm based on calculation of recursive patch similarity weights is proposed. A significant amount of computation in the non-local means scheme is dedicated to estimation of the patch similarity between pixel neighborhoods. The proposed recursive weights calculation scheme adopts the classic recursive mean calculation scheme for a multidimensional shift-vector in order to lower the computational complexity of the original non-local means method, thus speeding up this algorithm more than tenfold. Note that the output of the proposed algorithm is exactly the same as that of the original non-local means method. Hence this algorithm belongs to the class of true fast algorithms, unlike methods approaching to a certain degree the resul of the original algorithm.

Transmission Line Protective Relay Based on Recursive Least-Square Filters and Weights Analysis

The purpose of this study takes advantage of real-time hardware and software co-simulation to integrate a recursive least-square adaptive filter into a relay design for transmission line protection. The relay trip logic is controlled with a recursive least-square filter that analyzes the weights and cumulative error auto-correlation for magnitude and angles per phase. This analysis allows to detect and identify the fault type. The study focuses on the modelling of a consistent testing case study designed for transmission lines protections based on the EMTP reference model. The proposed solution could be scalable to protect other elements in power systems. It is consistent with current technological tools, and the RLS algorithm optimizes the computational performance of the relay.

An Optimal Algorithm for Online Non-Convex Learning

In many online learning paradigms, convexity plays a central role in the derivation and analysis of online learning algorithms. The results, however, fail to be extended to the non-convex settings, while non-convexity is necessitated by a large number of recent applications. The Online Non-Convex Learning (ønco) problem generalizes the classic Online Convex Optimization (øco) framework by relaxing the convexity assumption on the cost function (to a Lipschitz continuous function) and the decision set. The state-of-the-art result for the ønco demonstrates that the classic online exponential weighting algorithm attains a sublinear regret of $O(\sqrtTłog T )$. The regret lower bound for the øco, however, is $Ømega(\sqrtT )$, and to the best of our knowledge, there is no result in the context of the ønco problem achieving the same bound. This paper proposes the Online Recursive Weighting (\rw) algorithm with regret of $O(\sqrtT )$, matching the tight regret lower bound for the øco problem, and fills the regret gap between the state-of-the-art results in the online convex and non-convex optimization problems.

Kernel Recursive Generalized Maximum Correntropy

In this letter, a novel kernel adaptive algorithm, called kernel recursive generalized maximum correntropy algorithm (KRGMC), is derived in a kernel space and under the generalized maximum correntropy (GMC) criterion. The proposed kernel algorithm can effectively scale down the dynamic recursive weight coefficients influenced by the impulsive estimate error to avoid the significant performance degradation. The superior performance of the proposed algorithm is verified by numerical simulations about short-time series prediction in alpha-stable noise environment.

Kernel Least Mean $p$-Power Algorithm

This letter proposes a novel kernel least mean $p$ -power (KLMP) algorithm for nonlinear system identification in the presence of additive non-Gaussian impulsive noises, modeled by a symmetric $\alpha$ -stable distribution with heavy tail. The KLMP algorithm based on the fractional lower order statistics error criterion can effectively scale down the dynamic recursive weight coefficients affected by the impulsive estimation error to avoid the significant performance degradation. Simulation results demonstrate that the proposed algorithm has favorable convergence properties than the classical kernel least-mean-square algorithm using a conventional error criterion in the non-Gaussian impulsive environment.

Semi-cubic Hyponormality of Weighted Shifts with Stampfli Recursive Tail

Let \(\alpha :\sqrt{x_{m}},\ldots ,\sqrt{x_{1}},(\sqrt{u},\sqrt{v},\sqrt{w} )^{\wedge }\) be a backward m-step extension of a recursive weight sequence and let \( W_{\alpha }\) be the weighted shift associated with \(\alpha \). In this paper we characterize the semi-cubic hyponormality of \(W_{\alpha }\) having the positive determinant coefficient property and discuss some related examples.

Closed-Form Solution of TDOA-Based Geolocation and Tracking: A Recursive Weighted Least Square Approach

The location determination technique using time difference of arrival (TDOA) measurements has been widely used in the military and observation industry. The accuracy of geolocation estimation is a very significant problem because measurement data are affected by environmental noise. Environmental noise occurs due to measurement error and the non-line of sight (NLOS) problem. This paper presents a Kalman filter-based NLOS section identification method and an iterative estimation of emitter location using the recursive weighted least square (RWLS) algorithm. Using fixed receivers with known locations, we obtain TDOA data that contain environmental noise. We identify the NLOS section of each receiver using a Kalman filter. Using the identified line of sight (LOS) TDOA measurements, we accurately derive the estimated location of an emitter with a fast calculation speed using the proposed RWLS algorithm when we receive additional TDOA data. In order to confirm the performance of the RWLS algorithm, the presented simulation results show that the proposed technique achieves improved accuracy and speed for estimating the emitter location.

Real-time multi-signal frequency tracking with a bank of notch filters to estimate the respiratory rate from the ECG

Measuring the instantaneous frequency of a signal rapidly and accurately is essential in many applications. However, the instantaneous frequency by definition is a parameter difficult to determine. Fourier-based methods introduce estimation delays as computations are performed in a time-window. Instantaneous methods based on the Hilbert transform lack robustness. State-of-the-art adaptive filters yield accurate estimates, however, with an adaptation delay. In this study we propose an algorithm based on short length-3 FIR notch filters to estimate the instantaneous frequency of a signal at each sample, in a real-time manner and with very low delay. The output powers of a bank of the above-mentioned filters are used in a recursive weighting scheme to estimate the dominant frequency of the input. This scheme has been extended to process multiple inputs containing a common frequency by introducing an additional weighting scheme upon the inputs. The algorithm was tested on synthetic data and then evaluated on real biomedical data, i.e. the estimation of the respiratory rate from the electrocardiogram. It was shown that the proposed method provided more accurate estimates with less delay than those of state-of-the-art methods. By virtue of its simplicity and good performance, the proposed method is a worthy candidate to be used in biomedical applications, for example in health monitoring developments based on portable and automatic devices.

On-line Evolving Cloud-based Model Identification for Production Control

In this paper we present an on-line evolving fuzzy cloud-based identification method. The evolving part of the algorithm is improved with new mechanisms. In the part of adding clouds (fuzzy rules) a new condition is implemented in addition to existing ones. Moreover, completely new mechanism for removing the “less active” and “less informative” clouds is introduced. All these mechanisms prevent adding new clouds based on outliers or at least help deleting existing ones with little information. The cloud-based method uses vectorized non-parametric antecedent (IF) part which relies on the local density of the current data with the existing clouds. The parameters of the consequent (THEN) part (functional in this case) were identified using recursive Weight Least Square method.The comparison between the original and the improved algorithm was provided on simulated data input/output signals acquired from Tennessee Eastman (TE) benchmark process. Firstly, most representative production Performance Indicators (pPIs) were chosen, then for each pPI a model was identified. The provided results (quality measures) of the proposed method were evaluated using on-line and off-line 4-step prediction. These were further compared with the results obtained using eFuMo identification tool.

A Decentralized Positioning Scheme Based on Recursive Weighted Least Squares Optimization for Wireless Sensor Networks

This paper presents a new positioning scheme based on recursive weighted least squares (RWLS) optimization for wireless sensor networks. The proposed scheme is derived from minimizing a recursive-in-time cost function and then realized in an iterative decentralized manner. The target location is computed iteratively by taking a weighted average of the local estimates according to the reliability information, where a sensor computes a new estimate based on its own observation and the previous sensor's estimate. Computer simulation results demonstrate that the proposed scheme has better positioning accuracy than previous related methods.

방대한 기상 레이더 데이터의 원할한 처리를 위한 순환 가중최소자승법 기반 RBF 뉴럴 네트워크 설계 및 응용

방대한 기상 레이더 데이터의 원할한 처리를 위한 순환 가중최소자승법 기반 RBF 뉴럴 네트워크 설계 및 응용

Recency, consistent learning, and Nash equilibrium.

We examine the long-term implication of two models of learning with recency bias: recursive weights and limited memory. We show that both models generate similar beliefs and that both have a weighted universal consistency property. Using the limited-memory model we produce learning procedures that both are weighted universally consistent and converge with probability one to strict Nash equilibrium.

A Decentralized Positioning Method Based on Recursive Weighted Least Absolute Value Optimization for Wireless Sensor Networks

This letter presents a new positioning method based on recursive weighted least absolute value (RWLAV) optimization for wireless sensor networks. The proposed method is derived by minimizing a recursive-in-time cost function and can be realized in a decentralized and iterative manner. During each iteration, the target's location is computed by taking a weighted average (in terms of the measurement's reliability) of the previous location estimate and the local observation that is expressed as the difference between the previous location estimate and the current subgradient value. Computer simulation results show that the RWLAV method performs better than previous related methods.