Sparse Filter Research Articles

Advanced Sparse Filtering-Based Domain Adaptation for Fault Diagnosis in Variable Working Conditions

Traditional domain adaptation (DA) methods often encounter challenges with cross-domain feature extraction and the precise assessment of domain differences. To overcome these limitations, we introduce the Enhanced Sparse Filtering-Based Domain Adaptation (ESFBDA) method. This method distinguishes itself by enhancing sparse filtering (SF) with the integration of row-column normalization and a cosine penalty, specifically designed to minimize feature loss—a critical issue in existing DA techniques. Additionally, we employ Bootstrap resampling to refine domain distribution alignment, a novel step that boosts feature similarity and effectiveness in DA. This integrated approach ensures more accurate feature extraction, which is crucial for the classifier's fault detection capability. In our study, through two distinct experiments on WT-planetary gearbox fault diagnosis and bearing fault diagnosis, the ESFBDA method demonstrated remarkable accuracy, significantly surpassing traditional approaches and showcasing its robust applicability across varied diagnostic scenarios.

A novel design algorithm for low complexity sparse 2‐D FIR filters

SummaryThe hardware implementation complexity of traditional two‐dimensional (2‐D) finite impulse response (FIR) filters with high order is very high. In this paper, a novel sparse 2‐D FIR filter design algorithm has been proposed by combining the common subexpression elimination technique and sparse filter design algorithms to reduce hardware implementation complexity. First, an initial 2‐D FIR filter with sparse coefficients that meets frequency‐domain specifications is obtained by the sparse filter design algorithm. Then, each filter coefficient is quantized in canonical signed digit, and the sensitivities of all weight‐two subexpressions and isolated non‐zero bits are calculated. Finally, the filter coefficients with low implementation cost are reconstructed via iteratively choosing subexpressions and isolated non‐zero bits in terms of their sensitivities. Simulation results show that the proposed algorithm saves about of adders compared with other low complexity filter design method, which reduces effectively the implementation cost.

Advanced Sparse Filtering-Based Domain Adaptation for Fault Diagnosis in Variable Working Conditions

Traditional domain adaptation (DA) methods often encounter challenges with cross-domain feature extraction and the precise assessment of domain differences. To overcome these limitations, we introduce the Enhanced Sparse Filtering-Based Domain Adaptation (ESFBDA) method. This method distinguishes itself by enhancing sparse filtering (SF) with the integration of row-column normalization and a cosine penalty, specifically designed to minimize feature loss—a critical issue in existing DA techniques. Additionally, we employ Bootstrap resampling to refine domain distribution alignment, a novel step that boosts feature similarity and effectiveness in DA. This integrated approach ensures more accurate feature extraction, which is crucial for the classifier's fault detection capability. In our study, through two distinct experiments on Electro-Hydrostatic Actuator (EHA) internal leakage and bearing fault diagnosis, the ESFBDA method demonstrated remarkable accuracy, significantly surpassing traditional approaches and showcasing its robust applicability across varied diagnostic scenarios.

Towards multi-scene learning: A novel cross-domain adaptation model based on sparse filter for traction motor bearing fault diagnosis in high-speed EMU

Fault diagnosis of traction motor bearing is of great significance to improve the reliability and safety of high-speed electric multiple units (EMU). While the fault diagnosis method based on cross-domain adaptation has been successful in scenarios involving speed or load fluctuations, existing methods ignore the independence and diversity of features, resulting in unsatisfactory diagnostic results under multi-scene learning, thereby reducing the generalization ability. Moreover, the development of complex models is time-consuming, and their computational efficiency is low. To address these issues, this study proposes a novel cross-domain adaptation model based on sparse filtering (SFCDA), which consists of only two fully connected (FC) layers. Firstly, pre-training is conducted to utilize the soft reconstruction penalties to constrain the weights of sparse filtering and improve the independence of features. The weights of unsupervised training are used to initialize the parameters of the first FC layer of the SFCDA model. Secondly, a multiple sparse regularization (MSR) algorithm is proposed and used to constrain the SFCDA. Then, fine-tuning is conducted, in which the structural alignment function is used to measure the distribution distance between the source and target domain data. Minimizing the kernel norm can improve the diversity of features and enhance the robustness. Finally, the effectiveness of SFCDA in multi-scene learning is proved theoretically. It is validated in three fault diagnosis scenes in four different bearing datasets. The results show that the suggested approach is more straightforward and has a better fault diagnosis effect than the state-of-the-art domain adaptive approaches.

A sparse adaptive decision feedback equalization approach for the high data rate vertical underwater acoustic communication in deep sea

In underwater acoustic communication (UWAC) environments, multichannel decision feedback equalizer (MDFE) techniques combined with single carrier modulation schemes are widely employed to suppress multipath interference while adapting to time-varying channel response. While MDFE is a powerful technique, the estimation of the numerous filter taps is generally complex problem, and the numerically complexities can lead to the instability of solutions including the occurrence of the burst errors. In this study, we propose a method that introduces L1-norm regularization into the adaptive algorithm of MDFE, considering a sparse channel response environment in the vertical UWAC channel in deep-sea areas. We applied the proposed method to simulation and experimental data, comparing it with conventional methods. As a result, the proposed method demonstrated a clear improvement in bit error rate performance, obtaining sparse filter coefficient estimates and effectively suppressing burst errors.

Rolling bearing fault diagnosis based on the fusion of sparse filtering and discriminative domain adaptation method under multi-channel data-driven

Currently, the diagnostic performance of many deep learning algorithms may drop dramatically when the distribution of training data is significantly different from that of the test data. Moreover, the fault diagnosis approaches based on single-channel data may suffer problems such as large precision fluctuation, low reliability, and incomplete expression of fault features. To overcome the above deficiencies, a novel multi-channel data-driven fault recognition method based on the fusion of sparse filtering (SF) and discriminative domain adaptation (MSFDDA) is proposed in this article. Firstly, inspired by attention mechanisms and information fusion methods, a spectrum-based weighted multi-channel data fusion strategy is designed to fully utilize the data collected by sensors to obtain a more comprehensive representation of fault features. Then, the joint probability-based discriminative maximum mean discrepancy algorithm is introduced into the SF method to strengthen the capability of extracting the domain invariant features. Finally, two bearing datasets are employed to verify the validity of the MSFDDA method, which proved to be superior to other current domain adaptation methods.

A Novel Inverse Algorithm To Solve the Integrated Optimization of Dose, Dose Rate, and Linear Energy Transfer of Proton FLASH Therapy With Sparse Filters

Purpose:The recently proposed Integrated Physical Optimization Intensity Modulated Proton Therapy (IPO-IMPT) framework allows simultaneous optimization of dose, dose rate, and linear energy transfer (LET) for FLASH treatment planning. Finding solutions to IPO-IMPT is difficult due to computational intensiveness. Nevertheless, an inverse solution that simultaneously specifies the geometry of a sparse filter and weights of a proton intensity map is desirable for both clinical and preclinical applications. Such solutions can reduce effective biological dose to organs at risk in cancer patients as well as reduce the number of animal irradiations needed to derive extra biological dose models in preclinical studies. Methods:Unlike the initial forward heuristic, this inverse IPO-IMPT solution includes simultaneous optimization of sparse range compensation, sparse range modulation, and spot intensity. The daunting computational tasks vital to this endeavor were resolved iteratively with a distributed computing framework to enable Simultaneous Intensity and Energy Modulation and Compensation (SIEMAC). SIEMAC was demonstrated on a human central lung cancer patient and a minipig. Results:SIEMAC simultaneously improves maps of spot intensities and patient-field-specific sparse range compensators and range modulators. For the lung cancer patient, at our maximum nozzle current of 300 nA, dose rate coverage above 100 Gy/s increased from 57% to 96% in the lung and from 93% to 100% in the heart, and LET coverage above 4 keV/μm dropped from 68% to 9% in the lung and from 26% to <1% in the heart. For a simple minipig plan, the full-width-half-maximum of the dose, dose rate, and LET distributions decreased by 30%, 1.6%, and 57%, respectively, again with similar target dose coverage, thus reducing uncertainty in these quantities for preclinical studies. Conclusion:The inverse solution to IPO-IMPT demonstrated the capability to simultaneously modulate sub-spot proton energy and intensity distributions for clinical and preclinical studies.

Multi-domain feature joint optimization based on multi-view learning for improving the EEG decoding.

Brain-computer interface (BCI) systems based on motor imagery (MI) have been widely used in neurorehabilitation. Feature extraction applied by the common spatial pattern (CSP) is very popular in MI classification. The effectiveness of CSP is highly affected by the frequency band and time window of electroencephalogram (EEG) segments and channels selected. In this study, the multi-domain feature joint optimization (MDFJO) based on the multi-view learning method is proposed, which aims to select the discriminative features enhancing the classification performance. The channel patterns are divided using the Fisher discriminant criterion (FDC). Furthermore, the raw EEG is intercepted for multiple sub-bands and time interval signals. The high-dimensional features are constructed by extracting features from CSP on each EEG segment. Specifically, the multi-view learning method is used to select the optimal features, and the proposed feature sparsification strategy on the time level is proposed to further refine the optimal features. Two public EEG datasets are employed to validate the proposed MDFJO method. The average classification accuracy of the MDFJO in Data 1 and Data 2 is 88.29 and 87.21%, respectively. The classification result of MDFJO was significantly better than MSO (p < 0.05), FBCSP32 (p < 0.01), and other competing methods (p < 0.001). Compared with the CSP, sparse filter band common spatial pattern (SFBCSP), and filter bank common spatial pattern (FBCSP) methods with channel numbers 16, 32 and all channels as well as MSO, the MDFJO significantly improves the test accuracy. The feature sparsification strategy proposed in this article can effectively enhance classification accuracy. The proposed method could improve the practicability and effectiveness of the BCI system.

Manifold learning-assisted sparse filtering method for machinery fault diagnosis

Sparse filtering (SF) has received considerable attentions in the machinery fault diagnosis thanks to its ability to extract the fault-related features using their sparsity. However, the existing SF methods have dilemmas with the empirical selection of model parameters, the loss of fault-related information caused by a screening way for the target mode, and the singularity of results induced by some large-amplitude random impulses (LARIs). Hence, a manifold learning-assisted SF method is proposed for machinery fault-related feature enhancement in this study. First, an improved intrinsic component filtering (ICF) is presented for extracting the multiple modes with feature enhancement, where the parameters of ICF are adaptively determined by using the optimization object to avoid the empirical selection of parameters. Second, the manifold learning is introduced to compress the enhanced multiple modes to overcome the loss of fault-related information; thus the intrinsic manifolds are obtained for disclosing the buried fault-related features and suppressing the band-in noise. Third, an adaptively weighting strategy for these intrinsic manifolds is constructed to obtain a final representative mode for conducting the machinery fault diagnosis. Meanwhile, the LARIs coupled with intrinsic manifolds are calibrated according to their statistical information to resolve the singularity of the representative features. Simulation and experiments show that the proposed method is more effective in extracting fault-related features than some existing methods.

A Tiny Accelerator for Mixed-Bit Sparse CNN Based on Efficient Fetch Method of SIMO SPad

Convolution neural networks (CNNs) have been implemented with custom hardware on edge devices since its algorithms were successful in many artificial intelligence applications. Although lots of unstructured pruning and mix-bit quantization algorithms have been proposed to successfully compress CNNs, there are few hardware accelerators which can support both sparse and mix-bit CNNs. Besides, sparse matrix computation consumes lots of hardware resources such as registers or BRAM to fetch the needed input activations into processing element (PE). This paper presents a tiny accelerator for mixed-bit sparse CNNs featuring a novel scheme of single vector-based compressed sparse filter (CSF) method and single input multiple output scratch pad (SIMO SPad) to effectively compress weight and fetch the needed input activation. SIMO SPad is shared by multiple PEs, which saves 13.34% CLB LUTs, 46.24% CLB Registers. Furthermore, the accelerator supports mixed-bit sparse computation to obtain better accuracy and performance. When tested on VGG16, compared with 8-bit non-sparsity baselines, the performance of mixed-bit sparsity on Cifar10 and ImageNet improved by 4.85× and 3.33×, respectively, with small accuracy decrease degradation. Compared to state-of-the-art accelerators, the accelerator achieves 1.40× to 2.98× greater DSP efficiency, and offers 1.91× greater energy efficiency.

Lightweight and intelligent model based on enhanced sparse filtering for rotating machine fault diagnosis

Rotating machine fault diagnosis plays a vital role in reducing maintenance costs and preventing accidents. Machine learning (ML) methods and Internet of things (IoT) technologies have been recently introduced into machine fault diagnosis and have generated inspiring results. An ML model with more trainable parameters can typically generate a higher fault diagnostic accuracy. However, the IoT nodes have limited computation and storage resources. How to design an ML model with high accuracy and computational efficiency is still a difficulty and challenge. This work develops an enhanced sparse filtering (ESF) method for mining and fusing the features of the machine signals for fault diagnosis. First, a dimension reduction algorithm is utilized for obtaining the principal components of the vibration signals that are hindered by noises. The distinct features of the principal components are then exploited by using sparse filtering (SF). To reduce the overfitting of the SF model, the L1/2 norm is applied to regularize the objective function. Finally, the obtained features are combined as the inputs of a softmax classifier for machine fault pattern recognition. The effectiveness, superiority, and robustness of the proposed ESF method are validated by the simulated signals and the practical bearing and motor fault signals compared with the other conventional methods. The lightweight and intelligent ESF algorithm is also deployed onto an edge computing node to realize online motor fault diagnosis. The designed model and the proposed method show great potential in highly accurate and efficient rotation machine fault diagnosis.

Convolutional sparse filter with data and mechanism fusion: A few-shot fault diagnosis method for power transformer

In actual industrial scenarios, fault data is rare and fault labels are difficult to obtain, which brings many obstacles for fault diagnosis. For this situation, this research proposes a novel semi-supervised convolutional sparse filter with local mechanism similarity regularization (CSF-LMSR) to construct a more reliable few-shot diagnosis method for power transformer. First, a sparse filter with local mechanism similarity regularization term (SF-LMSR) is designed as a more interpretable unsupervised feature extractor with prior knowledge. This unsupervised process enables the model to extract satisfactory features from the whole dataset even with a lower proportion of labeled data. Second, SF-LMSR is combined with convolutional neural network (CNN) by a novel coupling mode of kernel replacement, which enhances the learning ability of CNN. This classification model still adopts supervised learning training, but the demand for labeled fault data is greatly reduced, which reduces the burden of labels. The effectiveness of the proposed method is verified using real power transformer dissolved gas analysis (DGA) datasets. It can be seen from experimental results that the proposed method does provide a new perspective for transformer fault diagnosis, and it is a successful attempt for the power industry few-shot diagnosis problem.

Discriminative feature learning and selection with label-induced sparse filtering for intelligent fault diagnosis of rotating machinery

Representation learning has demonstrated its powerful potential in intelligent fault diagnosis of rotating machinery, where sparse filtering (SF) is a popular and promising representation learning method. However, the currently SF-based approaches still have two limitations: (1) discriminant ability of the learned features is insufficient when handling complex signals collected from variable operating conditions and (2) they are incapable of selecting the most relevant and important features for fault classification. To address these issues, label-induced sparse filtering (LISF) is proposed by introducing a fully-connected label layer. On the one hand, discriminant information encoded in labels is exploited in LISF’s training, so discriminative ability of the learned features can be significantly enhanced. On the other hand, the importance of each feature is measured by a projection matrix, so the top-ranked relevant features can be filtered accordingly. Our method is evaluated by experiments on two case studies. It shows that LISF is able to learn discriminative features from complex signals, and achieves excellent diagnosis results over the existing advanced RL methods. Moreover, LISF can automatically select a set of a specified number of key features for fault classification, so as to improve diagnosis efficiency greatly.

Deep network-based maximum correlated kurtosis deconvolution: A novel deep deconvolution for bearing fault diagnosis

Deconvolution methods (DMs) which can adaptively design the filter for the feature extraction is the most effective tool to counteract the effect of the transmission path. Convolutional sparse filter (CSF) as a new deconvolution mode, which transfers the complicated numeric calculation to the simple feature learning for the optimization and solution of the deconvolution filter coefficient using neural network, has a remarkable superiority especially under the heavy noise condition compared with the traditional DMs. Yet, the problems of the filter length selection and the sensibility to random interference largely confine its application. Motived by this, a novel deep network-based maximum correlated kurtosis deconvolution (MCKD-DeNet) is proposed in this paper. Firstly, according to the multiple-inputs way of the neural network, a filter initialization is designed using the Hanning window. With different filters guided by the initialization, a serial of filtered signals is input to learn the fault feature. Secondly, correlated kurtosis, which can simultaneously evaluate the periodicity and impulsiveness of the signal, is chosen as the new cost function to train the neural network. And the input period is estimated between the layers by calculating the autocorrelation of the most informative filtered signal. Subsequently, the component with most fault information is locked as the output of MCKD-DeNet using the proposed dimension reduction method based on the correlation coefficient. Finally, compared with previous CSF and improved maximum correlated kurtosis deconvolution, the proposed MCKD-DeNet is verified to have the performance superiority by simulated signal with different noise levels and interference as well as experimental data collected from wind turbine experiment bench with bearing fault.

Unrolled Compressed Blind-Deconvolution

The problem of sparse multichannel blind deconvolution (S-MBD) arises frequently in many engineering applications such as radar/sonar/ultrasound imaging. To reduce its computational and implementation cost, we propose a compression method that enables blind recovery from much fewer measurements with respect to the full received signal in time. The proposed compression measures the signal through a filter followed by a subsampling, allowing for a significant reduction in implementation cost. We derive theoretical guarantees for the identifiability and recovery of a sparse filter from compressed measurements. Our results allow for the design of a wide class of compression filters. We, then, propose a data-driven unrolled learning framework to learn the compression filter and solve the S-MBD problem. The encoder is a recurrent inference network that maps compressed measurements into an estimate of sparse filters. We demonstrate that our unrolled learning method is more robust to choices of source shapes and has better recovery performance compared to optimization-based methods. Finally, in data-limited applications (fewshot learning), we highlight the superior generalization capability of unrolled learning compared to conventional deep learning.

A Sparse CNN Accelerator for Eliminating Redundant Computations in Intra- and Inter-Convolutional/Pooling Layers

Neural network pruning, which can be divided into unstructured pruning and structured pruning strategies, has been proven to be an efficient method to substantially reduce the number of computations of convolutional neural networks (CNNs). However, it remains difficult to combine the advantages of these two pruning strategies. This article proposes a high-performance accelerator for unstructured sparse CNNs. First, a convolution-based filter selection and clustering method (FSCM) is proposed to reorder unstructured sparse filters into uniform-size dense filters, eliminating redundant computations in the convolutional layers while maintaining a regular structure. In addition, a convolution and pooling calculation method (CPCM) is presented to reduce interlayers’ computational redundancy. Third, a hardware accelerator with high digital signal processing (DSP) efficiency is designed to take advantage of FSCM and CPCM. The proposed accelerator is implemented on a Xilinx XCVU9P platform at 300 MHz. With different CNN configurations, the performance and DSP efficiency for LeNet, AlexNet, and VGG16 are 1490.53 GOPS and 3.882, 913.85 GOPS and 1.785, and 862.16 GOPS and 1.68, respectively. Compared to previous field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) designs, the accelerator achieves up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$15.43\times $ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$7.73\times $ </tex-math></inline-formula> speedup in terms of performance and DSP efficiency.

Sparse Filtering Under Norm-Bounded Exogenous Disturbances Using Observers

The paper considers the sparse filtering problem under arbitrary norm-bounded exogenous disturbances. We propose a simple and universal observer-based approach to its solution, based on the LMI technique and the method of invariant ellipsoids; it allows the use of a reduced number of system outputs. From a technical point of view of application, we reduce the original problem to semi-definite programming, which is easily solved numerically. The proposed simple approach is easy to implement and can be equally extended to systems in continuous and discrete time.

A Two-Step Unsupervised Learning Approach to Diagnose Machine Fault Using Big Data

The modern industrial sector requires an intelligent fault diagnosis system to ensure reliable and safe processingsince traditional methods require expert diagnosis, which consumes time and requires labor. Furthermore, diagnosticresults are influenced by the expert’s expertise and in-depth knowledge of the machine. The objective of this paper isto solve the manual intervention problem and improve the fault diagnosis. We propose a novel two-stage unsupervisedlearning algorithm based on artificial intelligence (AI) that learns fault features efficiently from raw vibration signals.To accomplish the aforementioned goal, we encapsulate the two-stage learning technique such as sparse filtering andRectified Linear Unit (ReLU) regression function. As a first step, we used a two-layer neural network sparse filteringprocedure to extract vibration signals’ features. Based on vibration signals, ReLU regression determines the healthcondition of the machine in the second phase. ReLU is a linear function that improves the performance of neural network training. Here we utilized a sigmoid and softmax regression function to compare the performance of ReLU. Thesigmoid function works well for binary classification, whereas softmax works well for multiclass classification. A database of motor-bearing vibration signals containing signals about four different health conditions of machines, suchas Inner race faults (IF), Outer race faults (OF), and rolling faults (RF). The sparse filter is evaluated on different inputand output dimensions, which significantly increases the learning accuracy. We classified the health condition usingReLU and achieved 93.8% accuracy, which is higher than sigmoid and softmax. Through the two-step learning process, machine fault diagnosis is enhanced, as well as big data is effectively handled.

Generalized Gini indices: Complementary sparsity measures to Box-Cox sparsity measures for machine condition monitoring

Sparsity measures that can quantify the sparsity of signals are often used as objective functions of signal processing and machine learning algorithms (e.g., sparse filtering, compressive sensing, blind deconvolution, and the fast Kurtogram, etc.). Classic sparsity measures include kurtosis, Gini index, negative entropy, the ratio of Lp norm to Lq norm, smoothness index, L1 norm, L0 norm, etc. To enrich the library of sparsity measures, this paper aims to generalize a classic Gini index to construct generalized Gini indices (GGIs). Firstly, inspired by the ratio of different quasi-arithmetic means, investigations on the GGIs showed that the GGIs satisfy all six properties of good sparsity measures. The GGIs only have one parameter a⩾0. When a=1, the GGIs are reduced to the classic Gini index. Secondly, investigations reveal that: (1) the GGIs can monotonically quantify the sparsity changes of Bernoulli coefficients. Sparsity quantification curves given by the GGIs are complementary to those generated by our recently proposed Box-Cox sparsity measures (BCSMs). Especially, when the unique parameters of the GGIs and the BCSMs are equal to zero, these sparsity quantification curves of the GGIs and BCSMs are reduced to that of negative entropy; (2) the GGIs are convergent with an increase of signal lengths; (3) when the squared envelope of white Gaussian noise is quantified by the GGIs, theoretical values of the GGIs are derived as a/(a+1), which can be used as baselines for machine abnormality detection. At last, two bearing run-to-failure datasets are used to validate the effectiveness of the GGIs for machine condition monitoring and their results showed that the GGIs are effective in detecting incipient faults and indicating informative frequency bands. In future, the proposed GGIs can be used in any algorithms that need sparsity measures.

An efficient sparse pruning method for human pose estimation

Human pose estimation (HPE) is crucial for computer vision (CV). Moreover, it’s a vital step for computers to understand human actions and behaviours. However, the huge number of parameters and calculations in the HPE model have brought big challenges to deploy to resource-constrained mobile devices. Aiming to overcome the challenge, we propose a sparse pruning method (SPM) for the HPE model. First, L1 regularisation is added in the training phase of the original model, and network parameters of the convolution layers (CLs) and batch normalisation layers (BNLs) are sparsely trained to obtain a network structure with sparse weights. We then combine the sparse weights of filters with the scaling parameters of the BNLs to determine their importance. Finally, the structured pruning method is used to prune the sparse filters and corresponding channels. SPM can reduce the number of model parameters and calculations without affecting precision. Promising results indicate that SPM outperforms other advanced pruning methods.