Sparse Filter Research Articles

A depth video coding in-loop median filter based on joint weighted sparse representation

The existing depth video coding algorithms are generally based on in-loop depth filters, whose performance are unstable and easily affected by the outliers. In this paper, we design a joint weighted sparse representation-based median filter as the in-loop filter in depth video codec. It constructs depth candidate set which contains relevant neighboring depth pixel based on depth and intensity similarity weighted sparse coding, then the median operation is performed on this set to select a neighboring depth pixel as the result of the filtering. The experimental results indicate that the depth bitrate is reduced by about 9% compared with anchor method. It is confirmed that the proposed method is more effective in reducing the required depth bitrates for a given synthesis quality level.

An Intelligent Fault Diagnosis Method Using Unsupervised Feature Learning Towards Mechanical Big Data

Intelligent fault diagnosis is a promising tool to deal with mechanical big data due to its ability in rapidly and efficiently processing collected signals and providing accurate diagnosis results. In traditional intelligent diagnosis methods, however, the features are manually extracted depending on prior knowledge and diagnostic expertise. Such processes take advantage of human ingenuity but are time-consuming and labor-intensive. Inspired by the idea of unsupervised feature learning that uses artificial intelligence techniques to learn features from raw data, a two-stage learning method is proposed for intelligent diagnosis of machines. In the first learning stage of the method, sparse filtering, an unsupervised two-layer neural network, is used to directly learn features from mechanical vibration signals. In the second stage, softmax regression is employed to classify the health conditions based on the learned features. The proposed method is validated by a motor bearing dataset and a locomotive bearing dataset, respectively. The results show that the proposed method obtains fairly high diagnosis accuracies and is superior to the existing methods for the motor bearing dataset. Because of learning features adaptively, the proposed method reduces the need of human labor and makes intelligent fault diagnosis handle big data more easily.

Low‐complexity large‐scale multiple‐input multiple‐output channel estimation using affine combination of sparse least mean square filters

Large-scale multiple-input multiple-output (MIMO) system is considered one of promising technologies to realise next-generation wireless communication system (5G). So far, channel estimation problem is a big obstacle to develop large-scale MIMO system design due to high computational complexity and curse of dimensionality, which are caused by the long delay spread as well as a large number of antennas. Hence, devising any low-complexity channel estimation method could promote the successful development of the large-scale MIMO system. Due to the fact that, large-scale MIMO channels often exhibit sparse or/and cluster-sparse structure, in this study, the authors propose an effective low-complexity large-scale MIMO channel estimation method by using affine combination of sparse adaptive filtering filters. First, problem formulation and standard affine combination of adaptive least mean square (LMS) filters are introduced. Then they propose an effective affine combination method with two sparse LMS filters and design an approximate optimum affine combiner according to stochastic gradient search method. Later, to verify the proposed algorithm for large-scale MIMO channel estimation, both theoretical analysis and numerical simulations are provided to confirm effectiveness of the proposed algorithm which can achieve better estimation performance than the traditional methods.

Learning Co-Sparse Analysis Operators With Separable Structures

In the co-sparse analysis model a set of filters is applied to a signal out of the signal class of interest yielding sparse filter responses. As such, it may serve as a prior in inverse problems, or for structural analysis of signals that are known to belong to the signal class. The more the model is adapted to the class, the more reliable it is for these purposes. The task of learning such operators for a given class is therefore a crucial problem. In many applications, it is also required that the filter responses are obtained in a timely manner, which can be achieved by filters with a separable structure. Not only can operators of this sort be efficiently used for computing the filter responses, but they also have the advantage that less training samples are required to obtain a reliable estimate of the operator. The first contribution of this work is to give theoretical evidence for this claim by providing an upper bound for the sample complexity of the learning process. The second is a stochastic gradient descent (SGD) method designed to learn an analysis operator with separable structures, which includes a novel and efficient step size selection rule. Numerical experiments are provided that link the sample complexity to the convergence speed of the SGD algorithm.

Sparse Banded Matrix Filter for Image Denoising

Noise is one of the prime factors which degrade the quality of an image. Hence, image denoising is an essential image enhancement technique in the image processing domain. In this paper, we use low-pass sparse banded filter matrices for image denoising. Sparsity is the key concept in this filter design. We applied the designed low-pass filter both row-wise and column-wise to denoise the image. The proposed method is experimented on standard test images corrupted with different types of noises namely Gaussian, White Gaussian, Salt & Pepper and Speckle with noise level equals to 0.01, 0.05 and 0.1. The effectiveness of the proposed method of denoising is evaluated by the computation of standard quality metric known as Peak Signal-to-Noise Ratio (PSNR). The experimental result analysis shows that the proposed image denoising technique based on sparse banded filter matrices results in significant improvement in PSNR around 2dB to 8dB for different type of noises with noise level equal to 0.1 and is also aided by the visual analysis.

Optimizing spatial patterns with sparse filter bands for motor-imagery based brain–computer interface

BackgroundCommon spatial pattern (CSP) has been most popularly applied to motor-imagery (MI) feature extraction for classification in brain–computer interface (BCI) application. Successful application of CSP depends on the filter band selection to a large degree. However, the most proper band is typically subject-specific and can hardly be determined manually. New methodThis study proposes a sparse filter band common spatial pattern (SFBCSP) for optimizing the spatial patterns. SFBCSP estimates CSP features on multiple signals that are filtered from raw EEG data at a set of overlapping bands. The filter bands that result in significant CSP features are then selected in a supervised way by exploiting sparse regression. A support vector machine (SVM) is implemented on the selected features for MI classification. ResultsTwo public EEG datasets (BCI Competition III dataset IVa and BCI Competition IV IIb) are used to validate the proposed SFBCSP method. Experimental results demonstrate that SFBCSP help improve the classification performance of MI. Comparison with existing methodsThe optimized spatial patterns by SFBCSP give overall better MI classification accuracy in comparison with several competing methods. ConclusionsThe proposed SFBCSP is a potential method for improving the performance of MI-based BCI.

Deep learning human actions from video via sparse filtering and locally competitive algorithms

Physiological and psychophysical evidence suggest that early visual cortex compresses the visual input on the basis of spatial and orientation-tuned filters. Recent computational advances have suggested that these neural response characteristics may reflect a `sparse coding' architecture--in which a small number of neurons need to be active for any given image--yielding critical structure latent in natural scenes. Here we present a novel neural network architecture combining a sparse filter model and locally competitive algorithms (LCAs), and demonstrate the network's ability to classify human actions from video. Sparse filtering is an unsupervised feature learning algorithm designed to optimize the sparsity of the feature distribution directly without having the need to model the data distribution. LCAs are defined by a system of differential equations where the initial conditions define an optimization problem and the dynamics converge to a sparse decomposition of the input vector. We applied this architecture to train a classifier on categories of motion in human action videos. Inputs to the network were small 3D patches taken from frame differences in the videos. Dictionaries were derived for each action class and then activation levels for each dictionary were assessed during reconstruction of a novel test patch. Overall, classification accuracy was at ? 97 %. We discuss how this sparse filtering approach provides a natural framework for multi-sensory and multimodal data processing including RGB video, RGBD video, hyper-spectral video, and stereo audio/video streams.

Minimum Error Entropy Algorithms with Sparsity Penalty Constraints

Recently, sparse adaptive learning algorithms have been developed to exploit system sparsity as well as to mitigate various noise disturbances in many applications. In particular, in sparse channel estimation, the parameter vector with sparsity characteristic can be well estimated from noisy measurements through a sparse adaptive filter. In previous studies, most works use the mean square error (MSE) based cost to develop sparse filters, which is rational under the assumption of Gaussian distributions. However, Gaussian assumption does not always hold in real-world environments. To address this issue, we incorporate in this work an l1-norm or a reweighted l1-norm into the minimum error entropy (MEE) criterion to develop new sparse adaptive filters, which may perform much better than the MSE based methods, especially in heavy-tailed non-Gaussian situations, since the error entropy can capture higher-order statistics of the errors. In addition, a new approximator of l0-norm, based on the correntropy induced metric (CIM), is also used as a sparsity penalty term (SPT). We analyze the mean square convergence of the proposed new sparse adaptive filters. An energy conservation relation is derived and a sufficient condition is obtained, which ensures the mean square convergence. Simulation results confirm the superior performance of the new algorithms.

Compute Less to Get More: Using ORC to Improve Sparse Filtering

Sparse Filtering is a popular feature learning algorithm for image classification pipelines. In this paper, we connect the performance of Sparse Filtering with spectral properties of the corresponding feature matrices. This connection provides new insights into Sparse Filtering; in particular, it suggests early stopping of Sparse Filtering. We therefore introduce the Optimal Roundness Criterion (ORC), a novel stopping criterion for Sparse Filtering. We show that this stopping criterion is related with pre-processing procedures such as Statistical Whitening and demonstrate that it can make image classification with Sparse Filtering considerably faster and more accurate.

Edge Detection Using Sparse Banded Filter Matrices

Abstract Edges are intensity change, which occur on the boundary between two regions in an image. Edges can be used as feature descriptors of an object. Hence, edge detection plays an important role in computer vision applications. This paper presents the application of sparse banded filter matrices in edge detection. The filter design is formulated in terms of banded matrices. The sparsity property of the designed filter leads to efficient computation. In our proposed method, we applied sparse banded high-pass filter row-wise and column-wise to extract the vertical and the horizontal edges of the image respectively. The proposed technique is experimented on standard images and the results are compared with the state-of-the-art methods. The visual comparison of the experimental results shows that the proposed approach for edge extraction based on sparse banded filter matrices produces result comparable to the existing methods. The advantage of the proposed approach is that the continuous edges are attained without any parameter tuning.

A Convex Optimization Approach to Synthesizing Sparse Dynamic Output Feedback Controllers

This paper considers the problem of synthesizing dynamic output feedback controllers subject to structural sparsity constraints. It is well known that unless the plant and sparsity pattern satisfy the so-called quadratic invariance property, this problem is generically NP-hard. The main result of this paper shows that in these cases, a computationally attractive convex relaxation can be obtained by recasting the problem into an estimation form and exploiting recent results on sparse filter design. As illustrated with an example, the proposed approach compares favorably with existing techniques, in terms of its ability to find a suitable controller and the resulting closed-loop performance.

Online Sequential Extreme Learning Machine With Kernels.

The extreme learning machine (ELM) was recently proposed as a unifying framework for different families of learning algorithms. The classical ELM model consists of a linear combination of a fixed number of nonlinear expansions of the input vector. Learning in ELM is hence equivalent to finding the optimal weights that minimize the error on a dataset. The update works in batch mode, either with explicit feature mappings or with implicit mappings defined by kernels. Although an online version has been proposed for the former, no work has been done up to this point for the latter, and whether an efficient learning algorithm for online kernel-based ELM exists remains an open problem. By explicating some connections between nonlinear adaptive filtering and ELM theory, in this brief, we present an algorithm for this task. In particular, we propose a straightforward extension of the well-known kernel recursive least-squares, belonging to the kernel adaptive filtering (KAF) family, to the ELM framework. We call the resulting algorithm the kernel online sequential ELM (KOS-ELM). Moreover, we consider two different criteria used in the KAF field to obtain sparse filters and extend them to our context. We show that KOS-ELM, with their integration, can result in a highly efficient algorithm, both in terms of obtained generalization error and training time. Empirical evaluations demonstrate interesting results on some benchmarking datasets.

Sparse-filtering in directional lifting wavelet transform domain based Bayesian compressive sensing

Compressive sensing (CS) has been proposed for images that are sparse under a certain transform domain. However, many natural images are not strictly sparse in the transform domain, causing a tail-folding effect that degrades the performance of the CS reconstruction. To decrease such effect, a sparse-filtering (SF) in Directional Lifting Wavelet Transform (DLWT) domain based Bayesian compressive sensing (BCS) algorithm (DLWT-SF-TSW-BCS) is proposed. At the encoder, DLWT, an efficient multi-scale geometrical analysis (MGA) tool, is used to produce the sparse representation for natural images. Then sparse-filtering is adopted to cut off the small DLWT coefficients before random measurement. At the decoder, the interscale tree-structure redundancy in DLWT domain is further exploited in Bayesian reconstruction. Experimental results show that the proposed DLWT-SF-TSW-BCS algorithm significantly outperforms other state-of-the-art CS reconstruction algorithms, for example, peak signal to noise ratio (PSNR) gain up to 10.00 dB over the tree structured wavelet compressive sensing (TSW-CS).

Bit-Level Multiplierless FIR Filter Optimization Incorporating Sparse Filter Technique

Multiplierless FIR filter optimization has been extensively studied in the past decades to minimize the number of adders. A more accurate measurement of the implementation complexity is the number of full adders counted at bit-level. However, the high computational complexity of the optimization at bit-level hinders the technique from practical applications. In this paper, the sparse filter technique is exploited and makes the search space at bit-level significantly reduced. Thus, the bit-level optimization of multiplierless FIR filters for the first time becomes possible. When the sparse filter technique is employed for the multiplierless filter design, the sparsity of the filter is properly controlled so that the feasibility of the bit-level optimization in discrete space is maintained. Thereafter, in the reduced search space, a tree search algorithm is formulated at bit-level, and techniques to estimate the bit level hardware cost and to accelerate the search are presented. Design examples show that the proposed bit-level optimization method generates designs with lower hardware cost and power consumption than that of the best word-level optimization methods, while the design time is still at an acceptable level. The average power savings to 3 recent published techniques are 13.6%, 8.0% and 26.1%, respectively.

Dynamic RLS-DCD for Sparse System Identification

In this passage we propose a computationally efficient adaptive filtering algorithm for sparse system identification.The algorithm is based on dichotomous coordinate descent iterations, reweighting iterations,iterative support detection.In order to reduce the complexity we try to discuss in the support.we suppose the support is partial,and partly erroneous.Then we can use the iterative support detection to solve the problem.Numerical examples show that the proposed method achieves an identification performance better than that of advanced sparse adaptive filters (l1-RLS,l0-RLS) and its performance is close to the oracle performance.

Learning Object Groups for Scene Recognition

Scene recognition is an important task for many computer vision and robotics applications. Recent progress in high-level object-based image representation has shown superior performance on scene classification tasks. In this work, we make an observation that groups of objects tend to co-occur frequently in a scene. We therefore propose to a novel framework that automatically learns object groups, and use them to build an image representation for scene recognition tasks. We model each object group as a template that explicitly encodes the spatial configurations of objects. To encourage the informativeness and discriminability, we learn the object group templates in a sparse filtering framework. Experiment results show that our object group representation could achieve state-of-the-art performance for both scene discovery and scene classification tasks.

Imaging of the SEG/EAGE Salt Model Seismic Data Using Sparse $f$– $x$ Finite-Impulse-Response Wavefield Extrapolation Filters

In this paper, a stable explicit depth wavefield extrapolation is obtained using sparse frequency-space (f-x) finite impulse response (FIR) digital filters. The ideal impulse response of the wavefield extrapolation filter is nonsparse in nature, as will be shown. The problem of designing such filters to obtain stable images of the famous challenging Society of Exploration Geophysicists/European Association of Geoscientists and Engineers (SEG/EAGE) salt model seismic data was formulated as an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm minimization that is convex with a quadratic constraint. Then, sparse filters were obtained by employing hard thresholding to the filter coefficients' magnitude. An ad hoc threshold value was set to the minimum of the bin location values that were used to calculate the histogram of the f-x FIR filter coefficients' magnitude. Thus, the f-x FIR filter coefficients (both real and imaginary parts), at which the filter coefficients' magnitude is below the threshold, were set equal to zero. Poststack depth imaging of the SEG/EAGE salt model data set was then performed using the explicit depth wavefield extrapolation with the proposed nonsparse and sparse L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm minimization-based algorithms. Taking into account the amount of improved computational efficiency and compared with the results of other methods such as the phase shift plus interpolation migration algorithm, the results of the proposed sparse L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm minimization method showed practically stable images of the salt data set. The proposed method can therefore be used to design high accuracy sparse extrapolation filters.

On FIR Filter Approximation of Fractional-Order Differentiators and Integrators

This paper considers finite-length impulse response (FIR) filter approximation of differentiators and integrators, collectively called differintegrators. The paper introduces and compares three different FIR filter structures for this purpose, all of which are optimized in the minimax sense using iterative reweighted l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm minimization. One of the structures is the direct-form structure, but featuring equal-valued taps and zero-valued taps, the latter corresponding to sparse filters. The other two structures comprise two subfilters in parallel and cascade, respectively. In their basic forms, nothing is gained by realizing the filters in parallel or in cascade, instead of directly. However, as the paper will show, these forms enable substantial further complexity reductions, because they comprise symmetric and antisymmetric subfilters of different orders, and also features additional equal-valued and zero-valued taps. The cascade structure employs a structurally sparse filter. The additional sparsity, as well as tap equalities, are for all three structures found automatically in the design via the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm minimization. Design examples included reveal feasible multiplication complexity savings of more than 50% in comparison with regular (unconstrained) direct-form structures. In addition, an example shows that the proposed designs can even have lower complexity than existing infinite-length impulse response filter designs.

Positive sparse coding of natural images: a theory for simple cell tuning

One of the most celebrated results in neuroscience is the observation that simple cells in the visual cortex are tuned to the orientation and polarity of edges in visual stimuli [1]. While orientation tuning has been the subject of intense investigation, the polarity tuning of cells is poorly understood; a simple cell responds either to a bright edge with dark flanks, or to the opposite polarity, a dark edge with bright flanks. We ask, what is the function of this polarity tuning, if any, and how does it arise from the underlying neural circuitry? Perhaps the most influential functional theory of simple cell tuning is sparse coding [2]. According to this theory, the function of simple cells is to represent natural images sparsely (using as few active neurons as possible). This representation is desirable because it is metabolically and computational efficient. It explains orientation tuning because sparse coding neurons are tuned to edge-like stimuli at different orientations. However, there are two problems with this 'classical' sparse coding (Table ​(Table1).1). First of all, it predicts that every cell is tuned to both polarities, whereas simple cells only respond to a single polarity. Secondly, it requires neurons to represent images using negative firing rate values, which is clearly impossible for a spiking neural network. Table 1 Tuning properties We solve both of these problems, simultaneously, by proposing that simple cells provide a positive sparse coding of natural images. This is similar to 'classical' sparse coding but with positive firing rates alone. We find that positive sparse coding neurons have both orientation tuning and polarity tuning (Figure ​(Figure1,1, Table ​Table1).1). Orientation tuning emerges because natural images typically contain edges at many different orientations, and a sparse code will capture these natural statistics. Polarity tuning emerges as a natural consequence of the positivity constraint because a neuron with positive firing rates cannot represent edges with both polarities on its own. Figure 1 Positive sparse coding filters of natural images. Each image is the sparse coding filter for a single neuron. These are tuned to orientation and polarity (left side neurons tuned to the opposite polarity of right side neurons). This functional theory leads naturally to a spiking neural network implementation for simple cells, because spiking network models necessarily have positive firing rates. We find that a network of tightly balanced leaky integrate-and-fire neurons can produce spike trains that optimise our positive sparse coding cost function. In this way, we can marry spiking constraints with neural function to understand the orientation and polarity tuning of simple cells.

SLAM algorithm based on sparse extended information filter and particle filter

In the traditional particle filter algorithm,there is a problem that historical information can not be fully utilized because of small weight particle removed in resample and single iteration.A Simultaneous Localization And Mapping(SLAM) algorithm based on sparse extended information filter combing particle filter was brought forward.Information matrix maintained historical information,particles containing historical information obtained by Gibbs sampling,particles set described the posterior distribution well,and the accuracy of state estimation SLAM algorithm was improved.Plentiful Monte-Carlo simulations were carried out to show the positioning accuracy of this method is about 80% more than FastSLAM2.0 algorithm's.