Fixed Basis Functions Research Articles

High-Frequency Time Series Data Regression Based on the Functional Bayesian Model Averaging

Regression analysis of high-frequency time series data has been an important problem in the field of statistical learning. In this paper, we propose a novel approach based on multiple basis function expansions and Bayesian model averaging when the predictor variable is a high-frequency time series and the response variable is continuous scalar data. On the one hand, the proposed method avoids the curse of dimensionality and extracts the functional information of the original sequence by transforming the discrete data into continuous functions in a function space consisting of some fixed basis functions. On the other hand, for the choice of basis functions, this paper proposes to adaptively determine the optimal solution using Bayesian model averaging, effectively balancing the variance and bias of the predictive model. The real data analysis shows that the proposed method has smaller mean squared error and absolute error, and is robust compared to other methods. Finally, the proposed method can be further extended to time series data applications such as weather forecasting and stock price prediction.

Global Negative Correlation Learning: A Unified Framework for Global Optimization of Ensemble Models.

Ensembles are a widely implemented approach in the machine learning community and their success is traditionally attributed to the diversity within the ensemble. Most of these approaches foster diversity in the ensemble by data sampling or by modifying the structure of the constituent models. Despite this, there is a family of ensemble models in which diversity is explicitly promoted in the error function of the individuals. The negative correlation learning (NCL) ensemble framework is probably the most well-known algorithm within this group of methods. This article analyzes NCL and reveals that the framework actually minimizes the combination of errors of the individuals of the ensemble instead of minimizing the residuals of the final ensemble. We propose a novel ensemble framework, named global negative correlation learning (GNCL), which focuses on the optimization of the global ensemble instead of the individual fitness of its components. An analytical solution for the parameters of base regressors based on the NCL framework and the global error function proposed is also provided under the assumption of fixed basis functions (although the general framework could also be instantiated for neural networks with nonfixed basis functions). The proposed ensemble framework is evaluated by extensive experiments with regression and classification data sets. Comparisons with other state-of-the-art ensemble methods confirm that GNCL yields the best overall performance.

A health-adaptive time-scale representation (HTSR) embedded convolutional neural network for gearbox fault diagnostics

This research proposes a newly designed convolutional neural network (CNN) for gearbox fault diagnostics. A conventional CNN is a deep-learning model that offers distinctive performance for analyzing two-dimensional image data. To exploit this ability, prior work has been developed using time–frequency analysis, which derives image-like data that is fed into the CNN model. However, the existing time–frequency analysis approach employs fixed basis functions that are limited in their ability to capture fault-related signals in the image. To address this challenge, we propose a health-adaptive time-scale representation (HTSR) embedded CNN (HTSR-CNN). The proposed HTSR approach is designed to exploit the concept of TSR, which is informed by the physics of the time and frequency characteristics induced by the fault-related signals. Instead of using fixed basis functions, the HTSR is constructed using multiscale convolutional filters that behave like the adaptive basis functions. These multiscale filters are effectively learned to include the enriched fault-related information in the HTSR through end-to-end learning of the HTSR-CNN model. The performance of the proposed HTSR-CNN is validated by examining two case studies: vibration signals from a two-stage spur gearbox and vibration signals from a planetary gearbox. From the case study results, the proposed HTSR-CNN method is found to have superior performance for gearbox fault diagnostics, as compared to existing CNN-based fault diagnostic methods.

Decoupling P-NARX models using filtered CPD

Abstract Nonlinear Auto-Regressive eXogenous input (NARX) models are a popular class of nonlinear dynamical models. Often a polynomial basis expansion is used to describe the internal multivariate nonlinear mapping (P-NARX). Resorting to fixed basis functions is convenient since it results in a closed form solution of the estimation problem. The drawback, however, is that the predefined basis does not necessarily lead to a sparse representation of the relationship, typically resulting in very large numbers of parameters. So-called decoupling techniques were specifically designed to reduce large multivariate functions. It was found that, often, a more efficient parameterisation can be retrieved by rotating towards a new basis. Characteristic to the decoupled structure is that, expressed in the new basis, the relationship is structured such that only single-input single-output nonlinear functions are required. Classical decoupling techniques are unfit to deal with the case of single-output NARX models. In this work, this limitation is overcome by adopting the filtered CPD decoupling method of Decuyper et al. (2021b). The approach is illustrated on data from the Sliverbox benchmark: measurement data from an electronic circuit implementation of a forced Duffing oscillator.

Regularization of deep neural networks with spectral dropout

The big breakthrough on the ImageNet challenge in 2012 was partially due to the ‘Dropout’ technique used to avoid overfitting. Here, we introduce a new approach called ‘Spectral Dropout’ to improve the generalization ability of deep neural networks. We cast the proposed approach in the form of regular Convolutional Neural Network (CNN) weight layers using a decorrelation transform with fixed basis functions. Our spectral dropout method prevents overfitting by eliminating weak and ‘noisy’ Fourier domain coefficients of the neural network activations, leading to remarkably better results than the current regularization methods. Furthermore, the proposed is very efficient due to the fixed basis functions used for spectral transformation. In particular, compared to Dropout and Drop-Connect, our method significantly speeds up the network convergence rate during the training process (roughly ×2), with considerably higher neuron pruning rates (an increase of ∼30%). We demonstrate that the spectral dropout can also be used in conjunction with other regularization approaches resulting in additional performance gains.

Ab initio electron-phonon interactions using atomic orbital wave functions

The interaction between electrons and lattice vibrations determines key physical properties of materials, including their electrical and heat transport, excited electron dynamics, phase transitions, and superconductivity. We present a new ab initio method that employs atomic orbital (AO) wavefunctions to compute the electron-phonon (e-ph) interactions in materials and interpolate the e-ph coupling matrix elements to fine Brillouin zone grids. We detail the numerical implementation of such AO-based e-ph calculations, and benchmark them against direct density functional theory calculations and Wannier function (WF) interpolation. The key advantages of AOs over WFs for e-ph calculations are outlined. Since AOs are fixed basis functions associated with the atoms, they circumvent the need to generate a material-specific localized basis set with a trial-and-error approach, as is needed in WFs. Therefore, AOs are ideal to compute e-ph interactions in chemically and structurally complex materials for which WFs are challenging to generate, and are also promising for high-throughput materials discovery. While our results focus on AOs, the formalism we present generalizes e-ph calculations to arbitrary localized basis sets, with WFs recovered as a special case.

Diversity and degrees of freedom in regression ensembles

Abstract Ensemble methods are a cornerstone of modern machine learning. The performance of an ensemble depends crucially upon the level of diversity between its constituent learners. This paper establishes a connection between diversity and degrees of freedom (i.e. the capacity of the model), showing that diversity may be viewed as a form of inverse regularisation. This is achieved by focusing on a previously published algorithm Negative Correlation Learning (NCL), in which model diversity is explicitly encouraged through a diversity penalty term in the loss function. We provide an exact formula for the effective degrees of freedom in an NCL ensemble with fixed basis functions, showing that it is a continuous, convex and monotonically increasing function of the diversity parameter. We demonstrate a connection to Tikhonov regularisation and show that, with an appropriately chosen diversity parameter, an NCL ensemble can always outperform the unregularised ensemble in the presence of noise. We demonstrate the practical utility of our approach by deriving a method to efficiently tune the diversity parameter. Finally, we use a Monte-Carlo estimator to extend the connection between diversity and degrees of freedom to ensembles of deep neural networks.

Bayesian feature learning for seismic compressive sensing and denoising

Extracting the maximum possible information from the available measurements is a challenging task but is required when sensing seismic signals in inaccessible locations. Compressive sensing (CS) is a framework that allows reconstruction of sparse signals from fewer measurements than conventional sampling rates. In seismic CS, the use of sparse transforms has some success; however, defining fixed basis functions is not trivial given the plethora of possibilities. Furthermore, the assumption that every instance of a seismic signal is sparse in any acquisition domain under the same transformation is limiting. We use beta process factor analysis (BPFA) to learn sparse transforms for seismic signals in the time slice and shot record domains from available data, and we use them as dictionaries for CS and denoising. Algorithms that use predefined basis functions are compared against BPFA, with BPFA obtaining state-of-the-art reconstructions, illustrating the importance of decomposing seismic signals into learned features.

Singular spectral analysis based filtering of seismic signal using new Weighted Eigen Spectrogram

Filtering of non-stationary noisy seismic signals using the fixed basis functions (sine and cosine) generates artifacts in the final output and thereby leads to wrong interpretation. In order to circumvent the problem, we propose here, a new Weighted Eigen Spectrogram (WES) based robust time domain Singular Spectrum Analysis (SSA) frequency filtering algorithm. The new WES is used to simplify the Eigen triplet grouping procedure in SSA. We tested the robustness of the algorithm on synthetic seismic data assorted with field-simulated noise. Then we applied the method to filter the high-resolution seismic reflection field data. The band pass filtering of noisy seismic records suggests that the underlying algorithm is efficient for improving the signal to noise ratio (S/N) and also it is user-friendly.

Machinery vibration signal denoising based on learned dictionary and sparse representation

Mechanical vibration signal denoising has been an import problem for machine damage assessment and health monitoring. Wavelet transfer and sparse reconstruction are the powerful and practical methods. However, those methods are based on the fixed basis functions or atoms. In this paper, a novel method is presented. The atoms used to represent signals are learned from the raw signal. And in order to satisfy the requirements of real-time signal processing, an online dictionary learning algorithm is adopted. Orthogonal matching pursuit is applied to extract the most pursuit column in the dictionary. At last, denoised signal is calculated with the sparse vector and learned dictionary. A simulation signal and real bearing fault signal are utilized to evaluate the improved performance of the proposed method through the comparison with kinds of denoising algorithms. Then Its computing efficiency is demonstrated by an illustrative runtime example. The results show that the proposed method outperforms current algorithms with efficiency calculation.

Basis Functions in Image Reconstruction From Projections: A Tutorial Introduction

The series expansion approaches to image reconstruction from projections assume that the object to be reconstructed can be represented as a linear combination of fixed basis functions and the task of the reconstruction algorithm is to estimate the coefficients in such a linear combination based on the measured projection data. It is demonstrated that using spherically symmetric basis functions (blobs), instead of ones based on the more traditional pixels, yields superior reconstructions of medically relevant objects. The demonstration uses simulated computerized tomography projection data of head cross-sections and the series expansion method ART for the reconstruction. In addition to showing the results of one anecdotal example, the relative efficacy of using pixel and blob basis functions in image reconstruction from projections is also evaluated using a statistical hypothesis testing based task oriented comparison methodology. The superiority of the efficacy of blob basis functions over that of pixel basis function is found to be statistically significant.

Reactor power shape synthesis using Group Method of Data Handling

This paper describes the Group Method of Data Handling (GMDH) algorithm to reconstruct 20-node axial core power shapes from five-level in-core detector power measurements. Conventional methods in this field are the parameter identification framework by adopting fixed basis functions and the determination of parameters weighted to each terms of known functions like Fourier series or spline forms. This study proposes GMDH, an evolutionary algorithm, to find the optimal functional form and parameters simultaneously those describe the power distribution accurately. A total of 1060 cases of axially 1-dimensional core power distributions of 20-nodes are generated by 3-dimensional core design code covering beginning-of-life to end-of-life core burnup histories to validate the method. Axially five-point box powers at in-core detector positions are considered as measured snapshots. The reconstructed axial power shapes using GMDH method are compared to the reference power shapes. The results show that the proposed method is very robust and accurate compared with the spline fitting method.

Linear Basis Function Approach to Efficient Alchemical Free Energy Calculations. 1. Removal of Uncharged Atomic Sites.

We present a general approach to transform between molecular potential functions during free energy calculations using a variance minimized linear basis functional form. This approach splits the potential energy function into a sum of pairs of basis functions, which depend on coordinates, and 'alchemical' switches, which depend only on the coupling variable. The power of this approach is that, first, the calculation of the coupling parameter dependent terms is removed from inner loop force calculation routines, second, the flexibility in specifying basis functions and alchemical switches allows users to choose transformation pathways that maximize statistical efficiency, and third, it is possible to predict entirely in postprocessing, without any additional energy evaluations, the thermodynamic properties along any alchemical path with moderate overlap from an initial simulation that uses the same basis functions. This allows construction of optimized, minimum variance alchemical switches from a single simulation with fixed basis functions and trial alchemical switching functions. We describe how to construct these linear basis potentials for real molecular systems of different sizes and shapes, considering particularly the problems of eliminating singularities and minimizing variance of particle removal in dense fluids. The statistical error in free energy calculations using the optimized basis functions is lower than standard soft core models, and approach the minimum variance possible over all pair potentials. We recommend an optimized set of basis functions and alchemical switches for standard molecular free energy calculations.

Construction of customized redundant multiwavelet via increasing multiplicity for fault detection of rotating machinery

Fault detection from the vibration measurement data of rotating machinery is significant for avoiding serious accidents. However, non-stationary vibration signal with a large amount of noise makes this task challenging. Multiwavelet not only owns the advantage on multi-resolution analysis but also can offer multiple wavelet basis functions. So it has the possibility of detecting various fault features preferably. However, the fixed basis functions which are not related to the given signal may lower the accuracy of fault detection. Moreover, another major intrinsic deficiency of multiwavelet lies in its critically sampled filter-bank, which causes shift-variance and is harmful to extract the feature of periodical impulses. To overcome these deficiencies, a new method called customized redundant multiwavelet (CRM) is constructed via increasing multiplicity (IM). IM is a simple method to design a series of changeable multiwavelet which are available for the subsequent optimization process. By the rule of the envelope spectrum entropy minimum principle, optimal multiwavelet is searched for. Based on the customized multiwavelet filters, the filters of CRM can be calculated by inserting zeros. The proposed method is applied to analyze the simulation, gearbox and rolling element bearing vibration signals. Compared with some other conventional methods, the results demonstrate that the proposed method possesses robust performance in detecting fault features of rotating machinery.

Constrained non-negative sparse coding using learnt instrument templates for realtime music transcription

In this paper, we present a realtime signal decomposition method with single-pitch and harmonicity constrains using instrument specific information. Although the proposed method is designed for monophonic music transcription, it can be used as a candidate selection technique in combination with other realtime transcription methods to address polyphonic signals. The harmonicity constraint is particularly beneficial for automatic transcription because, in this way, each basis can define a single pitch. Furthermore, restricting the model to have a single-nonzero gain at each frame has been shown to be a very suitable constraint when dealing with monophonic signals. In our method, both harmonicity and single-nonzero gain constraints are enforced in a deterministic manner. A realtime factorization procedure based on Non-negative sparse coding (NNSC) with Beta-divergence and fixed basis functions is proposed. In this paper, the basis functions are learned using a supervised process to obtain spectral patterns for different musical instruments. The proposed method has been tested for music transcription of both monophonic and polyphonic signals and has been compared with other state-of-the-art transcription methods, and in these tests, the proposed method has obtained satisfactory results in terms of accuracy and runtime.

Construction of adaptive redundant multiwavelet packet and its application to compound faults detection of rotating machinery

It is significant to detect the fault type and assess the fault level as early as possible for avoiding catastrophic accidents. Due to diversity and complexity, the compound faults detection of rotating machinery under non-stationary operation turns to be a challenging task. Multiwavelet with two or more base functions may match two or more features of compound faults, which may supply a possible solution to compound faults detection. However, the fixed basis functions of multiwavelet transform, which are not related with the vibration signal, may reduce the accuracy of compound faults detection. Moreover, the decomposition results of multiwavelet transform not being own time-invariant is harmful to extract the features of periodical impulses. Furthermore, multiwavelet transform only focuses on the multi-resolution analysis in the low frequency band, and may leave out the useful features of compound faults. To overcome these shortcomings, a novel method called adaptive redundant multiwavelet packet (ARMP) is proposed based on the two-scale similarity transforms. Besides, the relative energy ratio at the characteristic frequency of the concerned component is computed to select the sensitive frequency bands of multiwavelet packet coefficients. The proposed method was used to analyze the compound faults of rolling element bearing. The results showed that the proposed method could enhance the ability of compound faults detection of rotating machinery.

Estimation of reflectance from camera responses by the regularized local linear model

Because of the limited approximation capability of using fixed basis functions, the performance of reflectance estimation obtained by traditional linear models will not be optimal. We propose an approach based on the regularized local linear model. Our approach performs efficiently and knowledge of the spectral power distribution of the illuminant and the spectral sensitivities of the camera is not needed. Experimental results show that the proposed method performs better than some well-known methods in terms of both reflectance error and colorimetric error.

Can dictionary-based computational models outperform the best linear ones?

Approximation capabilities of two types of computational models are explored: dictionary-based models (i.e., linear combinations of n-tuples of basis functions computable by units belonging to a set called "dictionary") and linear ones (i.e., linear combinations of n fixed basis functions). The two models are compared in terms of approximation rates, i.e., speeds of decrease of approximation errors for a growing number n of basis functions. Proofs of upper bounds on approximation rates by dictionary-based models are inspected, to show that for individual functions they do not imply estimates for dictionary-based models that do not hold also for some linear models. Instead, the possibility of getting faster approximation rates by dictionary-based models is demonstrated for worst-case errors in approximation of suitable sets of functions. For such sets, even geometric upper bounds hold.

Signal Fitting With Uncertain Basis Functions

A new paradigm for signal fitting is proposed. Unlike the customary approach in which fixed basis functions are used to represent the signal, the proposed method employs random basis functions. The advantage is an increase in robustness, leading to an overall decrease in modeling error. It also provides a new intepretation on the choice of regularization weightings for such applications as classification, spectral analysis, and adaptive beamforming.

Nonparametric Regression Method Based on Orthogonalization and Thresholding

In this paper, we consider a nonparametric regression problem using a learning machine defined by a weighted sum of fixed basis functions, where the number of basis functions, or equivalently, the number of weights, is equal to the number of training data. For the learning machine, we propose a training scheme that is based on orthogonalization and thresholding. On the basis of the scheme, vectors of basis function outputs are orthogonalized and coefficients of the orthogonalized vectors are estimated instead of weights. The coefficient is set to zero if it is less than a predetermined threshold level assigned component-wise to each coefficient. We then obtain the resulting weight vector by transforming the thresholded coefficients. In this training scheme, we propose asymptotically reasonable threshold levels to distinguish contributed components from unnecessary ones. To see how this works in a simple case, we derive an upper bound for the generalization error of the training scheme with the given threshold levels. It tells us that an increase in the generalization error is of O(log n/n) when there is a sparse representation of a target function in an orthogonal domain. In implementing the training scheme, eigen-decomposition or the Gram-Schmidt procedure is employed for orthogonalization, and the corresponding training methods are referred to as OHTED and OHTGS. Furthermore, modified versions of OHTED and OHTGS, called OHTED2 and OHTGS2 respectively, are proposed for reduced estimation bias. On real benchmark datasets, OHTED2 and OHTGS2 are found to exhibit relatively good generalization performance. In addition, OHTGS2 is found to be obtain a sparse representation of a target function in terms of the basis functions.