Trend Filtering Research Articles

Locally Adaptive Shrinkage Priors for Trends and Breaks in Count Time Series

Non-stationary count time series characterized by features such as abrupt changes and fluctuations about the trend arise in many scientific domains including biophysics, ecology, energy, epidemiology, and social science domains. Current approaches for integer-valued time series lack the flexibility to capture local transient features while more flexible models for continuous data types are inadequate for universal applications to integer-valued responses such as settings with small counts. We present a modeling framework, the negative binomial Bayesian trend filter (NB-BTF), that offers an adaptive model-based solution to capturing multiscale features with valid integer-valued inference for trend filtering. The framework is a hierarchical Bayesian model with a dynamic global-local shrinkage process. The flexibility of the global-local process allows for the necessary local regularization while the temporal dependence induces a locally smooth trend. In simulation, the NB-BTF outperforms a number of alternative trend filtering methods. Then, we demonstrate the method on weekly power outage frequency in Massachusetts townships. Power outage frequency is characterized by a persistent low level trend with occasional spikes. These illustrations show the estimation of a smooth, non-stationary trend with adequate uncertainty quantification.

Controlling the false discovery rate in transformational sparsity: Split Knockoffs

Abstract Controlling the False Discovery Rate (FDR) in a variable selection procedure is critical for reproducible discoveries, and it has been extensively studied in sparse linear models. However, it remains largely open in scenarios where the sparsity constraint is not directly imposed on the parameters but on a linear transformation of the parameters to be estimated. Examples of such scenarios include total variations, wavelet transforms, fused LASSO, and trend filtering. In this paper, we propose a data-adaptive FDR control method, called the Split Knockoff method, for this transformational sparsity setting. The proposed method exploits both variable and data splitting. The linear transformation constraint is relaxed to its Euclidean proximity in a lifted parameter space, which yields an orthogonal design that enables the orthogonal Split Knockoff construction. To overcome the challenge that exchangeability fails due to the heterogeneous noise brought by the transformation, new inverse supermartingale structures are developed via data splitting for provable FDR control without sacrificing power. Simulation experiments demonstrate that the proposed methodology achieves the desired FDR and power. We also provide an application to Alzheimer’s Disease study, where atrophy brain regions and their abnormal connections can be discovered based on a structural Magnetic Resonance Imaging dataset.

ℓ0 Trend Filtering

The [Formula: see text] trend filtering ([Formula: see text]-TF) is a new effective tool for nonparametric regression with the power of automatic knot detection in function values or derivatives. It overcomes the drawback of [Formula: see text]-TF that is known to have bias issues. To solve the [Formula: see text]-TF problem, we propose an alternating minimization induced active set (AMIAS) search method based on the necessary optimality conditions derived from an augmented Lagrangian framework. The proposed method takes full advantage of the primal and dual variables with complementary supports, and decouples the high-dimensional problem into two subsystems on the active and inactive sets, respectively. A sequential AMIAS algorithm with warm start initialization is developed for efficient determination of the cardinality parameter, along with the output of solution paths. Theoretically, the oracle estimator of [Formula: see text]-TF is justified to behave like regression splines under the continuous time setting with mild conditions. Our numerical experiments include simulation studies for comparing [Formula: see text]-TF to [Formula: see text]-TF and free-knot splines on several synthetic examples, and a real data application of time series segmentation on Hong Kong PM2.5 indexes. History: Accepted by Antonio Frangioni, Area Editor for Design & Analysis of Algorithms – Continuous. Funding: This work was supported in part by Hong Kong General Research Fund [No. 17306519]. C. Wen’s research is partially supported by National Science Foundation of China [12171449] and Fundamental Research Funds for the Central Universities [WK3470000027, YD2040002019]. X. Wang’s research is partially supported by National Natural Science Foundation of China [Grants 72171216, 12231017, 71921001, and 71991474], and the National Key R&D Program of China [No. 2022YFA1003803]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoc.2021.0313 . The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2021.0313 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0313 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Structure recovery and trend estimation for dynamic network analysis

Low‐dimensional parametric models for network dynamics have been successful as inferentially efficient and interpretable tools for modelling network evolution but have difficulty in settings with strong time inhomogeneity (particularly when sharp variation in parameters is possible and covariates are limited). Here, we propose to address this problem via a novel family of block‐structured dynamic exponential‐family random graph models (ERGMs), where the time domain is divided into consecutive blocks and the network parameters are assumed to evolve smoothly within each block. In particular, we let the latent ERGM parameters follow a piecewise polynomial model with an unknown block structure (e.g., change points). We propose an iterative estimation procedure that involves estimating the block structure using trend filtering and fitting ERGMs for networks belonging to the same time block. We demonstrate the utility of the proposed approach using simulation studies and applications to interbank transaction networks and citations among political blogs over the course of an electoral cycle.

COVID-19 Active Case Forecasts in Latin American Countries Using Score-Driven Models

With the aim of mitigating the damage caused by the coronavirus disease 2019 (COVID-19) pandemic, it is important to use models that allow forecasting possible new infections accurately in order to face the pandemic in specific sociocultural contexts in the best possible way. Our first contribution is empirical. We use an extensive COVID-19 dataset from nine Latin American countries for the period of 1 April 2020 to 31 December 2021. Our second and third contributions are methodological. We extend relevant (i) state-space models with score-driven dynamics and (ii) nonlinear state-space models with unobserved components, respectively. We use weekly seasonal effects, in addition to the local-level and trend filters of the literature, for (i) and (ii), and the negative binomial distribution for (ii). We find that the statistical and forecasting performances of the novel score-driven specifications are superior to those of the nonlinear state-space models with unobserved components model, providing a potential valid alternative to forecasting the number of possible new COVID-19 infections.

Trend filtering by adaptive piecewise polynomials

Trend filtering is a regression problem to estimate underlying trends in time series data. It is necessary to investigate data in various disciplines. We propose a trend filtering method by adaptive piecewise polynomials. More specifically, we adjust the location and the number of breakpoints or knots to obtain a better fitting to given data. The numerical results on synthetic and real data sets show that it captures distinct features such as abrupt changes or kinks and provides a simplified form and brief summary of given data.

Converting ADMM to a proximal gradient for efficient sparse estimation

In sparse estimation, such as fused lasso and convex clustering, we apply either the proximal gradient method or the alternating direction method of multipliers (ADMM) to solve the problem. It takes time to include matrix division in the former case, while an efficient method such as FISTA (fast iterative shrinkage-thresholding algorithm) has been developed in the latter case. This paper proposes a general method for converting the ADMM solution to the proximal gradient method, assuming that assumption that the derivative of the objective function is Lipschitz continuous. Then, we apply it to sparse estimation problems, such as sparse convex clustering and trend filtering, and we show by numerical experiments that we can obtain a significant improvement in terms of efficiency.

Tensor denoising with trend filtering

We extend the notion of trend filtering to tensors by considering the $k\mathrm{th}$-order Vitali variation – a discretized version of the integral of the absolute value of the $k\mathrm{th}$-order total derivative. We prove adaptive $\ell^0$-rates and not-so-slow $\ell^1$-rates for tensor denoising with trend filtering. For $k={1,2,3,4}$ we prove that the $d$-dimensional margin of a $d$-dimensional tensor can be estimated at the $\ell^0$-rate $n^{-1}$, up to logarithmic terms, if the underlying tensor is a product of $(k-1)\mathrm{th}$-order polynomials on a constant number of hyperrectangles. For general $k$ we prove the $\ell^1$-rate of estimation $n^{- \frac{H(d)+2k-1}{2H(d)+2k-1}}$, up to logarithmic terms, where $H(d)$ is the $d\mathrm{th}$ harmonic number. Thanks to an ANOVA-type of decomposition we can apply these results to the lower dimensional margins of the tensor to prove bounds for denoising the whole tensor. Our tools are interpolating tensors to bound the effective sparsity for $\ell^0$-rates, mesh grids for $\ell^1$-rates and, in the background, the projection arguments by Dalalyan, Hebiri, and Lederer (2017).

ℓ 2 and ℓ 1 Trend Filtering: A Kalman Filter Approach [Lecture Notes

Two of the most popular denoising algorithms are ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> and ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> trend filtering, which are used in science, engineering, and statistical signal and image processing. They are typically treated as separate entities, with the former as a linear time-invariant (LTI) filter, which is commonly used for smoothing the noisy data and detrending the time-series signals, while the latter is a nonlinear filtering method suited for the estimation of piecewise-polynomial signals (e.g., piecewise constant, piecewise linear, piecewise quadratic, and so on) observed in additive white Gaussian noise.

Ell _{1} Common Trend Filtering

The ell _{1} trend filtering enables us to estimate a continuous piecewise linear trend of univariate time series. This filter and its variants have subsequently been applied in various fields, including astronomy, climatology, economics, electronics, environmental science, finance, and geophysics. Although the ell _{1} trend filtering can estimate a continuous piecewise linear trend of univariate time series, it cannot estimate a common continuous piecewise linear trend of multiple time series. This paper develops a statistical procedure that enables us to estimate it, which is a multivariate extension of the ell _{1} trend filtering. We provide an algorithm for estimating it and a clue to specify the tuning parameter of the procedure, both required for its application. We also (i) numerically illustrate how well the algorithm works, (ii) provide an empirical illustration, and (iii) introduce a generalization of our novel method.

Sparse factor model based on trend filtering

We present a model based on trend filtering and regularization for performing factor analysis. Furthermore, the trend filtering method proposed in our model allows incorporating views represented as scenarios. Therefore, factor models can be optimized to explain not only trends of a given data series but also trends reflecting outlooks. As an application to finance, the proposed model constructs a sparse factor model on the trend of an index. Factor analyses on trend-filtered series provide factor models for describing trends, which are valuable for modeling long-term horizons. We also analyze the effect of trends on factor model selection in the US stock market. Further analyses include an investigation of factors during crisis periods and a comparison when various scenarios are considered.

Pattern similarity-based machine learning methods for mid-term load forecasting: A comparative study

Pattern similarity-based frameworks are widely used for classification and regression problems. Repeated, similar-shaped cycles observed in seasonal time series encourage the use of such frameworks for forecasting. In this paper, we use pattern similarity-based models for mid-term load forecasting. An integral part of these models is the use of patterns of time series sequences for time series representation. Pattern representation ensures input and output data unification through trend filtering and variance equalization. This simplifies the forecasting problem and allows us to use models based on pattern similarity. We consider four such models: nearest-neighbor model, fuzzy neighborhood model, kernel regression model, and general regression neural network. Three variants of the approach were proposed. A basic one and two hybrid solutions combining similarity-based and statistical methods (ARIMA and exponential smoothing).In the experimental part of the work, the proposed models were used to forecast the monthly electricity demand in 35 European countries. The results show the high performance of the proposed models, which outperform both the comparative classical statistical models and machine learning models in terms of accuracy, simplicity, and ease of optimization. Among the proposed variants, a hybrid approach combining similarity-based methods with exponential smoothing turned out to be the most accurate. The study highlights the many advantages of the proposed pattern similarity-based models such as clear operation principles, a small number of parameters to adjust, no training procedure, fast optimization procedure, good generalization ability, ability to work on the newest data without retraining, and delivery of multi-step forecasts.

Lazy Momentum with Growth-Trend timing: Resilient Asset Allocation (RAA)

Resilient Asset Allocation (RAA) is a more aggressive version of our Lethargic Asset Allocation (LAA) strategy. It combines a more robust “All Weather” portfolio with even slower growth-trend (GT) filter and a faster market crash-protection. GT timing goes risk-off only when both the US unemployment (UE) and the US capital markets are bearish. To arrive at RAA, we adapt LAA in three steps. First, the (risky, near-static) portfolio is changed to an even more robust and more diversified “all-weather” portfolio, now with five (instead of four) equal weighted assets and with only bonds as risk-off assets (“cash”). Second, the “canary” technology from our Defensive Asset Allocation (DAA) paper is used for determining the market trend with a faster filter. Third, we change the unemployment trend filter to a slower one, where we simply compare the recent unemployment rate with that of one year ago. As a result, RAA is more aggressive and more robust than LAA, while at the same time nearly as “lazy” with respect to trading and turnover (on average one trading month per year).

Compositional trend filtering

Trend filtering is known as the technique for detecting piecewise linear trends in univariate time series. This technique is extended to the setting of compositional data, which are multivariate data where only the relative information is of importance. According to this, we formulate the problem and present a procedure how to efficiently solve it. To show the usefulness of this method, we consider the number of COVID-19 infections in several European countries in a chosen time period.

Online Incipient Fault Detection Method Based on Improved ℓ1 Trend Filtering and Support Vector Data Description

Poor model generalization, missing or false alarms, and heavy dependence on expert’s experience are some of the major problems which exist in traditional incipient fault detection (IFD) methods. An IFD rolling bearing application method based on combination of improved $\lambda _{1}$ trend filtering (L1TF) and support vector data description (SVDD) is proposed. First, spectral distance index and multi-scale dispersion entropy based on normal vibration data, which is sensitive to incipient faults, are extracted. The improved $\lambda _{1}$ trend filter (IL1TF) method is employed for processing the feature values and obtaining a trend factor with less fluctuation and better incipient fault indication ability. Then, after determining the kernel function bandwidth of the SVDD by analyzing the characteristics of the training data, a suitable offline SVDD model is trained. Finally, incipient faults are identified by estimating the distance between the trend factor of the real-time data and the center of the hypersphere in the SVDD model. This method employs full performance of SVDD to detect abnormal data files, while reducing the influence of abnormal data files on the model via IL1TF. Furthermore, the method increases the discrimination between the incipient fault data and the normal data. By utilizing Intelligent Maintenance Systems of University of Cincinnati bearing laboratory data and Chinese petrochemical company’s centrifugal pump bearing engineering data, the effectiveness of the constructed model is demonstrated. In addition, the proposed method is compared against existing representative IFD methods. The results indicate that the method proposed in this paper can solve false alarms and detect incipient failure data files more accurately without depending on the external expert’s experience. This is of great significance for providing guidelines to enterprises which employ predictive maintenance techniques.

Sparse HP filter: Finding kinks in the COVID-19 contact rate

In this paper, we estimate the time-varying COVID-19 contact rate of a Susceptible–Infected–Recovered (SIR) model. Our measurement of the contact rate is constructed using data on actively infected, recovered and deceased cases. We propose a new trend filtering method that is a variant of the Hodrick–Prescott (HP) filter, constrained by the number of possible kinks. We term it the sparse HP filter and apply it to daily data from five countries: Canada, China, South Korea, the UK and the US. Our new method yields the kinks that are well aligned with actual events in each country. We find that the sparse HP filter provides a fewer kinks than the ℓ1 trend filter, while both methods fitting data equally well. Theoretically, we establish risk consistency of both the sparse HP and ℓ1 trend filters. Ultimately, we propose to use time-varying contact growth rates to document and monitor outbreaks of COVID-19.

Weighted Bayesian bootstrap for scalable posterior distributions

AbstractWe introduce and develop a weighted Bayesian bootstrap (WBB) for machine learning and statistics. WBB provides uncertainty quantification by sampling from a high dimensional posterior distribution. WBB is computationally fast and scalable using only off‐the‐shelf optimization software. First‐order asymptotic analysis provides a theoretical justification under suitable regularity conditions on the statistical model. We illustrate the proposed methodology in regularized regression, trend filtering and deep learning and conclude with directions for future research.

Universal Adaptive Stabilization-Based Trend Filtering for Impending Battery Voltage Collapse Detection

Battery operated devices are common in everyday life. Deciding when a warning for impending battery voltage collapse should be triggered is not trivial. This letter develops an impending battery terminal voltage collapse detection system using a universal adaptive stabilizer (UAS) and a well-known trend filter. This eliminates requiring knowledge of battery model parameter values, or initial state-of-charge (SOC). The proposed approach overcomes the need for extensive training when compared to using neural-networks based techniques. Also, the developed trend filter when used with a UAS, eliminates the need for selecting windows of data to be processed. This is advantageous compared to earlier work, which uses a different trend filtering mechanism, because selection of window sizes is not straightforward. Further, the approach used in this letter shows that the UAS based technique is implementable on a cell phone. Associated mathematical results, and experimental data from such an implementation are presented. Additionally, the technique is also applied to other larger capacity Li-ion batteries showing its versatility. The developed technique can also be used to detect when the state-of-health (SOH) of a Li-ion battery is about to enter an unsafe region.

Sparse HP Filter: Finding Kinks in the COVID-19 Contact Rate

In this paper, we estimate the time-varying COVID-19 contact rate of a Susceptible-Infected-Recovered (SIR) model. Our measurement of the contact rate is constructed using data on actively infected, recovered and deceased cases. We propose a new trend filtering method that is a variant of the Hodrick-Prescott (HP) filter, constrained by the number of possible kinks. We term it the sparse HP filter and apply it to daily data from five countries: Canada, China, South Korea, the UK and the US. Our new method yields the kinks that are well aligned with actual events in each country. We find that the sparse HP filter provides a fewer kinks than the l1 trend filter, while both methods fitting data equally well. Theoretically, we establish risk consistency of both the sparse HP and l1 trend filters. Ultimately, we propose to use time-varying contact growth rates to document and monitor outbreaks of COVID-19.

Fault diagnosis for rolling element bearing using variational mode decomposition and l1 trend filtering

In order to extract the faint fault features of bearings in strong noise background, a method based on variational mode decomposition and l1 trend filtering is proposed in this study. In the variational model, the mode number κ is determined difficulty, thus l1 trend filtering is applied to simplify the frequency spectrum of the original signals. In this case, this parameter can be defined easily. At the same time, a criterion based on kurtosis is used to adaptively select the regularization parameter of l1 trend filtering. The combined approach is evaluated by simulation analysis and the vibration signals of damaged bearings with a rolling element fault, an outer race fault and an inner race fault. The results demonstrate that the hybrid method is effective in detecting the three bearing faults. Moreover, compared with another approach based on multiscale morphology and empirical mode decomposition, the proposed method can extract more bearing fault features.