Unimodal Regression Research Articles

A dynamic programming approach for generalized nearly isotonic optimization

Shape restricted statistical estimation problems have been extensively studied, with many important practical applications in signal processing, bioinformatics, and machine learning. In this paper, we propose and study a generalized nearly isotonic optimization (GNIO) model, which recovers, as special cases, many classic problems in shape constrained statistical regression, such as isotonic regression, nearly isotonic regression and unimodal regression problems. We develop an efficient and easy-to-implement dynamic programming algorithm for solving the proposed model whose recursion nature is carefully uncovered and exploited. For special $$\ell _2$$ -GNIO problems, implementation details and the optimal $$\mathcal{O}(n)$$ running time analysis of our algorithm are discussed. Numerical experiments, including the comparisons among our approach, the powerful commercial solver Gurobi, and existing fast algorithms for solving $$\ell _1$$ -GNIO and $$\ell _2$$ -GNIO problems, on both simulated and real data sets, are presented to demonstrate the high efficiency and robustness of our proposed algorithm in solving large scale GNIO problems.

Adaptive confidence sets in shape restricted regression

A simple construction of adaptive confidence sets is proposed in isotonic, convex and unimodal regression. In univariate isotonic regression, the proposed confidence set enjoys uniform coverage over all non-decreasing regression functions. Furthermore, the diameter of the proposed confidence set automatically adapts to the unknown number of pieces of the true parameter, in the sense that the diameter is bounded from above by the minimax risk over the class of $k$-piecewise constant functions. The diameter of the confidence set is a simple increasing function of the number of jumps of the isotonic least-squares estimate. A similar construction is proposed in convex regression where the true regression function is convex and piecewise affine. Here, the confidence set enjoys uniform coverage and its diameter automatically adapts to the number of affine pieces of the true regression function. The diameter of the confidence set is an increasing function of the number of affine pieces of the convex least-squares estimate. We explain how to extend this technique to a non-convex set by proposing a similar adaptive confidence set in unimodal regression. The confidence set automatically adapts to the number of jumps of the true unimodal regression function and its diameter is an increasing function of the number of jumps of the unimodal least-squares estimate.

Adaptive risk bounds in unimodal regression

We study the statistical properties of the least squares estimator in unimodal sequence estimation. Although closely related to isotonic regression, unimodal regression has not been as extensively studied. We show that the unimodal least squares estimator is adaptive in the sense that the risk scales as a function of the number of values in the true underlying sequence. Such adaptivity properties have been shown for isotonic regression by Chatterjee et al. (Ann. Statist. 43 (2015) 1774–1800) and Bellec (Sharp oracle inequalities for Least Squares estimators in shape restricted regression (2016)). A technical complication in unimodal regression is the non-convexity of the underlying parameter space. We develop a general variational representation of the risk that holds whenever the parameter space can be expressed as a finite union of convex sets, using techniques that may be of interest in other settings.

Sharp oracle inequalities for Least Squares estimators in shape restricted regression

The performance of Least Squares (LS) estimators is studied in shape-constrained regression models under Gaussian and sub-Gaussian noise. General bounds on the performance of LS estimators over closed convex sets are provided. These results have the form of sharp oracle inequalities that account for the model misspecification error. In the presence of misspecification, these bounds imply that the LS estimator estimates the projection of the true parameter at the same rate as in the well-specified case. In isotonic and unimodal regression, the LS estimator achieves the nonparametric rate $n^{-2/3}$ as well as a parametric rate of order $k/n$ up to logarithmic factors, where $k$ is the number of constant pieces of the true parameter. In univariate convex regression, the LS estimator satisfies an adaptive risk bound of order $q/n$ up to logarithmic factors, where $q$ is the number of affine pieces of the true regression function. This adaptive risk bound holds for any collection of design points. While Guntuboyina and Sen [Probab. Theory Related Fields 163 (2015) 379–411] established that the nonparametric rate of convex regression is of order $n^{-4/5}$ for equispaced design points, we show that the nonparametric rate of convex regression can be as slow as $n^{-2/3}$ for some worst-case design points. This phenomenon can be explained as follows: Although convexity brings more structure than unimodality, for some worst-case design points this extra structure is uninformative and the nonparametric rates of unimodal regression and convex regression are both $n^{-2/3}$. Higher order cones, such as the cone of $\beta $-monotone sequences, are also studied.

Approximate solution of the inverse Richards’ problem

We propose a method for the estimation of time dependent distributions of pressure head, water content, and fluid flow in homogeneous unsaturated soils with unknown lower boundary conditions using surface measurements only.The unknown boundary condition is replaced by a piecewise constant temporal function and the resulting discontinuity is alleviated by the introduction of a mass balance condition on the solution at discontinuity points. This approach makes it possible to express the analytical solution of Richards’ one-dimensional equation as a linear function of a finite number of variables corresponding to the unknown coefficients of the piecewise constant function.While the estimation of unknown boundary belongs to a class of typically ill-posed inverse problems, the simplifications introduced in the algorithm provide for the regularization of this particular problem without the use of traditional smoothing techniques, such as Tikhonov's method and Morozov's discrepancy principle.A Bayesian estimation method and a unimodal regression algorithm have been employed to test the overall algorithm using simulated data.

Unimodal regression using Bernstein-Schoenberg splines and penalties.

Research in the field of nonparametric shape constrained regression has been intensive. However, only few publications explicitly deal with unimodality although there is need for such methods in applications, for example, in dose-response analysis. In this article, we propose unimodal spline regression methods that make use of Bernstein-Schoenberg splines and their shape preservation property. To achieve unimodal and smooth solutions we use penalized splines, and extend the penalized spline approach toward penalizing against general parametric functions, instead of using just difference penalties. For tuning parameter selection under a unimodality constraint a restricted maximum likelihood and an alternative Bayesian approach for unimodal regression are developed. We compare the proposed methodologies to other common approaches in a simulation study and apply it to a dose-response data set. All results suggest that the unimodality constraint or the combination of unimodality and a penalty can substantially improve estimation of the functional relationship.

Chironomid-based inference models for estimating mean July air temperature and water depth from lakes in Yakutia, northeastern Russia

We investigated the subfossil chironomid fauna of 150 lakes situated in Yakutia, northeastern Russia. The objective of this study was to assess the relationship between chironomid assemblage composition and the environment and to develop chironomid inference models for quantifying past regional climate and environmental changes in this poorly investigated area of northern Russia. The environmental data and sediment samples for chironomid analysis were collected in 5 consecutive years, 2003–2007, from several regions of Yakutia. The lakes spanned wide latitudinal and longitudinal ranges and were distributed through several environmental zones (arctic tundra, typical tundra, steppe-tundra, boreal coniferous forest), but all were situated within the zone of continuous permafrost. Mean July temperature (TJuly) varied from 3.4°C in the Laptev Sea region to 18.8°C in central Yakutia near Yakutsk. Water depth (WD) varied from 0.1 to 17.1 m. TJuly and WD were identified as the strongest predictor variables explaining the chironomid communitiy composition and distribution of the taxa in our data set. Quantitative transfer functions were developed using two unimodal regression calibration techniques: simple weighted averaging (WA) and weighted averaging partial least squares (WA-PLS). The two-component TJuly WA-PLS model had the best performance. It produced a strong coefficient of determination (r2boot = 0.87), root mean square error of prediction (RMSEP = 1.93), and max bias (max biasboot = 2.17). For WD, the one-component WA-PLS model had the best performance (r2boot = 0.62, RMSEP = 0.35, max biasboot = 0.47).

On Unimodal Regression in the Exponential Family

In this article, we discuss statistical methods for curve-estimation under the assumption of unimodality for variables with distributions belonging to the two-parameter exponential family with known or constant dispersion parameter. An important special case is a one-parameter distribution. We suggest a nonparametric method based on monotonicity properties. The method is applied to Swedish data on laboratory verified diagnoses of influenza and data on inflation from an episode of hyperinflation in Bulgaria.

Unimodal regression via prefix isotonic regression

This paper gives algorithms for determining real-valued univariate unimodal regressions, that is, for determining the optimal regression which is increasing and then decreasing. Such regressions arise in a wide variety of applications. They are shape-constrained nonparametric regressions, closely related to isotonic regression. For unimodal regression on n weighted points our algorithm for the L 2 metric requires only Θ ( n ) time, while for the L 1 metric it requires Θ ( n log n ) time. For unweighted points our algorithm for the L ∞ metric requires only Θ ( n ) time. All of these times are optimal. Previous algorithms were for the L 2 metric and required Ω ( n 2 ) time. All previous algorithms used multiple calls to isotonic regression, and our major contribution is to organize these into a prefix isotonic regression, determining the regression on all initial segments. The prefix approach reduces the total time required by utilizing the solution for one initial segment to solve the next.

Predictions by early indicators of the time and height of the peaks of yearly influenza outbreaks in Sweden

Methods for prediction of the peak of the influenza from early observations are suggested. These predictions can be used for planning purposes. In this study, new robust methods are described and applied to weekly Swedish data on influenza-like illness (ILI) and weekly laboratory diagnoses of influenza (LDI). Both simple and advanced rules for how to predict the time and height of the peak of LDI are suggested. The predictions are made using covariates calculated from data in early LDI reports. The simple rules are based on the observed LDI values, while the advanced ones are based on smoothing by unimodal regression. The suggested predictors were evaluated by cross-validation and by application to the observed seasons. The relationship between ILI and LDI was investigated, and it was found that the ILI variable is not a good proxy for the LDI variable. The advanced prediction rule regarding the time of the peak of LDI had a median error of 0.9 weeks, and the advanced prediction rule for the height of the peak had a median deviation of 28%. The statistical methods for predictions have practical usefulness.

On Automatic Absorption Detection for Imaging Spectroscopy: A Comparative Study

In this paper, we aim at presenting a survey on automatic absorption recovery methods for imaging spectroscopy. We commence by viewing the algorithms in the literature from a technical perspective and presenting an overview of the derivative analysis, fingerprint, and maximum modulus wavelet transform techniques. In addition to these methods, we also present a novel absorption recovery approach based upon unimodal regression and continuum removal. With this technical review of the methods under study, we perform a complexity analysis and examine the implementation issues pertaining to each of the alternatives. We show how detected absorption bands can be used for purposes of material identification. We conclude this paper by providing a performance study and providing identification results on hyperspectral imagery. To this end, we make use of a number of distance measures to evaluate the quality of the recovered absorptions, as compared to continuum-removed spectra.

Algorithms for unimodal segmentation with applications to unimodality detection

We study the problem of segmenting a sequence into k pieces so that the resulting segmentation satisfies monotonicity or unimodality constraints. Unimodal functions can be used to model phenomena in which a measured variable first increases to a certain level and then decreases. We combine a well-known unimodal regression algorithm with a simple dynamic-programming approach to obtain an optimal quadratic-time algorithm for the problem of unimodal k-segmentation. In addition, we describe a more efficient greedy-merging heuristic that is experimentally shown to give solutions very close to the optimal. As a concrete application of our algorithms, we describe methods for testing if a sequence behaves unimodally or not. The methods include segmentation error comparisons, permutation testing, and a BIC-based scoring scheme. Our experimental evaluation shows that our algorithms and the proposed unimodality tests give very intuitive results, for both real-valued and binary data.

Linear time isotonic and unimodal regression in the [formula omitted] and [formula omitted] norms

We consider L 1 -isotonic regression and L ∞ isotonic and unimodal regression. For L 1 -isotonic regression, we present a linear time algorithm when the number of outputs are bounded. We extend the algorithm to construct an approximate isotonic regression in linear time when the output range is bounded. We present linear time algorithms for L ∞ isotonic and unimodal regression.

Least squares estimators of the mode of a unimodal regression function

In this paper, we consider nonparametric least squares estimators of the mode of an unknown unimodal regression function. We establish almost sure convergence of these estimators with nearly optimal convergence rates, under the assumption of the exponential tail for the error distributions.

Use of evolutionary algorithms for the calculation of group contribution parameters in order to predict thermodynamic properties: Part 2: Encapsulated evolution strategies

The computation of parameters of group contribution models in order to predict thermodynamic properties usually leads to a multiparameter optimization problem where the model parameters are calculated using a regression method. A complex objective function occurs for which an optimization algorithm has to find the global minimum. Simple increment or simple group contribution models often result in unimodal regression problems, for which deterministically acting algorithms are suitable. If the model contains parameters in complex terms such as sums of exponential expressions, the optimization problem will be a nonlinear regression problem which often results in a multimodal optimization problem. In this case, the search of the global or at least a fairly good optimum becomes rather difficult. Evolutionary algorithms are suitable for solving such multimodal problems. Friese, T., Ulbig, P., & Schulz, S. (1998). Use of evolutionary algorithms for the calculation of group contribution parameters in order to predict thermodynamic properties (Part 1): Genetic algorithms. Computers and Chemical Engineering 22(11), 1559–1572 showed that the efficiency of genetic algorithms applied to the presented optimization problem, this paper shows that evolution strategies are suitable, as well. This work first describes the typical mode of acting of evolution strategies before a new variant, the so-called encapsulated evolution strategy using a multidimensional step-length control is introduced. This new type of strategy proved to be superior to conventional evolution strategies and genetic algorithms. In order to benefit from this new algorithm for other similar optimization problems, an optimum strategy type is determined and analyzed with the help of two visualized test systems representing the complex of optimization problems, which nonlinear parameter fittings of group contribution model parameters belong to.

The Effect of Truncation on the Identifiability of Regression Coefficients

Abstract When data are truncated and hence partly unobservable, parameters identifiable under the complete observation may not be identifiable. A class of distributions is characterized, under which regression coefficients are unidentifiable by truncation of a dependent variable in the regression model. In a class of strongly unimodal densities regression coefficients are shown to be unidentifiable only when the underlying distribution is proportional to the exponential distribution in support of the truncated observation. The exponential, double exponential, and Huber's least-informative distribution for a location parameter are apparent examples of the distributions under which the unidentifiability can occur. In a more restrictive class of symmetric and strongly unimodal densities the unidentifiability arises only when the error distribution has exponential tails and the degree of truncation is at least 50%. The likelihood function becomes completely flat with respect to the regression coefficients and...