Monotonicity Constraints Research Articles

Semiparametric probit models with univariate and bivariate current-status data.

Multivariate current-status data are frequently encountered in biomedical and public health studies. Semiparametric regression models have been extensively studied for univariate current-status data, but most existing estimation procedures are computationally intensive, involving either penalization or smoothing techniques. It becomes more challenging for the analysis of multivariate current-status data. In this article, we study the maximum likelihood estimations for univariate and bivariate current-status data under the semiparametric probit regression models. We present a simple computational procedure combining the expectation-maximization algorithm with the pool-adjacent-violators algorithm for solving the monotone constraint on the baseline function. Asymptotic properties of the maximum likelihood estimators are investigated, including the calculation of the explicit information bound for univariate current-status data, as well as the asymptotic consistency and convergence rate for bivariate current-status data. Extensive simulation studies showed that the proposed computational procedures performed well under small or moderate sample sizes. We demonstrate the estimation procedure with two real data examples in the areas of diabetic and HIV research.

Simple Choreographies of the Planar Newtonian N-Body Problem

In this paper, for the spatial Newtonian $2n$-body problem with equal masses, by proving the minimizers of the action functional under certain symmetric, topological and monotone constraints are collision-free, we found a family of spatial double choreographies, which have the common feature that half of the masses are circling around the $z$-axis clockwise along a spatial loop, while the motions of the other half masses are given by a rotation of the first half around the $x$-axis by $\pi$. Both loops are simple, without any self-intersection, and symmetric with respect to the $xz$-plane and $yz$-plane. The set of intersection points between the two loops is non-empty and contained in the $xy$-plane. The number of such double choreographies grows exponentially as $n$ goes to infinity.

A new method for interpolating in a convex subset of a Hilbert space

In this paper, interpolating curve or surface with linear inequality constraints is considered as a general convex optimization problem in a Reproducing Kernel Hilbert Space. We propose a new approximation method based on a discretized optimization problem in a finite-dimensional Hilbert space under the same set of constraints. We prove that the approximate solution converges uniformly to the optimal constrained interpolating function. An algorithm is derived and numerical examples with boundedness and mono-tonicity constraints in one and two dimensions are given.

RULEM: A novel heuristic rule learning approach for ordinal classification with monotonicity constraints

In many real world applications classification models are required to be in line with domain knowledge and to respect monotone relations between predictor variables and the target class, in order to be acceptable for implementation. This paper presents a novel heuristic approach, called RULEM, to induce monotone ordinal rule based classification models. The proposed approach can be applied in combination with any rule- or tree-based classification technique, since monotonicity is guaranteed in a post-processing step. RULEM checks whether a rule set or decision tree violates the imposed monotonicity constraints and existing violations are resolved by inducing a set of additional rules which enforce monotone classification. The approach is able to handle non-monotonic noise, and can be applied to both partially and totally monotone problems with an ordinal target variable. Two novel justifiability measures are introduced which are based on RULEM and allow to calculate the extent to which a classification model is in line with domain knowledge expressed in the form of monotonicity constraints. An extensive benchmarking experiment and subsequent statistical analysis of the results on 14 public data sets indicates that RULEM preserves the predictive power of a rule induction technique while guaranteeing monotone classification. On the other hand, the post-processed rule sets are found to be significantly larger which is due to the induction of additional rules. E.g., when combined with Ripper a median performance difference was observed in terms of PCC equal to zero and an average difference equal to −0.66%, with on average 5 rules added to the rule sets. The average and minimum justifiability of the original rule sets equal respectively 92.66% and 34.44% in terms of the RULEMF justifiability index, and 91.28% and 40.1% in terms of RULEMS, indicating the effective need for monotonizing the rule sets.

A Dual Active-Set Algorithm for Regularized Monotonic Regression

Monotonic (isotonic) regression is a powerful tool used for solving a wide range of important applied problems. One of its features, which poses a limitation on its use in some areas, is that it produces a piecewise constant fitted response. For smoothing the fitted response, we introduce a regularization term in the monotonic regression, formulated as a least distance problem with monotonicity constraints. The resulting smoothed monotonic regression is a convex quadratic optimization problem. We focus on the case, where the set of observations is completely (linearly) ordered. Our smoothed pool-adjacent-violators algorithm is designed for solving the regularized problem. It belongs to the class of dual active-set algorithms. We prove that it converges to the optimal solution in a finite number of iterations that does not exceed the problem size. One of its advantages is that the active set is progressively enlarging by including one or, typically, more constraints per iteration. This resulted in solving large-scale test problems in a few iterations, whereas the size of that problems was prohibitively too large for the conventional quadratic optimization solvers. Although the complexity of our algorithm grows quadratically with the problem size, we found its running time to grow almost linearly in our computational experiments.

Dealing with Monotonicity Constraints in Screening Problems: The Principal-Agent Model

This paper contains two parts clearly differentiated: a survey study about the use of monotonicity constraints in screening problems, and a research work extending the space of solutions in agency theory to non-smooth or even discontinuous action profiles. Specifically, we first systematize and compare, under a wide variety of assumptions on the endogenous variables of the model, existing approaches used for the derivation of necessary conditions in screening problems containing monotonicity constraints. Then, after identifying the most robust methods to generalize previous results, the paper derives a complete characterization of optimum contracts in the model of Jullien (2000), without resort to the usual continuity and smoothness assumptions on allocation functions. In this way, we conclude that most results in Jullien (2000) continue holding if, instead of the mentioned assumptions, the endogenous variables of the problem satisfy some weak regularity condition. Moreover, when the reservation utility fails to be type-dependent, as happens for instance in Guesnerie and Laffont (1978), the characterization of optimal contracts still holds without the need of imposing the referred regularity condition.

Bounded isotonic regression

Isotonic regression offers a flexible modeling approach under monotonicity assumptions, which are natural in many applications. Despite this attractive setting and extensive theoretical research, isotonic regression has enjoyed limited interest in practical modeling primarily due to its tendency to suffer significant overfitting, even in moderate dimension, as the monotonicity constraints do not offer sufficient complexity control. Here we propose to regularize isotonic regression by penalizing or constraining the range of the fitted model (i.e., the difference between the maximal and minimal predictions). We show that the optimal solution to this problem is obtained by constraining the non-penalized isotonic regression model to lie in the required range, and hence can be found easily given this non-penalized solution. This makes our approach applicable to large datasets and to generalized loss functions such as Huber’s loss or exponential family log-likelihoods. We also show how the problem can be reformulated as a Lasso problem in a very high dimensional basis of upper sets. Hence, range regularization inherits some of the statistical properties of Lasso, notably its degrees of freedom estimation. We demonstrate the favorable empirical performance of our approach compared to various relevant alternatives.

Bayesian quantile regression using random B-spline series prior

A Bayesian method for simultaneous quantile regression on a real variable is considered. By monotone transformation, the response variable and the predictor variable are transformed into the unit interval. A representation of quantile function is given by a convex combination of two monotone increasing functions ξ1 and ξ2 not depending on the prediction variables. In a Bayesian approach, a prior is put on quantile functions by putting prior distributions on ξ1 and ξ2. The monotonicity constraint on the curves ξ1 and ξ2 are obtained through a spline basis expansion with coefficients increasing and lying in the unit interval. A Dirichlet prior distribution is put on the spacings of the coefficient vector. A finite random series based on splines obeys the shape restrictions. The proposed method is extended to multidimensional predictors such that the quantile regression depends on the predictors through an unknown linear combination only. In the simulation study, the proposed approach is compared with a Bayesian method using Gaussian process prior through an extensive simulation study and some other Bayesian approaches proposed in the literature. An application to a data on hurricane activities in the Atlantic region is given. The proposed method is also applied on region-wise population data of USA for the period 1985–2010.

Wireless Max–Min Utility Fairness With General Monotonic Constraints by Perron–Frobenius Theory

This paper presents a systematic approach for solving wireless max–min utility fairness optimization problems in multiuser wireless networks with general monotonic constraints. These problems are often challenging to solve due to their nonconvexity. By establishing a connection between this class of optimization problems and the class of conditional eigenvalue problems that can be addressed by a generalized nonlinear Perron–Frobenius theory, we show how these problems can be solved optimally using an iterative algorithm that converges geometrically fast. The mathematical development in this paper unifies previous work and allows us to handle a broader class of competitive utility functions with general nonlinear monotonic constraints. Several representative applications illustrate the effectiveness of the proposed framework, including the max–min quality-of-service subject to robust interference temperature constraints in cognitive radio networks, the min–max weighted mean-square error subject to signal-to-interference-and-noise ratio constraints in multiuser downlink systems, the max–min throughput subject to nonlinear power constraints in energyefficient wireless networks, the max–min sigmoid utility in multimedia wireless networks, and the min–max outage probability subject to outage constraints in heterogeneous wireless networks. Numerical results are presented to demonstrate the fast-convergence behavior of the algorithms to the optimal fixed-point solution characterized by our generalized nonlinear Perron–Frobenius theoretic framework.

Monotonic classification extreme learning machine

Monotonic classification problems mean that both feature values and class labels are ordered and monotonicity relationships exist between some features and the decision label. Extreme Learning Machine (ELM) is a single-hidden layer feedforward neural network with fast training rate and good generalization capability, but due to the existence of training error, ELM cannot be directly used to handle monotonic classification problems. This work proposes a generalization of ELM for processing the monotonic classification, named as Monotonic Classification Extreme Learning Machine (MCELM) in which the monotonicity constraints are imposed to the original ELM model. Mathematically, MCELM is a quadratic programming problem in which the monotonicity relationships are considered as constraints and the training error is the objective to be minimized. The mathematical model of MCELM not only can make the generated classifier monotonic but also can minimize the classification error. MCELM does not need to tune parameters iteratively, and therefore, keeps the advantage of extremely fast training which is the essential characteristic of ELM. MCELM does not require that the monotonic relationships existing between features and the output are consistent, which essentially relaxes the assumption of consistent monotonicity used in most existing approaches to handling monotonic classification problems. In comparison with exiting approaches to handling monotonic classification, MCELM can indeed generate a monotonicity-reserving classifier which experimentally shows a much better generalization capability on both artificial and real world datasets.

Euler-Lagrange equation for a minimization problem over monotone transport maps

A variational time discretization for the compressible Euler equations has been introduced recently. It involves a minimization problem over the cone of monotone transport maps in each timestep. A matrix-valued measure field appears in the minimization as a Lagrange multiplier for the monotonicity constraint. We show that the absolutely continuous part of this measure field vanishes in the support of the density.

Semiparametric stochastic frontier models: A generalized additive model approach

The choice of the functional form of the frontier into a stochastic frontier model is typically neglected in applications and canonical functions are usually considered. This paper introduces a semiparametric approach for stochastic frontier estimation that extends previous works based on pseudo-likelihood estimators allowing flexibility in model selection and capability of imposing monotonicity and concavity constraints. For these purposes the present work introduces a generalized additive framework that moreover permits to model the influence of contextual/environmental factors to the hypothesized production process by the relative extension given by generalized additive models for location, scale and shape. Through some Monte Carlo simulations and an application to European agricultural data the flexibility of the proposed framework in analyzing efficiency is illustrated.

Multivariate decision trees with monotonicity constraints

Classification with monotonicity constraint is a fundamental task in social analysis, management and decision making, where a monotonic function guarantees that objects with better feature values are not assigned with worse decisions. In practice, an object may be better than another on some attributes, while worse on others. These objects are considered to be incomparable. Incomparable object pairs limit the performance of a monotone classifier. In this work, we design an algorithm to combat this issue by constructing multivariate decision trees with monotonicity constraints (MMT). The classification model is naturally deemed as an oblique tree as it discovers partitions via oblique hyperplane in the input space. Our algorithm generates the projections of the objects which are used to split the data by improved splitting criteria with rank mutual information (RMI) or rank Gini impurity (RGI). Moreover, an improved algorithm with L1-regularization is also proposed to compute the optimal subsets of features in the process of constructing the trees, which leads to a more compact tree. Experimental results show that the proposed algorithm improves the classification performance in monotone classification tasks. The proposed algorithm is also effective even if data is contaminated by non-monotonic noisy samples.

Partially linear single-index proportional hazards model with current status data

A partially linear single-index proportional hazards model with current status data is introduced, where the cumulative hazard function is assumed to be nonparametric and a nonlinear link function is assumed to take a parametric spline function. Efficient estimation and effective algorithm are established. Polynomial spline smoothing is invoked for the estimation of the cumulative baseline hazard function with monotonicity constraint on the functional, while a simultaneous sieve maximum likelihood (SML) estimation is proposed to estimate regression parameters. The proposed SML estimator for the parameter vector is shown to be asymptotically normal and semiparametric efficient. The spline estimator of the functional of the cumulative hazard function is shown to achieve the optimal nonparametric rate of convergence. A simulation study is conducted to examine the finite sample performance of the proposed estimators and algorithm, and an analysis of renal function recovery data is presented.

On ill-posedness of nonparametric instrumental variable regression with convexity constraints

This note shows that adding monotonicity or convexity constraints on the regression function does not restore well-posedness in nonparametric instrumental variable regression. The minimum distance problem without regularisation is still locally ill-posed.

Planning the Initial Motion of a Free Sliding/Rolling Ball

This paper investigates the motion of a free ball with initial velocity and angular velocity on a horizontal plane under friction. The ball will first slide along a parabolic trajectory (with its contact velocity in constant direction) and then roll along a straight trajectory. Curving of the sliding trajectory can be utilized for obstacle avoidance in path planning. A one-to-one correspondence exists between curved trajectories of the ball from one point to another and pairs of sliding and rolling directions that span a cone at the starting point to contain the ending point in the interior. This leads to a compact 2-D trajectory space, in which obstacles induce monotonic constraint curves. Based on the monotonicity, a plane sweep algorithm is described to find all collision-free trajectories. The work has several distinct features: handling of both sliding and rolling; planning in the configuration space of trajectories not that of the ball; computation of not just one collision-free trajectory but all such trajectories; and, finally, combining dynamics with computational geometry. The application of the work to spherical robots is discussed at the end.

An empirical study of Bayesian network parameter learning with monotonic influence constraints

Learning the conditional probability table (CPT) parameters of Bayesian networks (BNs) is a key challenge in real-world decision support applications, especially when there are limited data available. A conventional way to address this challenge is to introduce domain knowledge/expert judgments that are encoded as qualitative parameter constraints. In this paper we focus on a class of constraints which is naturally encoded in the edges of BNs with monotonic influences. Experimental results indicate that such monotonic influence constraints are widespread in practical BNs (all BNs used in the study contain such monotonic influences). To exploit expert knowledge about such constraints we have developed an improved constrained optimization algorithm, which achieves good parameter learning performance using these constraints, especially when data are limited. Specifically, this algorithm outperforms the previous state-of-the-art and is also robust to errors in labelling the monotonic influences. The method is applied to a real world medical decision support BN where we had access to expert-provided constraints and real hospital data. The results suggest that incorporating expert judgments about monotonic influence constraints can lead to more accurate BNs for decision support and risk analysis.

Uniform vs Differential Excise Taxes Under the Second-Order Approach

This paper solves the lack of a monotonicity constraint in optimal taxation models describing second-order conditions for incentive compatibility. We conclude that most findings obtained under the first-order approach, and among them notably Mirrlees’s results on Pareto efficient taxation and Corlett and Hague’s tax rule, may no longer hold beyond the assumption that the gradient of the utility function is affine with respect to ability. This reduces the desirability of differential excise taxes to a special case, while providing an argument for uniform excise taxes since, for the derivation under the second-order approach of the Atkinson and Stiglitz theorem, no further assumption is needed other than the usual weak separability of preferences.

Current prospects on ordinal and monotonic classification

Ordinal classification covers those classification tasks where the different labels show an ordering relation, which is related to the nature of the target variable. In addition, if a set of monotonicity constraints between independent and dependent variables has to be satisfied, then the problem is known as monotonic classification. Both issues are of great practical importance in machine learning. Ordinal classification has been widely studied in specialized literature, but monotonic classification has received relatively low attention. In this paper, we define and relate both tasks in a common framework, providing proper descriptions, characteristics, and a categorization of existing approaches in the state-of-the-art. Moreover, research challenges and open issues are discussed, with focus on frequent experimental behaviours and pitfalls, commonly used evaluation measures and the encouragement in devoting substantial research efforts in specific learning paradigms.

Estimating product-choice probabilities from recency and frequency of page views

This paper investigates the relationship between customers’ page views (PVs) and the probabilities of their product choices on e-commerce sites. For this purpose, we create a probability table consisting of product-choice probabilities for all recency and frequency combinations of each customers’ previous PVs. To reduce the estimation error when there are few training samples, we develop optimization models for estimating the product-choice probabilities that satisfy monotonicity, convexity and concavity constraints with respect to recency and frequency. Computational results demonstrate that our method has clear advantages over logistic regression and kernel-based support vector machine.