Solving Partially Monotone Problems with Neural Networks

Abstract

In many applications, it is a priori known that the target function should satisfy certain constraints imposed by, for example, economic theory or a human-decision maker. Here we consider partially monotone problems, where the target variable depends monotonically on some of the predictor variables but not all. We propose an approach to build partially monotone models based on the convolution of monotone neural networks and kernel functions. The results from simulations and a real case study on house pricing show that our approach has significantly better performance than partially monotone linear models. Furthermore, the incorporation of partial monotonicity constraints not only leads to models that are in accordance with the decision maker's expertise, but also reduces considerably the model variance in comparison to standard neural networks with weight decay. However, the main assumption for the implementation of most of these methods is that the function (output) being estimated should be monotone in all inputs (so-called total monotonicity). This in practice, of course, is not always the case. In this paper we consider partially monotone regression problems, where we assume that the dependent variable depends monotonically on some of the independent variables but not all. For example, common sense suggests that the house price has monotone increasing dependence on the number of rooms and the total house area, whereas for the number of floors this dependence does not necessarily hold. Such prior knowledge about monotone relationships can be incorporated as constraints in data mining algorithms in order to improve the accuracy and interpretability of the models derived as well as to reduce their variance on new data. The paper is organized as follows. In the next section we introduce notations and definitions related to monotonicity, which are needed for the follow-up discussion. The main contribution of this paper is the approach for partial monotonicity presented in Sect.IVA. The approach is based on the convolution of a special type of monotone neural networks, introduced in Sect.III, and kernel functions. In Sect.IVB we present the design and the results from simulation studies carried out to test the performance of the proposed approach for partial monotonicity. Sect.IVC demonstrates the application of the approach on a real case study of house pricing. Concluding remarks are given in Sect.V.