Deep Isotonic Embedding Network: A flexible Monotonic Neural Network

Abstract

Guaranteeing the monotonicity of a learned model is crucial to address concerns such as fairness, interpretability, and generalization. This paper develops a new monotonic neural network named Deep Isotonic Embedding Network (DIEN), which uses different modules to deal with monotonic and non-monotonic features respectively, and then combine outputs of these modules linearly to obtain the prediction result. A new embedding tool called Isotonic Embedding Unit is developed to process monotonic features and turn each one into an isotonic embedding vector. By converting non-monotonic features into a series of non-negative weight vectors and then combining them with isotonic embedding vectors that have special properties, we enable DIEN to guarantee monotonicity. Besides, we also introduce a module named Monotonic Feature Learning Network to capture complex dependencies between monotonic features. This module is a monotonic feedforward neural network with non-negative weights and can handle scenarios where there are few non-monotonic features or only monotonic features. In comparison to existing methods, DIEN does not require intricate structures like lattices or the use of additional verification techniques to ensure monotonicity. Additionally, the relationship between DIEN’s inputs and outputs is obvious and intuitive. Results from experiments on both synthetic and real-world datasets demonstrate DIEN’s superiority over existing methodologies.