A Simultaneous Training and Input Selection Algorithm for Classification Problems Using Piecewise Approximations

Abstract

Heart disease diagnosis using few measurements is a challenging an important task considering the increasing population. Artificial Neural Networks (ANNs) are promising mathematical architectures once the training is performed in an elegant manner to avoid theoretical challenges related to high nonlinearity, nonconvexity using few input variables to ensure generalization capability. This study shows the impact of the piecewise linear approximation of nonlinear functions in ANN architecture and training problem to benefit from the mixed integer linear problem formulation for the simultaneous input selection and training to obtain mixed integer programming based ANN (MIP-ANN). Proposed formulation is further tailored through linking constraints to remove the connections from the eliminated inputs to favor parameter identifiability. A publicly available dataset is considered as a case study of whose results are also compared to traditional ANN with all inputs (FC-ANN) and a relatively more straightforward but common input selection method (SKB-ANN). The results provide a comparable performance despite significant reduction in the input space in addition to significant computational and theoretical advantages thanks to advanced formulation.