Convex Optimization for Group Feature Selection in Networked Data

Abstract

Feature selection is at the heart of machine learning, and it is effective at facilitating data interpretability and improving prediction performance by defying the curse of dimensionality. Group feature selection is often used to reveal relationships in structured data and provide better predictive power compared with the standard feature selection methods without consideration of the grouped structure. We study a group feature selection problem in networked data in which edge weights are considered as features, while each node in the network is regarded as a group feature. This problem is particularly useful in feature selection for neuroimaging data, where the data are high dimensional and the intrinsic networked structure among the features (i.e., connectivities between regions) in brain data has to be captured properly. We propose a mathematical model based on the support vector machines (SVM), which entails the ℓ0norm regularization to restrict the number of nodes (i.e., groups). To cope with the computational challenge of the ℓ0norm regularization, we develop a convex relaxation reformulation of the proposed model as a convex semiinfinite programming (SIP). We then introduce a new iterative algorithm that achieves an optimal solution for this convex SIP. Experimental results for synthetic and real brain network data sets show that our approach gives better predictive performance compared with the state-of-the-art group feature selection and the standard feature selection methods. Our technique additionally yields a sparse subnetwork solution that is easier to interpret than those obtained by other methods.