The heterogeneous networks (HetNets) performance is highly dependent on the interference coordination among cells since both macrocells and small cells share the same spectrum. This paper focuses on the configuration of eICIC parameters: almost blank subframe ratio and cell range expansion bias defined for LTE-A. We propose an online learning mechanism based on a big data-driven framework and multiarmed bandit algorithms that retrieves data from the network to learn efficient configurations of eICIC parameters without requiring any previous knowledge about the network (e.g., traffic load, topology, scheduling algorithm). Our numerical results show that our approach attains a significant improvement with respect to the state of the art of online learning algorithms in networks under stationary and variable conditions (e.g., number of small cells, traffic load).