A novel stochastic configuration network with iterative learning using privileged information and its application

Abstract

In the real-world applications, some auxiliary information is commonly available in addition to the standard data, which is often ignored by the traditional learning algorithms. To effectively utilize the auxiliary information for assisting in building learner models with improved performance, this paper presents a novel stochastic configuration network (SCN) incorporating the iterative learning using privileged information (LUPI) paradigm, termed ISCN+. Meanwhile, to make the comments generated by the privileged information more closely match the ISCN+, these comments should be iteratively updated as the number of hidden nodes of the SCN increases. To this end, the training of SCN and the learning of comments generated by the privileged information is integrated into a new objective function, and the alternating optimization strategy is adopted to optimize the parameters of ISCN + and update the comments generated by the privileged information. Moreover, the L1/2-norm-regularization-based sparse ISCN+ (S-ISCN+) is proposed to further improve the generalization capacity and reduce the complexity of ISCN+. Furthermore, the convergence analysis of the optimization process is provided. The experimental results on two benchmark data sets and a real-world data set demonstrate the effectiveness of the proposed method.