Multi‐instance multi‐label learning of natural scene images: via sparse coding and multi‐layer neural network

Abstract

The classification of natural scene images is multi‐instance multi‐label (MIML) for many labels that exist in a natural scene image. The traditional method of solving MIML is to degenerate it into single‐instance single‐label learning (SISL). However, the precision of the method could decrease due to information loss during the degeneration process. How to reasonably solve the MIML problem is key to obtaining high accuracy in this research area. An MIML algorithm based on instances via combining sparse coding with a deep neural network is proposed. First, an instance‐based sparse representation with dictionary learning is adopted. Second, an MIML description model based on a deep network is proposed, which can realise parameter self‐learning in combination with sparse representations. Third, the residuals of the sparse representation are introduced to the deep neural network. The results of the experiments show that the method outperforms a number of state‐of‐the‐art approaches.