Iterative Deep Neighborhood: A Deep Learning Model Which Involves Both Input Data Points and Their Neighbors.

Rong Liu,Jing-Yan Wang,Yonggang Yan,Yan Liu

doi:10.1155/2020/9868017

Rong Liu, Jing-Yan Wang + Show 2 more

Open Access

https://doi.org/10.1155/2020/9868017

Copy DOI

Abstract

Deep learning models, such as deep convolutional neural network and deep long-short term memory model, have achieved great successes in many pattern classification applications over shadow machine learning models with hand-crafted features. The main reason is the ability of deep learning models to automatically extract hierarchical features from massive data by multiple layers of neurons. However, in many other situations, existing deep learning models still cannot gain satisfying results due to the limitation of the inputs of models. The existing deep learning models only take the data instances of an input point but completely ignore the other data points in the dataset, which potentially provides critical insight for the classification of the given input. To overcome this gap, in this paper, we show that the neighboring data points besides the input data point itself can boost the deep learning model's performance significantly and design a novel deep learning model which takes both the data instances of an input point and its neighbors' classification responses as inputs. In addition, we develop an iterative algorithm which updates the neighbors of data points according to the deep representations output by the deep learning model and the parameters of the deep learning model alternately. The proposed algorithm, named “Iterative Deep Neighborhood (IDN),” shows its advantages over the state-of-the-art deep learning models over tasks of image classification, text sentiment analysis, property price trend prediction, etc.

Highlights

An abstractive feature can be generated from the highlevel layers. e deep learning model can release the problem feature engineering by learning effective features automatically. e feature engineering process of traditional machine learning models relies on the domain knowledge of feature engineers heavily, and it is time consuming and the designed features cannot be generated to other domains
Compared to the traditional machine learning and feature engineering process, the deep learning can automatically find the features which are relevant to the learning problem and store the features in the neural units of multiple layers
Our contributions are given as three parts: (i) Firstly, we proposed a new deep learning model for pattern classification problem. e key difference between our model and the traditional deep learning model is the input structure. e traditional deep learning model only takes the input instances of an input data point, but our model can take both the input data point and its neighboring points, to be specific, the classification responses of the neighbors, as the inputs of the model

Summary

Introduction

A deep learning model usually has more than three layers, and by using multiple layers, the model extracts hierarchical features from the original data. In this way, an abstractive feature can be generated from the highlevel layers. E deep learning model can release the problem feature engineering by learning effective features automatically. E feature engineering process of traditional machine learning models relies on the domain knowledge of feature engineers heavily, and it is time consuming and the designed features cannot be generated to other domains. Compared to the traditional machine learning and feature engineering process, the deep learning can automatically find the features which are relevant to the learning problem and store the features in the neural units of multiple layers.

Methods

Results

Conclusion