A Novel Method Based on Line-Segment Visualizations for Hyper-Parameter Optimization in Deep Networks

Abstract

Recently, deep learning has been widely applied in various areas and achieved remarkable research findings. The major reason that makes the deep learning paradigm successful is that it can effectively learn a hierarchical feature structure for the training data. However, most deep learning algorithms rely on massive well-labeled training datasets and hyper-parameter configurations. This paper proposed a novel methodology that uses the geometric characteristics of line-segment representations to optimize the hyper-parameters for the deep networks. The methodology is applied to a line-segment-based stacked auto-encoder to verify its effectiveness. It is found that the line-segment-based visualizations can increase the interpretability of the deep models and facilitate the configurations for the hyper-parameters.