Unsupervised Deep Learning Algorithm Research Articles

3D MR image synthesis using unsupervised deep learning algorithm in MS patients (P5.2-025)

3D MR image synthesis using unsupervised deep learning algorithm in MS patients (P5.2-025)

Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations

Deep Learning (DL) aims at learning the \emph{meaningful representations}. A meaningful representation refers to the one that gives rise to significant performance improvement of associated Machine Learning (ML) tasks by replacing the raw data as the input. However, optimal architecture design and model parameter estimation in DL algorithms are widely considered to be intractable. Evolutionary algorithms are much preferable for complex and non-convex problems due to its inherent characteristics of gradient-free and insensitivity to local optimum. In this paper, we propose a computationally economical algorithm for evolving \emph{unsupervised deep neural networks} to efficiently learn \emph{meaningful representations}, which is very suitable in the current Big Data era where sufficient labeled data for training is often expensive to acquire. In the proposed algorithm, finding an appropriate architecture and the initialized parameter values for a ML task at hand is modeled by one computational efficient gene encoding approach, which is employed to effectively model the task with a large number of parameters. In addition, a local search strategy is incorporated to facilitate the exploitation search for further improving the performance. Furthermore, a small proportion labeled data is utilized during evolution search to guarantee the learnt representations to be meaningful. The performance of the proposed algorithm has been thoroughly investigated over classification tasks. Specifically, error classification rate on MNIST with $1.15\%$ is reached by the proposed algorithm consistently, which is a very promising result against state-of-the-art unsupervised DL algorithms.

Sparse Coding Guided Spatiotemporal Feature Learning for Abnormal Event Detection in Large Videos

Abnormal event detection in large videos is an important task in research and industrial applications, which has attracted considerable attention in recent years. Existing methods usually solve this problem by extracting local features and then learning an outlier detection model on training videos. However, most previous approaches merely employ hand-crafted visual features, which is a clear disadvantage due to their limited representation capacity. In this paper, we present a novel unsupervised deep feature learning algorithm for the abnormal event detection problem. To exploit the spatiotemporal information of the inputs, we utilize the deep three-dimensional convolutional network (C3D) to perform feature extraction. Then, the key problem is how to train the C3D network without any category labels. Here, we employ the sparse coding results of the hand-crafted features generated from the inputs to guide the unsupervised feature learning. Specifically, we define a multilevel similarity relationship between these inputs according to the statistical information of the shared atoms. In the following, we introduce the quadruplet concept to model the multilevel similarity structure, which could be used to construct a generalized triplet loss for training the C3D network. Furthermore, the C3D network could be utilized to generate the features for sparse coding again, and this pipeline could be iterated for several times. By jointly optimizing between the sparse coding and the unsupervised feature learning, we can obtain robust and rich feature representations. Based on the learned representations, the sparse reconstruction error is applied to predicting the anomaly score of each testing input. Experiments on several publicly available video surveillance datasets in comparison with a number of existing works demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods.

Community discovery in networks with deep sparse filtering

In the past decade, network community discovery has attracted great attention from quite a few researchers, and community structure is one of the most significant properties in complex networks. This paper presents a novel method for network community discovery based on deep sparse filtering. The features of the network are extracted by sparse filtering, an unsupervised deep learning algorithm, from an efficient representation of the network. Consequently, extracted features are employed to partition the network. Experiment results on both synthetic and real-world network datasets indicate that the proposed algorithm especially based on S⌀rensen–Dice’s similarity matrix representation of the network is efficient and it outperforms several state-of-art algorithms in discovering community structure.

A novel relocation method for simultaneous localization and mapping based on deep learning algorithm

Relocation is one of the most common problems in Simultaneous Localization and Mapping (SLAM). This paper presents a novel relocation method, using unsupervised deep learning algorithm to extract the feature of Light Detection and Ranging (LiDAR) data, and narrows the scope of relocation by classifying these features to reduce the randomness of the relocation. Compared with the other methods which is based on matching the manual feature points, this method avoids some limitations of manual features. We modify the Particle Filter SLAM (PF-SLAM), and use our relocation method to replace the original method for experimentation. The experimental results demonstrate that this method can be relocation whit high success rate only use a small amount of computational resource in a short time. Training neural network will consume a lot of computing resources, we also propose a cloud computing framework to the implementation of this method for the mobile robot which computational resources are limited.

Histopathological Image Classification With Color Pattern Random Binary Hashing-Based PCANet and Matrix-Form Classifier.

The computer-aided diagnosis for histopathological images has attracted considerable attention. Principal component analysis network (PCANet) is a novel deep learning algorithm for feature learning with the simple network architecture and parameters. In this study, a color pattern random binary hashing-based PCANet (C-RBH-PCANet) algorithm is proposed to learn an effective feature representation from color histopathological images. The color norm pattern and angular pattern are extracted from the principal component images of R, G, and B color channels after cascaded PCA networks. The random binary encoding is then performed on both color norm pattern images and angular pattern images to generate multiple binary images. Moreover, we rearrange the pooled local histogram features by spatial pyramid pooling to a matrix-form for reducing the dimension of feature and preserving spatial information. Therefore, a C-RBH-PCANet and matrix-form classifier-based feature learning and classification framework is proposed for diagnosis of color histopathological images. The experimental results on three color histopathological image datasets show that the proposed C-RBH-PCANet algorithm is superior to the original PCANet and other conventional unsupervised deep learning algorithms, while the best performance is achieved by the proposed feature learning and classification framework that combines C-RBH-PCANet and matrix-form classifier.

Biclustering of Omics Data using Rectified Factor Networks

Biclustering has effectively been employed in biological sciences and e-commerce for medication design and recommender systems, respectively, and has become a prominent technique for evaluating big datasets presented as a matrix of samples times attributes. One of the most successful biclustering methods, Factor Analysis for Bicluster Acquisition (FABIA), is a generative model in which each bicluster is represented by two sparse membership vectors: one for the samples and one for the features. Due to the high computational complexity of computing the posterior, FABIA is limited to approximately 20 code units. Additionally, code units are not always sufficiently decorrelated, making sample membership difficult to determine. To circumvent the limitations of existing biclustering approaches, we propose using the recently introduced unsupervised Deep Learning algorithm Rectified Factor Networks (RFNs). RFNs use their posterior means to efficiently build exceedingly sparse, non-linear, high-dimensional representations of the input. RFN learning is a generalized alternating minimization approach that ensures non-negative and normalized posterior means and is based on the posterior regularization method. Each code unit represents a bicluster, consisting of samples for which the coding unit is active and features for which the code unit has activating weights. RFN beat 13 other biclustering algorithms, including FABIA, on four hundred benchmark datasets and three gene expression datasets with identified clusters. RFN was able to detect DNA sequences that imply interbreeding with other hominins that began before modern humans' ancestors left Africa, based on data from the 1000 Genomes Project.