Labeled Data Points Research Articles

Seeded random walk for multi-view semi-supervised classification

Recently, multi-view learning has captured widespread attention in the machine learning area, yet it is still crucial and challenging to exploit beneficial patterns from multi-view data. Specifically, very limited work has been devoted to multi-view semi-supervised learning, where only a small number of labeled data points are available for model training. Therefore, a simple yet efficient seeded random walk scheme is proposed in this paper to address the multi-view semi-supervised classification problem, where known labeled data points serve as random seeds to be walked with certain probability. In this scheme, the semi-supervised classification indicator is obtained based primarily on an arrival probability and a reward matrix, which are computed by leveraging an initial distribution from some random seeds. Besides, theoretical analyses are then provided to indicate a connection of the proposed model with the existing manifold ranking method. Finally, comprehensive experiments on eight publicly available data sets demonstrate the superiority of the proposed model against compared state-of-the-art semi-supervised methods and fully supervised classifiers. Furthermore, experimental results also suggest that the proposed method comes with positive robustness and promising generalization capability in terms of data classification.

Accelerated manifold embedding for multi-view semi-supervised classification

Multi-view semi-supervised learning has gained much attention since a great number of unlabeled multi-view data are easy to obtain while few labeled data are available. Accordingly, how to utilize the relationship between labeled and unlabeled data is of significance in multi-view semi-supervised learning. In this paper, we propose an auto-weighted manifold embedding model to address multi-view semi-supervised classification problems, where only a small percentage of labeled data points are used for model training. In the proposed model, data points close to each other in the feature space will be assigned to similar classes in the label space through manifold embedding. Accordingly, the class information of labeled data will be employed in the prediction process for unlabeled data. Moreover, an optimal weight for each view of multi-view data is learned automatically, which enables an encouraging fusion quality for multi-view manifold embedding. To further speed up the calculation process and reduce the computational complexity of the proposed model, an effective accelerated auto-weighted manifold embedding scheme is developed. Besides, theoretical analyses are then provided to indicate a tight approximation bound for the primary manifold embedding method, while the accelerated method is designed by an order decrease of magnitude of computational complexity. Finally, comprehensive experiments on eight publicly available data sets demonstrate the superiority of the proposed models over compared state-of-the-art semi-supervised methods and fully supervised classifiers. Furthermore, the experimental results are suggestive of positive robustness and promising generalization capacity of the proposed methods.

Reducing the burden of data collection in a fingerprinting-based VLP system using a hybrid of improved co-training semi-supervised regression and adaptive boosting algorithms

The positioning accuracy of the visible light-based fingerprinting technique depends greatly on the number of initially collected and labeled data points. The greater is the number of initial fingerprints collected, the higher is the achieved accuracy of the system. However, collecting many labeled fingerprints while simultaneously ensuring data reliability is quite time consuming. To increase the practicality and applicability of fingerprinting-based algorithms, we combine the improved co-training semi-supervised regression and adaptive boosting (Ada-XCoReg) algorithms. Ada-XCoReg is a combination of double-weighted k-nearest neighbors, cross-correlation, and an adaptive boosting technique that increases the amount of input data and enhances positioning accuracy based on randomly unlabeled data points. By applying the proposed solution to the experimental model, we proved that our approach achieved a positioning accuracy of 6.14 cm, 5.29 cm, and 2.51 cm when the numbers of initially-labeled fingerprints were 16, 25, and 49, respectively.

Using Convolutional Neural Network with Cheat Sheet and Data Augmentation to Detect Breast Cancer in Mammograms.

The American Cancer Society expected to diagnose 276,480 new cases of invasive breast cancer in the USA and 48,530 new cases of noninvasive breast cancer among women in 2020. Early detection of breast cancer, followed by appropriate treatment, can reduce the risk of death from this disease. DL through CNN can assist imaging specialists in classifying the mammograms accurately. Accurate classification of mammograms using CNN needs a well-trained CNN by a large number of labeled mammograms. Unfortunately, a large number of labeled mammograms are not always available. In this study, a novel procedure to aid imaging specialists in detecting normal and abnormal mammograms has been proposed. The procedure supplied the designed CNN with a cheat sheet for some classical attributes extracted from the ROI and an extra number of labeled mammograms through data augmentation. The cheat sheet aided the CNN through encoding easy-to-recognize artificial patterns in the mammogram before passing it to the CNN, and the data augmentation supported the CNN with more labeled data points. Fifteen runs of 4 different modified datasets taken from the MIAS dataset were conducted and analyzed. The results showed that the cheat sheet, along with data augmentation, enhanced CNN's accuracy by at least 12.2% and enhanced the precision of the CNN by at least 2.2. The mean accuracy, sensitivity, and specificity obtained using the proposed procedure were 92.1, 91.4, and 96.8, respectively, while the average area under the ROC curve was 94.9.

Automated Curation of CNMF-E-Extracted ROI Spatial Footprints and Calcium Traces Using Open-Source AutoML Tools.

In vivo 1-photon (1p) calcium imaging is an increasingly prevalent method in behavioral neuroscience. Numerous analysis pipelines have been developed to improve the reliability and scalability of pre-processing and ROI extraction for these large calcium imaging datasets. Despite these advancements in pre-processing methods, manual curation of the extracted spatial footprints and calcium traces of neurons remains important for quality control. Here, we propose an additional semi-automated curation step for sorting spatial footprints and calcium traces from putative neurons extracted using the popular constrained non-negative matrixfactorization for microendoscopic data (CNMF-E) algorithm. We used the automated machine learning (AutoML) tools TPOT and AutoSklearn to generate classifiers to curate the extracted ROIs trained on a subset of human-labeled data. AutoSklearn produced the best performing classifier, achieving an F1 score >92% on the ground truth test dataset. This automated approach is a useful strategy for filtering ROIs with relatively few labeled data points and can be easily added to pre-existing pipelines currently using CNMF-E for ROI extraction.

Extreme Learning Machine-Based Ensemble Transfer Learning for Hyperspectral Image Classification

Although the extreme learning machine (ELM) has been successfully applied to hyperspectral image (HSI) classification, the development of the ELM is restricted by insufficient training data. In this article, we propose a novel extreme learning machine-based ensemble transfer learning algorithm for hyperspectral image classification named TL-ELM. TL-ELM not only retains the input weights and hidden biases of the ELM learned from the target domain, but also utilizes instances in the source domain to iteratively adjust the output weights of the ELM, which are used as the weights of the training models, and then ensembles the training models with their weights for the final classification. In experiments, we choose different regions in northern Italy, namely, Pavia University and Pavia Centre, as the source dataset and target dataset, respectively, and through a comparison with other transfer learning algorithms, we demonstrate that our proposed TL-ELM algorithm is superior on HSI classification tasks with only a few labeled data points in the target domain. Furthermore, we set Pavia University as the source dataset and Pavia Centre as the target dataset to demonstrate that our proposed method can effectively transfer useful instances between different HSIs.

DCBT-Net: Training Deep Convolutional Neural Networks With Extremely Noisy Labels

Obtaining data with correct labels is crucial to attain the state-of-the-art performance of Convolutional Neural Network (CNN) models. However, labeling datasets is significantly time-consuming and expensive process because it requires expert knowledge in a particular domain. Therefore, real-life datasets often exhibit incorrect labels due to the involvement of nonexperts in the data-labeling process. Consequently, there are many cases of incorrectly labeled data in the wild. Although the issue of poorly labeled datasets has been studied, the existing methods are complex and difficult to reproduce. Thus, in this study, we proposed a simpler algorithm called “Deep Clean Before Training Net” (DCBT-Net) that is based on cleaning wrongly labeled data points using the information from eigenvalues of the Laplacian matrix obtained from similarities between the data samples. The cleaned data were trained using deep CNN (DCNN) to attain the state-of-the-art results. This system achieved better performance than the existing approaches. In conducted experiments, the performance of the DCBT-Net was tested on three commercially available datasets, namely, Modified National Institute of Standards and Technology (MNIST) database of handwritten digits, Canadian Institute for Advanced Research (CIFAR) and WebVision1000 datasets. The proposed method achieved better results when assessed using several evaluation metrics compared with the existing state-of-the-art methods. Specifically, the DCBT-Net attained an average 15%, 20%, and 3% increase in accuracy score using MNIST database, CIFAR-10 dataset, and WebVision dataset, respectively. Also, the proposed approach demonstrated better results in specificity, sensitivity, positive predictive value, and negative predictive value evaluation metrics.

Optimization-Oriented RAW Modeling of IEEE 802.11ah Heterogeneous Networks

The new medium access method of IEEE 802.11ah, called restricted access window (RAW), divides stations into different groups, and only allows stations in the same group to access the channel simultaneously, in order to reduce collisions and thus achieve better performance (e.g., throughput). However, the existing station grouping strategies only support homogeneous scenarios where all stations use the same modulation and coding scheme (MCS) and packet size. A surrogate model is an efficient mathematical model that represents the behavior of a complex system, trained with a limited set of labeled input–output data samples. In this article, we present a surrogate model that can accurately predict RAW performance under a given RAW configuration in heterogeneous networks. Different from the homogeneous scenario, heterogeneous networks are defined by a large number of parameters, leading to an enormous design space, i.e., the order of 1017 possible data points. This is too big to achieve feasible training convergence. In this article, we present a novel training methodology that leads to a new design space with highly reduced size, i.e., the order of 105 data points. The surrogate model converges when less than 6000 labeled data points are used for training, which is only a tiny portion of the whole design space. The results show that, the relative error between model prediction and simulation results is less than 0.1 for 95% of the data points, in the areas of the design space studied. Its low complexity and high precision make the proposed model a valuable tool to develop real-time RAW optimization algorithms for heterogeneous IEEE 802.11ah networks.

Semi-Supervised Topological Analysis for Elucidating Hidden Structures in High-Dimensional Transcriptome Datasets.

Topological data analysis (TDA) is a powerful method for reducing data dimensionality, mining underlying data relationships, and intuitively representing the data structure. The Mapper algorithm is one such tool that projects high-dimensional data to 1-dimensional space by using a filter function that is subsequently used to reconstruct the data topology relationships. However, domain context information and prior knowledge have not been considered in current TDA modeling frameworks. Here, we report the development and evaluation of a semi-supervised topological analysis (STA) framework that incorporates discrete or continuously labeled data points and selects the most relevant filter functions accordingly. We validate the proposed STA framework with simulation data and then apply it to samples from Genotype-Tissue Expression data and ovarian cancer transcriptome datasets. The graphs generated by STA for these 2 datasets, based on gene expression profiles, are consistent with prior knowledge, thereby supporting the effectiveness of the proposed framework.

Localized Linear Regression in Networked Data

The network Lasso (nLasso) has been proposed recently as an efficient learning algorithm for massive networked data sets (big data over networks). It extends the well-known least absolute shrinkage and selection operator (Lasso) from learning sparse (generalized) linear models to network models. Efficient implementations of the nLasso have been obtained using convex optimization methods lending to scalable message passing protocols. In this paper, we analyze the statistical properties of nLasso when applied to localized linear regression problems involving networked data. Our main result is a sufficient condition on the network structure and available label information such that nLasso accurately learns a localized linear regression model from a few labeled data points. We also provide an implementation of nLasso for localized linear regression by specializing a primaldual method for solving the convex (non-smooth) nLasso problem.

An Overview of Unsupervised Deep Feature Representation for Text Categorization

High-dimensional features are extensively accessible in machine learning and computer vision areas. How to learn an efficient feature representation for specific learning tasks is invariably a crucial issue. Due to the absence of class label information, unsupervised feature representation is exceedingly challenging. In the last decade, deep learning has captured growing attention from researchers in a broad range of areas. Most of the deep learning methods are supervised, which is required to be fed with a large amount of accurately labeled data points. Nevertheless, acquiring sufficient accurately labeled data is unaffordable in numerous real-world applications, which is suggestive of the needs of unsupervised learning. Toward this end, quite a few unsupervised feature representation approaches based on deep learning have been proposed in recent years. In this paper, we attempt to provide a comprehensive overview of unsupervised deep learning methods and compare their performances in text categorization. Our survey starts with the autoencoder and its representative variants, including sparse autoencoder, stacked autoencoder, contractive autoencoder, denoising autoencoder, variational autoencoder, graph autoencoder, convolutional autoencoder, adversarial autoencoder, and residual autoencoder. Aside from autoencoders, deconvolutional networks, restricted Boltzmann machines, and deep belief nets are introduced. Then, the reviewed unsupervised feature representation methods are compared in terms of text clustering. Extensive experiments in eight publicly available data sets of text documents are conducted to provide a fair test bed for the compared methods.

On Convergence of the Class Membership Estimator in Fuzzy $k$-Nearest Neighbor Classifier

The fuzzy $k$ -nearest neighbor classifier (F $k$ NN) improves upon the flexibility of the $k$ -nearest neighbor classifier by considering each class as a fuzzy set and estimating the membership of an unlabeled data instance for each of the classes. However, the question of validating the quality of the class memberships estimated by F $k$ NN for a regular multiclass classification problem still remains mostly unanswered. In this paper, we attempt to address this issue by first proposing a novel direction of evaluating a fuzzy classifier by highlighting the importance of focusing on the class memberships estimated by F $k$ NN instead of its misclassification error. This leads us to finding novel theoretical upper bounds, respectively, on the bias and the mean squared error of the class memberships estimated by F $k$ NN. Additionally the proposed upper bounds are shown to converge toward zero with increasing availability of the labeled data points, under some elementary assumptions on the class distribution and membership function. The major advantages of this analysis are its simplicity, capability of a direct extension for multiclass problems, parameter independence, and practical implication in explaining the behavior of F $k$ NN in diverse situations (such as in presence of class imbalance). Furthermore, we provide a detailed simulation study on artificial and real data sets to empirically support our claims.

Semisupervised Discriminant Multimanifold Analysis for Action Recognition.

Although recent semisupervised approaches have proven their effectiveness when there are limited training data, they assume that the samples from different actions lie on a single data manifold in the feature space and try to uncover a common subspace for all samples. However, this assumption ignores the intraclass compactness and the interclass separability simultaneously. We believe that human actions should occupy multimanifold subspace and, therefore, model the samples of the same action as the same manifold and those of different actions as different manifolds. In order to obtain the optimum subspace projection matrix, the current approaches may be mathematically imprecise owe to the badly scaled matrix and improper convergence. To address these issues in unconstrained convex optimization, we introduce a nontrivial spectral projected gradient method and Karush-Kuhn-Tucker conditions without matrix inversion. Through maximizing the separability between different classes by using labeled data points and estimating the intrinsic geometric structure of the data distributions by exploring unlabeled data points, the proposed algorithm can learn global and local consistency and boost the recognition performance. Extensive experiments conducted on the realistic video data sets, including JHMDB, HMDB51, UCF50, and UCF101, have demonstrated that our algorithm outperforms the compared algorithms, including deep learning approach when there are only a few labeled samples.

Data labeling method based on Cluster similarity using Rough Entropy for Categorical Data Clustering

In present research, Data mining is become one of the growing area which deals with data. Clustering is recognized as an efficient methodology in data grouping; to improve the efficiency of the clustering many researchers have used data labeling method. Labeling method works on similar data points, into the proper clusters. In categorical domain applying data labeling is not so easy when compare with numerical domain. In numeral domain it is easy to find difference between to data points, but in categorical it is not easy. Since data labeling on categorical is a challenging issue till date and it is quite complex to implement. The proposed methodology is deals on this problem. According proposed method a sample data will be taken. That sampled data further divides sliding windows, and then a normal clustering algorithm will be applied on one sliding window and divides into clusters. Rough membership Entropy function is used to find the similarity between unlabelled data points to labeled data points. The proposed methodology has two important features those are 1) The Data points will moved into their proper clusters, means the quality clusters will take places, 2) Proposed methodology will execute with high efficiency rate. In this paper the proposed methodology is applied on KDD Cup99 data sets, and the results shows appreciably more proficient than earlier works.

A deep learning approach for pose estimation from volumetric OCT data.

Tracking the pose of instruments is a central problem in image-guided surgery. For microscopic scenarios, optical coherence tomography (OCT) is increasingly used as an imaging modality. OCT is suitable for accurate pose estimation due to its micrometer range resolution and volumetric field of view. However, OCT image processing is challenging due to speckle noise and reflection artifacts in addition to the images' 3D nature. We address pose estimation from OCT volume data with a new deep learning-based tracking framework. For this purpose, we design a new 3D convolutional neural network (CNN) architecture to directly predict the 6D pose of a small marker geometry from OCT volumes. We use a hexapod robot to automatically acquire labeled data points which we use to train 3D CNN architectures for multi-output regression. We use this setup to provide an in-depth analysis on deep learning-based pose estimation from volumes. Specifically, we demonstrate that exploiting volume information for pose estimation yields higher accuracy than relying on 2D representations with depth information. Supporting this observation, we provide quantitative and qualitative results that 3D CNNs effectively exploit the depth structure of marker objects. Regarding the deep learning aspect, we present efficient design principles for 3D CNNs, making use of insights from the 2D deep learning community. In particular, we present Inception3D as a new architecture which performs best for our application. We show that our deep learning approach reaches errors at our ground-truth label's resolution. We achieve a mean average error of 14.89 ± 9.3µm and 0.096 ± 0.072° for position and orientation learning, respectively.

Incremental Graph Embedding Based on Spatial-Spectral Neighbors for Hyperspectral Image Classification

Recently, graph embedding-based methods have been developed in dimensionality reduction (DR) and classification of hyperspectral image (HSI). The key step for graph embedding methods is the construction of graph. The commonly used method is to manually choose nearest neighbors and then, compute edge weights using the spectral feature. However, the adjacency graph is inappropriate due to the negligence of spatial information. What is more, the construction of graph only takes training samples or $k$ nearest neighbors into account, which may lead to unsuitable graph representation. In this paper, we propose a novel incremental graph embedding (IGE) algorithm to construct a spatial-spectral neighbor graph for the HSI classification. The IGE can spread the discrimination information contained in training samples to their neighbors until each testing sample has a pseudo label. The spatial affinity weights between unlabeled data points and their labeled neighbors are calculated according to the construction strategies of the spatial-spectral neighbor graph. The pseudo label of the unlabeled data point is determined based on the maximum spatial affinity weights. Moreover, three weight strategies are designed for those samples nearby the decision boundary to improve the separability of different classes. In addition, the window size of spatial neighbors is able to be adjusted adaptively according to whether labeled data points in spatial neighbors exit. Experimental results on two datasets, Indian Pine and Pavia University have demonstrated that our algorithm is remarkably superior to other conventional DR algorithms on improving classification performance.

Deep unsupervised learning with consistent inference of latent representations

Utilizing unlabeled data to train deep neural networks (DNNs) is a crucial but challenging task. In this paper, we propose an end-to-end approach to tackle this problem with consistent inference of latent representations. Specifically, each unlabeled data point is considered as a seed to generate a set of latent labeled data points by adding various random disturbances or transformations. Under the expectation maximization framework, DNNs can be trained in an unsupervised way by minimizing the distances between the data points with the same latent representations. Furthermore, several variants of our approach can be derived by applying regularized and sparse constraints during optimization. Theoretically, the convergence of the proposed method and its variants are fully analyzed. Experimental results show that the proposed approach can significantly improve the performance on various tasks, including image classification and clustering. Such results also indicate that our method can guide DNNs to learn more invariant feature representations in comparison with traditional unsupervised methods.

Driving Style Classification Using a Semisupervised Support Vector Machine

Supervised learning approaches are widely used for driving style classification; however, they often require a large amount of labeled training data, which is usually scarce in a real-world setting. Moreover, it is time-consuming to manually label huge amounts of driving data due to uncertainties of driver behavior and variances among the data analysts. To address this problem, a semisupervised approach, a semisupervised support vector machine (S3VM), is employed to classify drivers into aggressive and normal styles based on a few labeled data points. First, a few data clusters are selected and manually labeled using a $ k$ -means clustering method. Then, a specific differentiable surrogate of a loss function is developed, which makes it feasible to use standard optimization tools to solve the nonconvex optimization problem. One of the most popular quasi-Newton algorithms is then used to assign the optimal label to all of the training data. Finally, we compare the S3VM method with a support vector machine method for classifying driving styles from different amounts of labeled data. Experiments show that the S3VM method can improve the classification accuracy by about 10% and reduce the labeling effort by using only a few labeled data clusters among huge amounts of unlabeled data.

Active learning via local structure reconstruction

To select the most representative data points for labeling, two typical active learning methods, Transductive Experimental Design (TED) and Robust Representation and Structured Sparsity (RRSS), have been recently proposed. They yield impressive results. However, both of them neglected the local structure of data points which is helpful for selecting representative data points. Therefore, in this paper, we propose a novel active learning method via local structure reconstruction to select representative data points. Specifically, we construct a simple but effective graph to search the local relationship of data points. Then an optimization model is formulated to fulfill the data point reconstruction and select the most representative data points. Furthermore, we define a simple but useful classifier based on a linear regression model for better exploring the potential classification performance of selected data points. Experimental results on two synthetic datasets and two face databases demonstrate the effectiveness of our method and the efficiency of the defined classifier.

Laplacian hashing for fast large-scale image retrieval and its applications for midway processing in a cascaded face detection structure

In this paper, we propose a new philosophy different from that of the well-known Locality-Sensitive Hashing (LSH): if two data points are close, we wish that the probability for them to fall into the same hash buckets is high; whereas if two data points are far away, we do not care the probability of them falling into the same hash buckets. Our new philosophy is a relaxation of the LSH requirement, by ignoring the side effects of placing differently labeled data points into the same hash bucket. Based on such relaxation, a new hashing method, namely the Laplacian Hashing, is derived, which is natural to incorporate any kernel functions and "similar" / "dissimilar" weakly supervised information. Another contribution of this paper is that, it is the first time that a fast hashing method is applied for the midway processing in a cascaded face detection structure. Experimental results show that, our method is on average not worse than the state of the arts in terms of accuracy, but much faster and thus can handle much larger training datasets within reasonable computation time.