Training Process Of Classifier Research Articles

Feature selection based on unsupervised clustering evaluation for predicting neoadjuvant chemoradiation response for patients with locally advanced rectal cancer

Accurate response prediction allows for personalized cancer treatment of locally advanced rectal cancer (LARC) with neoadjuvant chemoradiation. In this work, we designed a convolutional neural network (CNN) feature extractor with switchable 3D and 2D convolutional kernels to extract deep learning features for response prediction. Compared with radiomics features, convolutional kernels may adaptively extract local or global image features from multi-modal MR sequences without the need of feature predefinition. We then developed an unsupervised clustering based evaluation method to improve the feature selection operation in the feature space formed by the combination of CNN features and radiomics features. While normal process of feature selection generally includes the operations of classifier training and classification execution, the process needs to be repeated many times after new feature combinations were found to evaluate the model performance, which incurs a significant time cost. To address this issue, we proposed a cost effective process to use a constructed unsupervised clustering analysis indicator to replace the classifier training process by indirectly evaluating the quality of new found feature combinations in feature selection process. We evaluated the proposed method using 43 LARC patients underwent neoadjuvant chemoradiation. Our prediction model achieved accuracy, area-under-curve (AUC), sensitivity and specificity of 0.852, 0.871, 0.868, and 0.735 respectively. Compared with traditional radiomics methods, the prediction models (AUC = 0.846) based on deep learning-based feature sets are significantly better than traditional radiomics methods (AUC = 0.714). The experiments also showed following findings: (1) the features with higher predictive power are mainly from high-order abstract features extracted by CNN on ADC images and T2 images; (2) both ADC_Radiomics and ADC_CNN features are more advantageous for predicting treatment responses than the radiomics and CNN features extracted from T2 images; (3) 3D CNN features are more effective than 2D CNN features in the treatment response prediction. The proposed unsupervised clustering indicator is feasible with low computational cost, which facilitates the discovery of valuable solutions by highlighting the correlation and complementarity between different types of features.

An efficient framework for the similarity prediction with query recommendation in E‐learning system

An efficient framework for the similarity prediction with query recommendation in E‐learning system

Parallel Cross-Sparse Filtering Networks and Its Application on Fault Diagnosis of Rotating Machinery

Intelligent diagnosis method has become a new focus for researchers, which can get rid of the dependence of diagnostic experience and prior knowledge. However, in practical application, to deal with the new fault type of mechanical equipment, the number of fault labels of the diagnosis model needs to be increased. We must retrain the whole training model, which is a time-consuming process. To solve this problem, higher requirements are put forward for the generalization ability and universality of the algorithm. In view of the feature extraction advantages of cross-sparse filtering (Cr-SF), which can be regarded as an unsupervised minimum entropy learning method using the maximization of the proxy of sparsity, this paper proposed a parallel network based on Cr-SF. The feature extraction process of each sample is independent, and the feature extraction and classifier training process are separated. Therefore, the most prominent advantage of the proposed method is that when a new fault occurs, it only needs to extract the feature of the new fault separately and then input it to the classifier at the last layer for training. The experimental results show that the proposed method can obtain high accuracy and stability and can significantly improve the adaptability of intelligent fault diagnosis in practical application.

A Deep Transfer Learning Fault Diagnosis Method Based on WGAN and Minimum Singular Value for Non-Homologous Bearing

In real industry, due to changes in operating conditions and differences in systems of interest, domain shift is a common problem, which results in the degradation of the diagnostic performance. Moreover, insufficient labeled or unlabeled samples greatly limit the adaptability of fault diagnosis methods. To solve these problems, the latest domain adversarial transfer learning techniques are studied. However, most of the existing studies just reduce the distribution discrepancy of two domains with fixed distance metrics, which cannot adapt well to distribution variation during testing. This article proposes a new deep transfer learning method based on Wasserstein generative adversarial networks (WGAN) and minimum singular value for non-nomologous bearings, where the training samples and the testing samples are from related not the same machinery. In the proposed method, a domain critic network is adopted to provide the discrepancy metric for improving domain adaptation ability. In addition, the minimum singular value is used to capture effective categories information measurement in the classifier training process. The proposed method is verified by two different bearing datasets from Case Western Reserve University and the Intelligent Maintenance System. The experiments indicated that the proposed method can achieve superior performance over other existing methods in terms of diagnosis accuracy.

A low-cost and efficient electronic nose system for quantification of multiple indoor air contaminants utilizing HC and PLSR

A novel quantification technique is presented for electronic nose (E-nose), which is based on a double-step strategy combined with hierarchical classifier (HC) and partial least squares regression (PLSR). With the tree structure of HC, the complexity of classifier training process can be reduced in the case of unbalanced samples. For each level of the class hierarchy, the extreme learning machine-based artificial neural network (ELM-ANN) is applied for classification. In order to improve the classification performance of ELM-ANN, the multiple time-domain features are selected as training inputs, and a novel optimization method of the number of hidden layer neurons is given. To validate the effectiveness of the presented quantification technique, an E-nose system is designed to quantify the gases including six toxic gases (hydrogen sulfide, carbon monoxide, ammonia, toluene, formaldehyde, acetone) and three kinds of binary gas mixtures. This presented hierarchical classifier has demonstrated outstanding performance for the identification of target gases, such as the macro-averaged precision for unlabeled data is improved from 80% to 92% compared with non-hierarchical classifiers. Furthermore, an excellent performance of concentration estimation is obtained utilizing PLSR, where average values of the coefficient of determination for training and test samples are equal to 0.957 and 0.927, respectively. Overall, our work demonstrates that the proposed approach is applicable in E-nose-based odor quantification.

Towards Effective Classification of aMCI Based on Resting-State Multiscale Brain Features and Machine Learning Approaches

Smart healthcare has undergone new opportunities and challenges with the arrival of the Industry 4.0 era. The intelligent imaging diagnosis system is a staple part of smart healthcare, helping doctors make clinical decisions. Nevertheless, intelligent diagnosis analysis is still confronted with the issue that it is challenging to extract effective features from the limited and high-dimensional data, particularly in resting-state data of amnesic mild cognitive impairment (aMCI). Furthermore, the intelligent imaging diagnosis system for aMCI is conductive to make timely predicting groups that may convert to Alzheimer’s disease (AD). To improve the system’s detection performance and reduce its data redundancy, we first develop an adaptive structure feature generation strategy (ASFGS) based on the Laplacian matrix and sparse autoencoder to obtain the structural features of brain functional network (BFN). Concurrently, we present a multiscale local feature detection strategy (MLFDS) to overcome the low utilization of local features of BFN. And finally, multiscale features, including structural features and multiscale local features, are fused by concatenation method to further improve the detection performance of aMCI system. Support vector machine based on radial basis function (RBF-SVM) for small data learning is adopted to evaluate the effectiveness of the proposed features. Besides, we employ leave-one-out cross-validation strategy to avoid the overfitting problem of classifier training process. The experiment results elucidate that the accuracy (ACC) and the area under the curve (AUC) in this work provide 86.57% and 86.36%, respectively, which outperforms the traditional methods and offers new insights for accuracy requirements of the aMCI system.

Multilabel Classification With Label-Specific Features and Classifiers: A Coarse- and Fine-Tuned Framework.

Multilabel classification deals with instances assigned with multiple labels simultaneously. It focuses on learning a mapping from feature space to label a space for out-of-sample extrapolation. The mapping can be seen as a feature selection process in the feature domain or as a classifier training process in the classifier domain. The existing methods do not effectively learn the mapping when combining these two domains together. In this article, we derive a mechanism to extract label-specific features in local and global levels. We also derive a mechanism to train label-specific classifiers in individual and joint levels. Extracting features globally and training classifiers jointly can be seen as a dual process of learning the mapping function on two domains in a coarse-tuned way, while extracting features locally and training classifiers individually can be seen as a dual process of learning the mapping function on two domains in a fine-tuned way. The two-level feature selection and the two-level classifier training are derived to make the entire mapping learning process robust. Finally, extensive experimental results on several benchmarks under four domains are presented to demonstrate the effectiveness of the proposed approach.

A Blockchain-Driven IIoT Traffic Classification Service for Edge Computing

Nowadays, more and more sensors, devices and applications are connected in Industrial Internet of Things (IIoT), producing massive real-time flows which need to be scheduled for Quality-of-Service provision. To realize application-aware and adaptive flow scheduling, the problem of traffic classification must be addressed at first. When edge computing paradigm is introduced into IIoT, the traffic classification service can be deployed on edge node in the near-end. Recently, deep-learning-based IIoT traffic classification methods show better performance, but the computational cost of deep learning model is too high to be deployed on edge node. Moreover, increasingly unknown flows generated by new devices and emerging industrial APPs lead to frequent training of traffic classifiers. It is difficult to migrate the complex process of classifier training from cloud server to edge nodes with limited resources. To address these issues, we take the benefits of hash mechanism and consensus mechanism in blockchain to design a lightweight IIoT traffic classification service, which is more applicable for edge computing paradigm. First, inspired by the hash mechanism in blockchain and the learning to hash for big data, we propose a new learning-to-hash method named extension hashing. By this method, we can build the set of binary coding tress (BCT set), then generating hash table for more efficient k-nearest neighbor-based classification without complex classifier training. Then, we design a new voting-based consensus algorithm to synchronize the BCT sets and the hash tables across edge nodes, thereby providing the traffic classification service. Finally, we conduct data-driven simulations to evaluate the proposed service. By comparing traffic classification results on public data set, we can see that the proposed service achieves the highest classification accuracy with the minimal time cost and memory usage.

A collection input based support tensor machine for lesion malignancy classification in digital breast tomosynthesis

Digital breast tomosynthesis (DBT) with improved lesion conspicuity and characterization has been adopted in screening practice. DBT-based diagnosis strongly depends on physicians’ experience, so an automatic lesion malignancy classification model using DBT could improve the consistency of diagnosis among different physicians. Tensor-based approaches that use the original imaging data as input have shown promising results for many classification tasks. However, DBT data are pseudo-3D volumetric imaging as the slice spacing of DBT is much coarser than that of the in-plane resolution. Thus, directly constructing DBT as the third-order tensor in a conventional tensor-based classifier with introducing additional information to the original DBT data along the slice-spacing dimension will lead to inconsistency across all three dimensions. To avoid such inconsistency, we introduce a collection input based support tensor machine (CISTM)-based classifier that uses the tensor collection as input for classifying lesion malignancy in DBT. In CISTM, instead of introducing the third dimension directly into the geometry construction, the third-dimension structural relationship is related by weight parameters in the decision function, which is dynamically and automatically constructed during the classifier training process and is more consistent with the pseudo-3D nature of DBT. We tested our method on a DBT dataset of 926 images among which 262 were malignant and 664 were benign. We compared our method with the latest tensor-based method, KSTM (kernelled support tensor machine), which does not consider the unique non-uniform resolution property of DBT. Experimental results illustrate that the CISTM-based classifier is effective for classifying breast lesion malignancy in DBT and that it outperforms the KSTM-based classifier.

Cloud detection in satellite images using multi-objective social spider optimization

Cloud detection algorithms have emerged to automate image data analysis because of its prime influential factor in remote sensing image quality. Cloud detection algorithm still needs domain-expert intervention and large number of training examples to ensure good performance whose acquirement becomes difficult due to unavailability of labeled data as well as the time and process heads involved. The paper puts forward multi-objective social spider optimization (MOSSO) based efficient clustering technique to detect clouds in the visible range. This paper explains the proposed MOSSO algorithm along-with the analysis carried on 14 benchmark two-objective test problems against MOEA/D, MODE, MOPSO and SPEA2 multi-objective algorithms. Further, the strengths and weaknesses of the proposed algorithm are analyzed and have been used for the implementation of an efficient clustering technique named as MOSSO-C. Optimal centroid matrix for clustering is attained in MOSSO-C through environmental selection whose performance evaluation has been done on six synthetic databases and are compared with above mentioned conventional multi-objective algorithms. The obtained results encourage the use of MOSSO-C technique to get labeled data for training process of neural network classifier. This approach efficiently classifies the cloudy pixels against various Earth’s surfaces (water, vegetation and land). The paper also discusses the performance evaluation of proposed technique on four Landsat 8 data which shows on an average 96.37% performance accuracy in detecting cloudy pixels.

Adaboost-SVM-based probability algorithm for the prediction of all mature miRNA sites based on structured-sequence features

The significant role of microRNAs (miRNAs) in various biological processes and diseases has been widely studied and reported in recent years. Several computational methods associated with mature miRNA identification suffer various limitations involving canonical biological features extraction, class imbalance, and classifier performance. The proposed classifier, miRFinder, is an accurate alternative for the identification of mature miRNAs. The structured-sequence features were proposed to precisely extract miRNA biological features, and three algorithms were selected to obtain the canonical features based on the classifier performance. Moreover, the center of mass near distance training based on K-means was provided to improve the class imbalance problem. In particular, the AdaBoost-SVM algorithm was used to construct the classifier. The classifier training process focuses on incorrectly classified samples, and the integrated results use the common decision strategies of the weak classifier with different weights. In addition, the all mature miRNA sites were predicted by different classifiers based on the features of different sites. Compared with other methods, the performance of the classifiers has a high degree of efficacy for the identification of mature miRNAs. MiRFinder is freely available at https://github.com/wangying0128/miRFinder.

Isometric Shape Representation by Integrating Shape Function Maps and Deep Learning

With the wide application of isometric 3D models, the representation and recognition of them receive increasing attention. Most existing methods for shape representation and analysis either focus on using the constrained hand-craft models or purely devote efforts on developing complicated deep learning methods. In order to make full use of the advantages of both, this paper presents a novel mixture modeling approach by integrating both the shape function map (SFM) and the deep convolutional neural network (CNN). First, multiple SFM maps are constructed to grasp the rigid and non-rigid information that are usually considered separately for shape representation. Then, to fully characterize the low-level information existing in SFM, diverse sets of deep features are learned on different SFM maps during the training process of classifiers. Finally, both the rigid and non-rigid deep features are integrated for more discriminative feature abstraction. The experimental results on standard shape benchmark datasets have validated the superior performance of our proposed approach on feature extraction, classification and retrieval. Besides, our evaluation results on extensive shape datasets (e.g. noisy, CAD and protein shapes) have again verified the effectiveness of the proposed algorithm.

Parallel naive Bayes algorithm for large-scale Chinese text classification based on spark

The sharp increase of the amount of Internet Chinese text data has significantly prolonged the processing time of classification on these data. In order to solve this problem, this paper proposes and implements a parallel naive Bayes algorithm (PNBA) for Chinese text classification based on Spark, a parallel memory computing platform for big data. This algorithm has implemented parallel operation throughout the entire training and prediction process of naive Bayes classifier mainly by adopting the programming model of resilient distributed datasets (RDD). For comparison, a PNBA based on Hadoop is also implemented. The test results show that in the same computing environment and for the same text sets, the Spark PNBA is obviously superior to the Hadoop PNBA in terms of key indicators such as speedup ratio and scalability. Therefore, Spark-based parallel algorithms can better meet the requirement of large-scale Chinese text data mining.

Cost-sensitive ensemble classification algorithm for medical image

Medical image classification is an important part in domain-specific application image mining. In this paper, we quantify the domain knowledge about medical image for feature extraction. We propose a cost-sensitive ensemble classification (CEC) algorithm which uses a new training method and adopts a new method to acquire parameters. In the weak classifier training process, we mark the samples that are wrongly classified in the former iteration, use the method of re-sampling in the samples that are correctly classified, and put all the wrongly classified samples in the next training. The classification can pay more attention to those samples that are hardly classified. The weight parameters of weak classifiers are determined not only by the error rates, but also by their abilities to recognise the positive samples. Experimental results show that our algorithm is more efficient for medical image classification.

Statistical Features-Based Real-Time Detection of Drifted Twitter Spam

Twitter spam has become a critical problem nowadays. Recent works focus on applying machine learning techniques for Twitter spam detection, which make use of the statistical features of tweets. In our labeled tweets data set, however, we observe that the statistical properties of spam tweets vary over time, and thus, the performance of existing machine learning-based classifiers decreases. This issue is referred to as “Twitter Spam Drift”. In order to tackle this problem, we first carry out a deep analysis on the statistical features of one million spam tweets and one million non-spam tweets, and then propose a novel Lfun scheme. The proposed scheme can discover “changed” spam tweets from unlabeled tweets and incorporate them into classifier's training process. A number of experiments are performed to evaluate the proposed scheme. The results show that our proposed Lfun scheme can significantly improve the spam detection accuracy in real-world scenarios.

Collaborative Sparse Coding for Multiview Action Recognition

Multiview action recognition has received increasing attention over the past decade. Various approaches have been proposed to extract view-invariant features; among them, self-similarity matrices (SSMs) have shown outstanding performance. However, SSMs become sensitive when there's a very large view change. To make SSMs more robust to viewpoint changes, the authors propose a collaborative sparse coding framework. They integrate the classifier training process and sparse coding process into a unified collaborative filtering framework; this lets more discriminative sparse video representations and classifiers be learned by optimizing the dictionary and classifier jointly. Experimental results demonstrate the effectiveness of the framework.

Novel approaches using evolutionary computation for sparse least square support vector machines

This paper introduces two new approaches to building sparse least square support vector machines (LSSVM) based on genetic algorithms (GAs) for classification tasks. LSSVM classifiers are an alternative to SVM ones because the training process of LSSVM classifiers only requires to solve a linear equation system instead of a quadratic programming optimization problem. However, the absence of sparseness in the Lagrange multiplier vector (i.e. the solution) is a significant problem for the effective use of these classifiers. In order to overcome this lack of sparseness, we propose both single and multi-objective GA approaches to leave a few support vectors out of the solution without affecting the classifier׳s accuracy and even improving it. The main idea is to leave out outliers, non-relevant patterns or those ones which can be corrupted with noise and thus prevent classifiers to achieve higher accuracies along with a reduced set of support vectors. Differently from previous works, genetic algorithms are used in this work to obtain sparseness not to find out the optimal values of the LSSVM hyper-parameters.

Detecting tympanostomy tubes from otoscopic images via offline and online training

Tympanostomy tube placement has been commonly used nowadays as a surgical treatment for otitis media. Following the placement, regular scheduled follow-ups for checking the status of the tympanostomy tubes are important during the treatment. The complexity of performing the follow up care mainly lies on identifying the presence and patency of the tympanostomy tube. An automated tube detection program will largely reduce the care costs and enhance the clinical efficiency of the ear nose and throat specialists and general practitioners. In this paper, we develop a computer vision system that is able to automatically detect a tympanostomy tube in an otoscopic image of the ear drum. The system comprises an offline classifier training process followed by a real-time refinement stage performed at the point of care. The offline training process constructs a three-layer cascaded classifier with each layer reflecting specific characteristics of the tube. The real-time refinement process enables the end users to interact and adjust the system over time based on their otoscopic images and patient care. The support vector machine (SVM) algorithm has been applied to train all of the classifiers. Empirical evaluation of the proposed system on both high quality hospital images and low quality internet images demonstrates the effectiveness of the system. The offline classifier trained using 215 images could achieve a 90% accuracy in terms of classifying otoscopic images with and without a tympanostomy tube, and then the real-time refinement process could improve the classification accuracy by 3–5% based on additional 20 images.

Semi-Supervised Learning Based Social Image Semantic Mining Algorithm

As social image semantic mining is of great importance in social image retrieval, and it can also solve the problem of semantic gap. In this paper, a novel social image semantic mining algorithm based on semi-supervised learning is proposed. Firstly, labels which tagged the images in the test image dataset are extracted, and noisy semantic information are pruned. Secondly, the labels are propagated to construct an extended collection. Thirdly, image visual features are extracted from the unlabeled images by three steps, including watershed segmentation, region feature extraction and codebooks construction. Fourthly, vectors of image visual feature are obtained by dimension reduction. Fifthly, after the process of semi-supervised learning and classifier training, the confidence score of semantic terms for the unlabeled image are calculated by integrating different types of social image features, and then the heterogeneous feature spaces are divided into several disjoint groups. Finally, experiments are conducted to make performance evaluation. Compared with other existing methods, it can be seen than the proposed can effectively extract semantic information of social images

Parallel multiclass stochastic gradient descent algorithms for classifying million images with very-high-dimensional signatures into thousands classes

The new parallel multiclass stochastic gradient descent algorithms aim at classifying million images with very-high-dimensional signatures into thousands of classes. We extend the stochastic gradient descent (SGD) for support vector machines (SVM-SGD) in several ways to develop the new multiclass SVM-SGD for efficiently classifying large image datasets into many classes. We propose (1) a balanced training algorithm for learning binary SVM-SGD classifiers, and (2) a parallel training process of classifiers with several multi-core computers/grid. The evaluation on 1000 classes of ImageNet, ILSVRC 2010 shows that our algorithm is 270 times faster than the state-of-the-art linear classifier LIBLINEAR.