Classification Objective Function Research Articles

Online learning for streaming data classification in nonstationary environments

AbstractIn this article, we implement the classification of nonstationary streaming data. Due to the inability to obtain full data in the context of streaming data, we adopt a strategy based on clustering structure for data classification. Specifically, this strategy involves dynamically maintaining clustering structures to update the model, thereby updating the objective function for classification. Simultaneously, incoming samples are monitored in real‐time to identify the emergence of new classes or the presence of outliers. Moreover, this strategy can also deal with the concept drift problem, where the distribution of data changes with the inflow of data. Regarding the handling of novel instances, we introduce a buffer analysis mechanism to delay their processing, which in turn improves the prediction performance of the model. In the process of model updating, we also introduce a novel renewable strategy for the covariance matrix. Numerical simulations and experiments on datasets show that our method has significant advantages.

A novel discrete deep learning-based intelligent methodology for energy consumption classification

Energy consumption classification is one of the most widely-used approaches in the energy area that is applied in various applications such as household, commercial, urban, rural, industrial, etc. Energy consumption due to its substantial positive influence on the quality of made decisions, production and distribution management, and cost reduction, has considerable literature. In the classification literature, numerous statistical, intelligence, and hybrid methods have been developed and proposed in order to yield more accurate results. The common point of all these models is that whole of techniques follow a prevalent and repetitive procedure of learning process based on continuous distance-based cost function. While the objective function of classification is discrete. While the mismatch of the continuous cost function in classification problems with the discrete objective function causes them to be illogical or inefficient. Therefore, in this paper, a novel discrete learning process is offered to eliminate the inconsistency between the cost function and the classification objective function. The main difference between the proposed learning methodology rather than conventional versions is its cost function. In the proposed learning methodology, a mismatching function is considered as a cost function, which is dissimilar to previously developed ones, which are continuous functions based on distance, is a discrete function based on direction. In this way, in the proposed learning process, unknown parameters are discretely adjusted and at once jumped to the target. This is in contrast to conventional continuous learning algorithms in which the unknown parameters are continuously adjusted and step-by-step near the target. In this paper, deep multilayer neural networks (DMNs) have been exemplarily applied to implement the proposed learning algorithm. Although more consistency of goal and cost functions will logically has not a negative effect on the classification rate of classifiers. However, in this paper, to show the superiority of the proposed discrete deep multilayer neural network (DDMN) over conventional continuous-based classifiers, four energy consumption benchmark data sets have been employed. These data sets include the Tamilnadu Electricity Board Hourly Readings, Tetouan City Energy Consumption, Household Electric Power consumption, and steel industry Energy Consumption. Empirical results indicate that the proposed DDMN model, as pre-expected, can yield better performance in all cases than the DMN model. The DDMN model can achieve a 93.27% classification rate on average and upgrade its classic version approximately by 4.77%.

Adversarial Decoupling and Modality-Invariant Representation Learning for Visible-Infrared Person Re-Identification

Visible-infrared person re-identification (RGB-IR ReID) has now attracted increasing attention due to its surveillance applications under low-light environments. However, the large intra-class variations between different domains are still a challenging issue in the field of computer vision. To address the above issue, we propose a novel adversarial Decoupling and Modality-invariant Representation learning (DMiR) method to explore potential spectrum-invariant yet identity-discriminative representations for cross-modality pedestrians. Our model consists of three key components, including Domain-related Representation Disentanglement (DrRD), Modality-invariant Discriminative Representation (MiDR) and Representation Orthogonal Decorrelation (ROD). First, two subnets named Identity-Net and Domain-Net are designed to extract identity-related features and domain-related features, respectively. Given this two-stream structure, the DrRD is introduced to achieve adversarial decoupling against domain-specific features via a min-max disentanglement process. Specifically, the classification objective function on Domain-Net is minimized to extract spectrum-specific information while maximizing it to reduce domain-specific information. Second, in Identity-Net, we introduce MiDR to enhance intra-class compactness and reduce domain variations by exploring positive and negative pair variations, semantic-wise differences, and pair-wise semantic variations. Finally, the correlation between the two decomposed features, i.e., identity-related features and domain-related features, may lead to the introduction of modal information in identity representations, and vice versa. Therefore, we present the ROD constraint to make the two decomposed features unrelated to each other, which can more effectively separate the two-component features and enhance feature representations. Practically, we construct ROD at the feature-level and parameter-level, and finally select feature-level ROD as the decorrelation strategy because of its superior decorrelation performance. The whole scheme leads to disentangling spectrum-dependent information, as well as purifying identity information. Extensive experiments are carried out on two mainstream RGB-IR ReID datasets, and the results demonstrate the effectiveness of our method.

An improving sparse coding algorithm for wireless passive target positioning

With the developments of smart city, wireless passive localization (WPL) technique that detects targets without carrying any devices draws a lot of research attention. Some machine learning methods, such as sparse coding and deep learning, have been developed to formulate the WPL as a classification problem. Despite these methods have been proved to be effective in WPL, it remains hot topics to design a precise objective function that could detect the position of the targets with high accuracy and robustness. In this paper, we exploited a sparsity regularizer, named log-regularizer, in the classification objective function. By virtue of the distinguished ability in measuring sparsity, our proposed improved sparse coding algorithm (ISCA) with log-regularizer could position targets accurately with robust performance even in the challenging environments Experimental results show that our proposal achieves better results compared with five other machine learning algorithms. Even the input data is severely polluted by noise (SNR = −10 dB), the proposed method could still obtain high localization accuracy of 99.4%.

Auditory Inspired Convolutional Neural Networks for Ship Type Classification with Raw Hydrophone Data.

Detecting and classifying ships based on radiated noise provide practical guidelines for the reduction of underwater noise footprint of shipping. In this paper, the detection and classification are implemented by auditory inspired convolutional neural networks trained from raw underwater acoustic signal. The proposed model includes three parts. The first part is performed by a multi-scale 1D time convolutional layer initialized by auditory filter banks. Signals are decomposed into frequency components by convolution operation. In the second part, the decomposed signals are converted into frequency domain by permute layer and energy pooling layer to form frequency distribution in auditory cortex. Then, 2D frequency convolutional layers are applied to discover spectro-temporal patterns, as well as preserve locality and reduce spectral variations in ship noise. In the third part, the whole model is optimized with an objective function of classification to obtain appropriate auditory filters and feature representations that are correlative with ship categories. The optimization reflects the plasticity of auditory system. Experiments on five ship types and background noise show that the proposed approach achieved an overall classification accuracy of 79.2%, which improved by 6% compared to conventional approaches. Auditory filter banks were adaptive in shape to improve accuracy of classification.

Hyperspectral Image Classification via Discriminative Sparse Representation with Extended LBP Texture

Hyperspectral images (HSI) have rich texture information, so combining texture information and image spectral information can improve the recognition accuracy. Sparse representation has significant success in image classification. In this paper, we propose a new discriminative sparse-based classification framework using spectral data and extended Local Binary Patterns (LBP) texture. Firstly, we propose an extended LBP coding for HSI classification. Then we formulate an optimization problem that combines the objective function of classification with the representation error by sparsity. Furthermore, we use a procedure similar to K-SVD algorithm to learn the discriminative dictionary. The experimental results show that the proposed discriminative spasity-based classification of image including the extended LBP texture outperforms the classical HSI classification algorithms.

A modified area under the ROC curve and its application to marker selection and classification

The area under the ROC curve (AUC) can be interpreted as the probability that the classification scores of a diseased subject is larger than that of a non-diseased subject for a randomly sampled pair of subjects. From the perspective of classification, we want to find a way to separate two groups as distinctly as possible via AUC. When the difference of the scores of a marker is small, its impact on classification is less important. Thus, a new diagnostic/classification measure based on a modified area under the ROC curve (mAUC) is proposed, which is defined as a weighted sum of two AUCs, where the AUC with the smaller difference is assigned a lower weight, and vice versa. Using mAUC is robust in the sense that mAUC gets larger as AUC gets larger as long as they are not equal. Moreover, in many diagnostic situations, only a specific range of specificity is of interest. Under normal distributions, we show that if the AUCs of two markers are within similar ranges, the larger mAUC implies the larger partial AUC for a given specificity. This property of mAUC will help to identify the marker with the higher partial AUC, even when the AUCs are similar. Two nonparametric estimates of an mAUC and their variances are given. We also suggest the use of mAUC as the objective function for classification, and the use of the gradient Lasso algorithm for classifier construction and marker selection. Application to simulation datasets and real microarray gene expression datasets show that our method finds a linear classifier with a higher ROC curve than some other existing linear classifiers, especially in the range of low false positive rates.

Regularized ROC method for disease classification and biomarker selection with microarray data

An important application of microarrays is to discover genomic biomarkers, among tens of thousands of genes assayed, for disease classification. Thus there is a need for developing statistical methods that can efficiently use such high-throughput genomic data, select biomarkers with discriminant power and construct classification rules. The ROC (receiver operator characteristic) technique has been widely used in disease classification with low-dimensional biomarkers because (1) it does not assume a parametric form of the class probability as required for example in the logistic regression method; (2) it accommodates case-control designs and (3) it allows treating false positives and false negatives differently. However, due to computational difficulties, the ROC-based classification has not been used with microarray data. Moreover, the standard ROC technique does not incorporate built-in biomarker selection. We propose a novel method for biomarker selection and classification using the ROC technique for microarray data. The proposed method uses a sigmoid approximation to the area under the ROC curve as the objective function for classification and the threshold gradient descent regularization method for estimation and biomarker selection. Tuning parameter selection based on the V-fold cross validation and predictive performance evaluation are also investigated. The proposed approach is demonstrated with a simulation study, the Colon data and the Estrogen data. The proposed approach yields parsimonious models with excellent classification performance.

IMPROVEMENT ON BEHAVIOR SCORES BY DUAL-MODEL SCORING SYSTEM

In credit risk management, the on-line analytical process has been accepted by most credit card issuers. The major tools used in such an OLAP are statistics and neural networks. Through a designed algorithm, the OLAP generates scores for each account or for each customer which depends on the level of the processing. Generally speaking, logistic regressions and feed-forward networks are the major players in OLAP of this field and usually are used separately. This paper discusses an approach — Dual-Model Scoring System — to combine these two major players and use them together in the credit scoring. Primarily, the classification problem for two classes are considered. By the Bayesian rule, the objective function of classification can be reduced to estimate the Bayesian posterior probability. Such a probability is estimated by using the MLE approach in logistic regressions and the Two-Stage (Gibbs) learning algorithm3 in feed-forward networks. The motivation of the proposal comes from the following two considerations: (1) Both logistic regression and neural networks have their advantages and disadvantages and the combined of these two can enhance their predictive ability and offset their weakness. (2) To reduce the false positive rate in the decision region. Besides the discussion of the architecture design of Dual-Model Scoring System, the paper has demonstrated the power of the present proposal in a real data set.