Class Probability Distributions Research Articles

A deep neural network to search for new long-lived particles decaying to jets

A tagging algorithm to identify jets that are significantly displaced from the proton-proton (pp) collision region in the CMS detector at the LHC is presented. Displaced jets can arise from the decays of long-lived particles (LLPs), which are predicted by several theoretical extensions of the standard model. The tagger is a multiclass classifier based on a deep neural network, which is parameterised according to the proper decay length c τ 0 of the LLP. A novel scheme is defined to reliably label jets from LLP decays for supervised learning. Samples of pp collision data, recorded by the CMS detector at a centre-of-mass energy of 13 TeV, and simulated events are used to train the neural network. Domain adaptation by backward propagation is performed to improve the simulation modelling of the jet class probability distributions observed in pp collision data. The potential performance of the tagger is demonstrated with a search for long-lived gluinos, a manifestation of split supersymmetric models. The tagger provides a rejection factor of 10 000 for jets from standard model processes, while maintaining an LLP jet tagging efficiency of 30%–80% for gluinos with 1 mm≤c τ 0≤ 10 m. The expected coverage of the parameter space for split supersymmetry is presented.

Making up the shortages of the Bayes classifier by the maximum mutual information classifier

The Bayes classifier is often used because it is simple, and the maximum posterior probability (MPP) criterion it uses is equivalent to the least error rate criterion. However, it has issues in the following circumstances: (i) if information instead of correctness is more important, we should use the maximum likelihood criterion or maximum information criterion, which can reduce the rate of failure to report small probability events. (ii) For unseen instance classifications, the previously optimised classifier cannot be properly used when the probability distribution of true classes is changed. (iii) When classes’ feature distributions instead of transition probability functions (TPFs) are stable, it is improper to train the TPF, such as the logistic function, with parameters. (iv) For multi-label classifications, it is difficult to optimise a group of TPFs with parameters that the Bayes classifier needs. This study addresses these issues by comparing the MPP criterion with the maximum likelihood criterion and maximum mutual information (MMI) criterion. It suggests using the MMI criterion for most unseen instance classifications. It presents a new iterative algorithm, the channel matching (CM) algorithm, for the MMI classification. It uses two examples to show the advantages of the CM algorithm: fast and reliable.

A New Adaptive Weighted Deep Forest and Its Modifications

A new adaptive weighted deep forest algorithm which can be viewed as a modification of the confidence screening mechanism is proposed. The main idea underlying the algorithm is based on adaptive weigting of every training instance at each cascade level of the deep forest. The confidence screening mechanism for the deep forest proposed by Pang et al., strictly removes instances from training and testing processes to simplify the whole algorithm in accordance with the obtained random forest class probability distributions. This strict removal may lead to a very small number of training instances at the next levels of the deep forest cascade. The presented modification is more flexible and assigns weights to instances in order to differentiate their use in building decision trees at every level of the deep forest cascade. It overcomes the main disadvantage of the confidence screening mechanism. The proposed modification is similar to the AdaBoost algorithm to some extent. Numerical experiments illustrate the outperformance of the proposed modification in comparison with the original deep forest. It is also illustrated how the proposed algorithm can be extended for solving the transfer learning and distance metric learning problems.

Метод формалізації даних при вирішенні задач розпізнавання повітряних об'єктів

В роботі розроблена методика формалізації структур даних ознак повітряних об’єктів яка враховує характеристики точності і достовірності джерел інформації з нестохастичною невизначеністю і рекурентними правилами узагальнення оцінок значень ознак при багаторазовому спостереженні повітряних об’єктів. В основу розробленої методики були покладені методи формалізації процесу розпізнавання, оцінки ступеня істинності різнорідних ознак, об'єднання незалежних результатів розпізнавання, пошуку рішень про класи розпізнаваних повітряних об'єктів з урахуванням неповноти і надмірності даних про ознаки. Розроблений метод формалізації процесу розпізнавання повітряних об’єктів на основі описів класів різнорідними ознаками, відрізняється від відомих використанням ієрархічної функціональної мережі. Удосконалення методу об'єднання незалежних результатів розпізнавання на основі правила Демпстера полягає в попередньому перетворенні одного з неузгоджених розподілів можливостей класів алфавіту в розподіл ймовірностей. Це дозволяє проводити коректне об'єднання неузгоджених розподілів можливостей класів алфавіту з використанням правила Демпстера. Проведено дослідження застосовності розробленого правила при об'єднанні неузгоджених розподілів ймовірностей класів алфавіту. Розроблений метод пошуку рішень про класи розпізнаваних об'єктів на графі функціональної мережі відрізняється від відомих розробленим способом отримання інтервальної оцінки ступеня істинності класів в умовах відсутності даних щодо окремих ознак повітряних об’єктів і запропонованим евристичним правилом, що дозволяє скоротити розмірність безлічі вершин мережі, які перебираються при пошуку рішення про клас повітряних об’єктів. Метод дозволяє проводити пошук рішень про класи повітряних об’єктів в умовах неповноти і надмірності даних про ознаки.

Improvement of the Deep Forest Classifier by a Set of Neural Networks

A Neural Random Forest (NeuRF) and a Neural Deep Forest (NeuDF) as classification algorithms, which combine an ensemble of decision trees and neural networks, are proposed in the paper. The main idea underlying NeuRF is to combine the class probability distributions produced by decision trees by means of a set of neural networks with shared parameters. The networks are trained in accordance with a loss function which measures the classification error. Every neural network can be viewed as a non-linear function of probabilities of a class. NeuDF is a modification of the Deep Forest or gcForest proposed by Zhou and Feng, using NeuRFs. The numerical experiments illustrate the outperformance of NeuDF and show that the NeuRF is comparable with the random forest

Sparsity adaptive matching pursuit for face recognition

Sparse representation methods have exhibited promising performance for pattern recognition. However, these methods largely rely on the data sparsity available in advance and are usually sensitive to noise in the training samples. To solve these problems, this paper presents sparsity adaptive matching pursuit based sparse representation for face recognition (SAMPSR). This method adaptively explores the valid training samples that exactly represent the test via iterative updating. Next, the test samples are reconstructed via the valid training samples, and classification is performed subsequently. The two-phase strategy helps to improve the discriminating power of class probability distribution, and thus alleviates effect of the noise from the training samples to some extent and correctly performs classification. In addition, the method solves the sparse coefficient by comparing the residual between the test sample and the reconstructed sample instead of using the sparsity. A large number of experiments show that our method achieves promising performance.

Learning GPS Point Representations to Detect Anomalous Bus Trajectories

Discovering anomalous bus trajectories can benefit transportation agencies to improve their services by helping them to deal with unexpected events such as detours or accidents. In this work, we propose a deep-learning strategy, which we name Spatial-Temporal Outlier Detector (STOD), that predicts the spatial/temporal anomaly degree of a bus trajectory by using learned representations of its GPS points. To calculate the score, STOD learns the regular behavior of bus trajectories by building a model that predicts the route id of buses. The degree of uncertainty on this prediction, measured by the entropy of the output class probability distribution, indicates the anomaly score of the trajectory. To perform the classification, STOD represents each point of a trajectory by the concatenation of two different representations. The first one (PAC embedding) is generated by the Point Activity Classifier (PAC) by leveraging temporal and spatial features on a stacked deep-learning model to predict the semantics of the point in terms of its bus activity (in route, bus stop, traffic signal, and other stops). The second representation (Geo embedding) captures the spatial relationship between a point and its geographical neighbors by applying a word embedding technique on the set of all trajectories. The experimental evaluation shows that our model is effective for filtering noisy trajectories since it outputs higher anomaly scores for both spatial and temporal anomalous trajectories than regular ones.

An imprecise deep forest for classification

An imprecise deep forest classifier, which can be regarded as a modification of the deep forest proposed by Zhou and Feng, is presented in the paper. In the proposed classifier, the precise class probabilities at leaf nodes of decision trees in the deep forest are replaced with interval-valued probabilities produced by Walley’s imprecise Dirichlet model. The obtained sets of probabilities are averaged over all trees in a random forest as the imprecise probability masses. In order to use the stacking algorithm in the deep forest, a number of the class probability distributions are generated from sets of the random forest class probabilities such that decision trees of the next level of the deep forest cascade are trained on the basis of the extended training set which is added by new generated class distributions. Numerical examples illustrate how the incorporated imprecision may lead to outperforming results, especially, when the training set is rather small. The proposed classifier is the first modification of the deep forest, which efficiently deals with small datasets and takes into account a lack of sufficient training data. The ideas underlying the classifier may be a basis for development of a set on new machine learning models for small datasets.

Chronological Age Estimation Under the Guidance of Age-Related Facial Attributes

Although the researches of facial attributes’ analysis have been launched for decades, the estimation of chronological age attribute remains a big challenge. Previous researchers have found that some facial attributes (e.g., gender and race attributes) have close connections with the age attribute and make age estimation under a specific condition decided by various combinations of those age-related attributes which should be more reasonable. In this paper, we propose a generic framework based on a convolutional neural network, which can consider different conditions for age estimation and jointly output age and age-related facial attributes in the end. Compared with conventional methods, it is more efficient and universal. Besides, we view age estimation as a special multi-class ordinal classification problem and use a losses combination function to optimize the predicted probability distribution of individual age classes. These operations further improve the performance of age estimation. Finally, the proposed method achieves state-of-the-art results on both controlled and wild face datasets.

A deep forest classifier with weights of class probability distribution subsets

A modification of the Deep Forest or gcForest proposed by Zhou and Feng for solving classification problems is proposed in the paper and called as PM-DF. The main idea for improving classification performance of the Deep Forest is to assign weights to subsets of the class probability distributions at the leaf nodes computed for every training example. The subsets of probability distributions are defined by using Walley’s imprecise pari-mutuel model which compactly divides the unit simplex of probabilities into subsets and allows us to simplify the algorithm of the weight calculation. The weights of the distribution subsets can be viewed in this case as second-order probabilities over subsets of the probability simplex. The optimal weights are computed by solving the standard quadratic optimization problem. The numerical experiments illustrate PM-DF.

Data-Driven State of Health Estimation of Li-Ion Batteries With RPT-Reduced Experimental Data

In order to accurately estimate the state of health (SOH) of a Li-ion battery, a reference performance test (RPT) needs to be conducted between several charging/discharging cycles for obtaining accurate data concerning the battery capacity and degradation. However, it is not practical to perform RPTs frequently because they are time-consuming and expensive; moreover, the Li-ion battery undergoes unnecessary degradation during test operations. Therefore, the RPTs should be performed as infrequently as possible. In this paper, a neural network-based SOH estimation scheme with reduced experimental data measured by the RPT is proposed for achieving economic efficiency and mitigating the dispensable degradation being caused by additional experiments. For the RPT-reduced experimental data, the continuous SOH estimation problem is formulated into a classification problem. The neural network learns how to estimate the SOH values using short time-series voltage and current data, labeled as the corresponding SOH values by the RPTs. Even in the SOH regions, where the data are not labeled as any given class and there is no prior knowledge on the corresponding SOH, the proposed SOH estimation scheme works well by performing regression with the class probability distribution.

Mutual information algorithms for optimal attribute selection in data driven partitions of databases

Clustering algorithms like k-means, BIRCH, CLARANS and DBSCAN are designed to be scalable and they are developed to discover clusters in the full dimensional space of a database. Nevertheless their characteristics depend upon the size of the database. A database/data warehouse may store terabytes of data. Complex data analysis (mining) may take a very long time to run on the complex dataset. One has to obtain a reduced representation of the dataset that is much smaller in volume—but yet produces the same or almost the same analytical results—in order to accelerate information processing. Reduced representations yield simplified models that are easier to interpret, avoid the curse of dimensionality and enhance generalization by reducing overfitting. Data reduction methods include data cube aggregation, attribute subset selection, fitting data into models, dimensionality reduction, hierarchies as well as other approaches. On the other hand, data-dependent partitions—like the Gessaman’s partition and tree-quantization partition—allow for processing different partitions of a dataset separately. Hence parallel processing may be used as an option for big data. Online analytical processing is a practical approach that deals with multi-dimensional queries in DB management. Feature selection is considered as a specific case of a more general paradigm which is called Structure Learning in cases of an outcome associated to a set of attributes. Feature selection aims at selecting a minimum set of features such that the probability distribution of different classes given the values of those features is as close as possible to the original distribution given the values of all features. A mutual information approach based upon representing complex datasets in DB as a minimal set of coherent attribute sets of reduced dimensions is herein proposed. The novelty of the proposed approach consists of employing piecewise analysis of compact clusters in order to increase overall Shannon’s mutual information-entropy as a variant to conventional Classification and Regression Trees. Numerical data regarding a test-bed system for anomaly detection are provided in order to illustrate the aforementioned approach.

Deep Dilation on Multimodality Time Series for Human Activity Recognition

Providing accurate information on people’s activities and behaviors plays an important role in innumerable applications, such as medical, security, and entertainment. In recent years, deep learning has been applied in human activity recognition, and achieved a better performance. However, if the spatial dependency of inter-sensors is considered, it is possible to enhance the discriminative ability. In this paper, we present a novel deep learning framework for human activity recognition problems. First, on the basis of our previous work, we utilize dilated convolutional neural network to extract features of inter-sensors and intra-sensors. Since the extracted features are local and short-temporal, it is necessary to utilize RNN to model the long-temporal dependencies. However, duo to that LSTM and GRU often rely on the completely previous computations, it will result in slow inference and hard convergence. Hence, inspired by the idea of dilation operation, we present a novel recurrent model to learn the temporal dependencies at different time scales. Then, at the topmost layer, a fully-connected layer with softmax function is utilized to generate a class probability distribution, and the predicted activity is obtained. Eventually, we evaluate the proposed framework in two open human activity datasets, OPPORTUNITY and PAMAP2. Results demonstrate that the proposed framework achieves a higher classification performance than the state-of-the-art methods. Moreover, it takes the least time to recognize an activity. Besides, it also performs faster and easier to converge in the training stage.

Ordinal Classification of Imbalanced Data with Application in Emergency and Disaster Information Services

Previous ordinal classification methods implicitly assume that the class distribution within a dataset is balanced, which is often not the case for real-world datasets. If the dataset is imbalanced, the previous methods tend to be biased toward the majority class. The authors propose a new method for ordinal classification that attempts to mitigate the impact of imbalanced datasets. They propose a modified version of the weighted k-nearest neighbors method that determines the class membership using the I±th quantile of the estimated class probability distribution and thus mitigates the impact of the class imbalance.

Probabilistic framework of visual anomaly detection for unbalanced data

This paper proposes a novel probabilistic detection framework of weighted combining semi-supervised k-means clustering and Posterior Probability SVM (PPSVM) for unbalanced data based on robot vision. Within the framework, an algorithm for learning synchronously the k in k-means and features is introduced based on hybrid wrapper and filter criterion. Then the optimal hierarchical probabilistic model by combining k-means and PPSVM is used to anomaly detection so as to alleviate the problems of imbalanced data with small samples, improve the detection accuracy, and deal with the difficult problem of defining the anomaly classes. The other contributions of our approach include the following three aspects: (1) it classifies anomaly candidates by using their class probability distributions rather than the direct extracted features; (2) the relevant classes are automatically built by learning the samples׳ multimodal Gaussian distribution; and (3) the cost-sensitive idea and filter criterion are integrated in learning k and features via cost function of Tabu search. Experimental results on real-world data sets show the proposed approach obtains a satisfactory detection performance within limited time in inspecting the condition of Heating, and Ventilation and Air-Conditioning (HVAC) ductwork.

Local structure based multi-phase collaborative representation for face recognition with single sample per person

Many real-world face recognition applications can only provide single sample for each person, while most face recognition approaches require a large set of training samples, which leads to single sample per person (SSPP) problem. In this paper, we propose local structure based multi-phase collaborative representation classification (LS_MPCRC) to solve SSPP problem. By adopting the “divide-conquer-aggregate” strategy, we successfully alleviate the dilemma of high data dimensionality and small samples, where we first divide the face into local blocks, and classify each local block, and then integrate all the classification results by voting. For each local block, we further divide it into overlapped local patches and assume that these patches lie in a linear subspace. This subspace assumption reflects local structure relationship of the overlapped patches and makes CRC robust for SSPP problem. Motivated by the fact that the entropy of the class probability distribution is a measure about classification confidence, we further apply multi-phase technique to reduce entropy, where useless classes are eliminated after each phase classification. This strategy finally produces a sparse class probability distribution with higher classification confidence. Experimental results show that the proposed method generalizes well to SSPP problem and outperforms many state-of-the-art methods. It also shows strong robustness to the large variation of expression, illumination, little poses variation, occlusion and time variation.

Dynamic behaviors of spreading in generalized Fibonacci time quasiperiodic quantum walks

Quantum walk (QW), the quantum mechanical counterpart of classical random walk, has recently been studied in various fields. The evolution of the discrete time quantum walk can be described as follows: the walker changes its spin state by the coin operator C, then takes one step left or right according to its spin state. For homogeneous quantum walk, the coin operator is independent of time and the standard deviation of the position grows linearly in time. It is quadratically faster than that in the classical random walk. In this work, we numerically study the dynamical behaviors of spreading in a one-dimensional discrete time quasiperiodic quantum walk (DTQQW). The DTQQW is that the coin operator is dependent on time and takes two different coins C() and C() arranged in generalized Fibonacci (GF) sequences. The GF sequences are constructed from A by the recursion relation: AAmBn, BA, for m, n are positive integers. They can be classified into two classes according to the wandering exponent . For 0, they belong to the first class, and for 0, they belong to the second class. For one dimensional system, the behaviors of two classes of GF systems are different either for the electronic spectrum of an electron in quasiperiodic potentials or for the quantum phase transitions of the quasiperiodic spin chains. In this paper, we discuss the cases of two different C operators (C();C()) arranged in GF sequences and find that the spreading behaviors are superdiffusion (the standard deviation of the position ~t; 0:5 1) for the two classes of GF DTQQW. For the second class of GF DTQQW, the exponent values are larger than those of the first class of GF DTQQW in the case of two identical C operators. By exploring the probability distribution in the real space, we find that for the first class of GF DTQQW, the probability distributions are almost the same for different initial states and are similar to the classical Gaussian distribution. For the probability distributions of the second class of GF DTQQW, there are two peaks at the two edges and the height of the two peaks can be different for different initial states. They are similar to the ballistic distribution of the homogeneous quantum walk. Therefore, we conclude that for the first class of GF DTQQW, the spreading behaviors are close to those of the classical random walk ( = 0:5) while for the second class of GF DTQQW, they are close to those of the homogeneous quantum walk ( = 1). This result is quite different from the characteristics of the quantum phase transitions in two classes of GF quasiperiodic quantum spin chains.

Big Numeric Data Classification Using Grid-based Bayesian Inference in the MapReduce Framework

In the current era of data-intensive services, the handling of big data is a crucial issue that affects almost every discipline and industry. In this study, we propose a classification method for large volumes of numeric data, which is implemented in a distributed programming framework, i.e., MapReduce. The proposed method partitions the data space into a grid structure and it then models the probability distributions of classes for grid cells by collecting sufficient statistics using distributed MapReduce tasks. The class labeling of new data is achieved by k-nearest neighbor classification based on Bayesian inference.

Arabic web pages clustering and annotation using semantic class features

To effectively manage the great amount of data on Arabic web pages and to enable the classification of relevant information are very important research problems. Studies on sentiment text mining have been very limited in the Arabic language because they need to involve deep semantic processing. Therefore, in this paper, we aim to retrieve machine-understandable data with the help of a Web content mining technique to detect covert knowledge within these data. We propose an approach to achieve clustering with semantic similarities. This approach comprises integrating k-means document clustering with semantic feature extraction and document vectorization to group Arabic web pages according to semantic similarities and then show the semantic annotation. The document vectorization helps to transform text documents into a semantic class probability distribution or semantic class density. To reach semantic similarities, the approach extracts the semantic class features and integrates them into the similarity weighting schema. The quality of the clustering result has evaluated the use of the purity and the mean intra-cluster distance (MICD) evaluation measures. We have evaluated the proposed approach on a set of common Arabic news web pages. We have acquired favorable clustering results that are effective in minimizing the MICD, expanding the purity and lowering the runtime.

Optimal query-based relevance feedback in medical image retrieval using score fusion-based classification.

In this paper, a new content-based medical image retrieval (CBMIR) framework using an effective classification method and a novel relevance feedback (RF) approach are proposed. For a large-scale database with diverse collection of different modalities, query image classification is inevitable due to firstly, reducing the computational complexity and secondly, increasing influence of data fusion by removing unimportant data and focus on the more valuable information. Hence, we find probability distribution of classes in the database using Gaussian mixture model (GMM) for each feature descriptor and then using the fusion of obtained scores from the dependency probabilities, the most relevant clusters are identified for a given query. Afterwards, visual similarity of query image and images in relevant clusters are calculated. This method is performed separately on all feature descriptors, and then the results are fused together using feature similarity ranking level fusion algorithm. In the RF level, we propose a new approach to find the optimal queries based on relevant images. The main idea is based on density function estimation of positive images and strategy of moving toward the aggregation of estimated density function. The proposed framework has been evaluated on ImageCLEF 2005 database consisting of 10,000 medical X-ray images of 57 semantic classes. The experimental results show that compared with the existing CBMIR systems, our framework obtains the acceptable performance both in the image classification and in the image retrieval by RF.