Prior Data Distribution Research Articles

Enhanced Parameter Estimation of DENsity CLUstEring (DENCLUE) Using Differential Evolution

The task of finding natural groupings within a dataset exploiting proximity of samples is known as clustering, an unsupervised learning approach. Density-based clustering algorithms, which identify arbitrarily shaped clusters using spatial dimensions and neighbourhood aspects, are sensitive to the selection of parameters. For instance, DENsity CLUstEring (DENCLUE)—a density-based clustering algorithm—requires a trial-and-error approach to find suitable parameters for optimal clusters. Earlier attempts to automate the parameter estimation of DENCLUE have been highly dependent either on the choice of prior data distribution (which could vary across datasets) or by fixing one parameter (which might not be optimal) and learning other parameters. This article addresses this challenge by learning the parameters of DENCLUE through the differential evolution optimisation technique without prior data distribution assumptions. Experimental evaluation of the proposed approach demonstrated consistent performance across datasets (synthetic and real datasets) containing clusters of arbitrary shapes. The clustering performance was evaluated using clustering validation metrics (e.g., Silhouette Score, Davies–Bouldin Index and Adjusted Rand Index) as well as qualitative visual analysis when compared with other density-based clustering algorithms, such as DPC, which is based on weighted local density sequences and nearest neighbour assignments (DPCSA) and Variable KDE-based DENCLUE (VDENCLUE).

Graph-Graph Similarity Network.

Graph learning aims to predict the label for an entire graph. Recently, graph neural network (GNN)-based approaches become an essential strand to learning low-dimensional continuous embeddings of entire graphs for graph label prediction. While GNNs explicitly aggregate the neighborhood information and implicitly capture the topological structure for graph representation, they ignore the relationships among graphs. In this article, we propose a graph-graph (G2G) similarity network to tackle the graph learning problem by constructing a SuperGraph through learning the relationships among graphs. Each node in the SuperGraph represents an input graph, and the weights of edges denote the similarity between graphs. By this means, the graph learning task is then transformed into a classical node label propagation problem. Specifically, we use an adversarial autoencoder to align embeddings of all the graphs to a prior data distribution. After the alignment, we design the G2G similarity network to learn the similarity between graphs, which functions as the adjacency matrix of the SuperGraph. By running node label propagation algorithms on the SuperGraph, we can predict the labels of graphs. Experiments on five widely used classification benchmarks and four public regression benchmarks under a fair setting demonstrate the effectiveness of our method.

Use of directed quasi-metric distances for quantifying the information of gene families

A large hindrance to analyzing information in genetic or protein sequence data has been a lack of a mathematical framework for doing so. In this paper, we present a multinomial probability space X as a general foundation for multicategory discrete data, where categories refer to variants/alleles of biosequences. The external information that is infused in order to generate a sample of such data is quantified as a distance on X between the prior distribution of data and the empirical distribution of the sample. A number of distances on X are treated. All of them have an information theoretic interpretation, reflecting the information that the sampling mechanism provides about which variants that have a selective advantage and therefore appear more frequently compared to prior expectations. This includes distances on X based on mutual information, conditional mutual information, active information, and functional information. The functional information distance is singled out as particularly useful. It is simple and has intuitive interpretations in terms of 1) a rejection sampling mechanism, where functional entities are retained, whereas non-functional categories are censored, and 2) evolutionary waiting times. The functional information is also a quasi-metric on X, with information being measured in an asymmetric, mountainous landscape. This quasi-metric property is also retained for a robustified version of the functional information distance that allows for mutations in the sampling mechanism. The functional information quasi-metric has been applied with success on bioinformatics data sets, for proteins and sequence alignment of protein families.

Domain augmentation generalization network for real-time fault diagnosis under unseen working conditions

Recent years have witnessed the successful development of domain adaptation methods to tackle cross-domain fault diagnosis problems. However, these methods require the target domain with a prior data distribution. It limits their application in real-time diagnosis scenarios, where unseen working conditions are often encountered. In this case, the domain adaptation approach is not applicable because the target data are usually unavailable in advance. With this in mind, this paper proposes a domain generalization-based method for intelligent fault diagnosis under unseen working conditions. The core idea is to explore diverse domain-invariant representations while strengthening the feature space's discriminative structure, making the model sufficiently robust to out-of-distribution data and thus generalize well to unseen domains. Specifically, multisource augmentation is developed and combined with adversarial training to boost feature diversity and learn correlations among multiple domains, thereby enhancing the robustness and generalization of feature representations. The sample adaptive screening and weighting strategy is further deployed to dynamically optimize data augmentation and network training to obtain a more discriminative decision boundary. Experimental results of extensive diagnosis tasks built on rolling bearing and gearbox datasets validate the effectiveness and superiority of the proposed method in generalization performance improvement.

Data augmentation based on multiple oversampling fusion for medical image segmentation

A high-performance medical image segmentation model based on deep learning depends on the availability of large amounts of annotated training data. However, it is not trivial to obtain sufficient annotated medical images. Generally, the small size of most tissue lesions, e.g., pulmonary nodules and liver tumours, could worsen the class imbalance problem in medical image segmentation. In this study, we propose a multidimensional data augmentation method combining affine transform and random oversampling. The training data is first expanded by affine transformation combined with random oversampling to improve the prior data distribution of small objects and the diversity of samples. Secondly, class weight balancing is used to avoid having biased networks since the number of background pixels is much higher than the lesion pixels. The class imbalance problem is solved by utilizing weighted cross-entropy loss function during the training of the CNN model. The LUNA16 and LiTS17 datasets were introduced to evaluate the performance of our works, where four deep neural network models, Mask-RCNN, U-Net, SegNet and DeepLabv3+, were adopted for small tissue lesion segmentation in CT images. In addition, the small tissue segmentation performance of the four different deep learning architectures on both datasets could be greatly improved by incorporating the data augmentation strategy. The best pixelwise segmentation performance for both pulmonary nodules and liver tumours was obtained by the Mask-RCNN model, with DSC values of 0.829 and 0.879, respectively, which were similar to those of state-of-the-art methods.

Feature Extraction of Color Symbol Elements in Interior Design Based on Extension Data Mining

In order to improve the indoor color symbol elements in the design of visual communication ability, the interior design color symbol element feature extraction process is needed. We need to put forward a kind of extension data mining based on the interior design color symbol element feature extraction methods, build the visualization visual expression and big data characteristics of the distribution of the interior design of texture color symbol elements in intelligent information collection model, indoor color symbol elements in the design of the texture image and character segmentation, and extraction of texture and color symbol element in interior design edge feature point, based on the texture and edge profile characteristics of the distribution in the design of indoor color symbol elements of texture segmentation and matching, and extract the edge of the color symbol elements in interior design outline feature points. The extracted edge contour feature points of color image in interior design are used as the extended data feature components for texture matching, and the color symbol element feature extraction is realized according to the prior data distribution and extension data mining results. Simulation results show that this method has good adaptive performance in texture matching and feature extraction of color symbol elements in interior design and has a strong texture discrimination ability, which improves the visual communication ability of color symbol elements in interior design.

Targeting uncertainty in smart CPS by confidence-based logic

Since Smart Cyber–Physical Systems (sCPS) are complex and decentralized systems of dynamically cooperating components, architecture-based adaptation is of high importance in their design. In this context, a key challenge is that they typically operate in uncertain environments. Thus, an inherent requirement in sCPS design is the need to deal with the uncertainty of data coming from the environment. Existing approaches often rely on the fact that an adequate model of the environment and/or base probabilities or a prior distribution of data are available. In this paper, we present a specific logic (CB logic), which, based on statistical testing, allows specifying transition guards in architecture-based adaptation without requiring knowledge of the base probabilities or prior knowledge about the data distribution. Applicable in state machines’ transition guards in general, CB logic provides a number of operators over time series that simplify the filtering, resampling, and statistics-backed comparisons of time series, making the application of multiple statistical procedures easy for non-experts. The viability of our approach is illustrated on a running example and a case study demonstrating how CB logic simplifies adaptation triggers. Moreover, a library with a Java and C ++ implementation of CB logic’s key operators is available on GitHub.

A Zero-Shot Learning Method Using Artificial Neural Network for Drift Calibration of Gas Sensor Array

A Zero-Shot Learning Method Using Artificial Neural Network for Drift Calibration of Gas Sensor Array

Fuzzy Support Vector Machine With Relative Density Information for Classifying Imbalanced Data

Fuzzy support vector machine (FSVM) has been combined with class imbalance learning (CIL) strategies to address the problem of classifying skewed data. However, the existing approaches hold several inherent drawbacks, causing the inaccurate prior data distribution estimation, further decreasing the quality of the classification model. To solve this problem, we present a more robust prior data distribution information extraction method named relative density, and two novel FSVM-CIL algorithms based on the relative density information in this paper. In our proposed algorithms, a K-nearest neighbors-based probability density estimation (KNN-PDE) alike strategy is utilized to calculate the relative density of each training instance. In particular, the relative density is irrelevant with the dimensionality of data distribution in feature space, but only reflects the significance of each instance within its class; hence, it is more robust than the absolute distance information. In addition, the relative density can better seize the prior data distribution information, no matter the data distribution is easy or complex. Even for the data with small injunctions or a large class overlap, the relative density information can reflect its details well. We evaluated the proposed algorithms on an amount of synthetic and real-world imbalanced datasets. The results show that our proposed algorithms obviously outperform to some previous work, especially on those datasets with sophisticated distributions.

Structural Regular Multiple Criteria Linear Programming for Classification Problem

Classification problem has attracted an increasing amount of interest. Various classifiers have been proposed in the last decade, such as ANNs, LDA, and SVM. Regular Multiple Criteria Linear Programming (RMCLP) is an effective classification method, which was proposed by Shi and his colleagues and have been applied to handle different real-life data mining problems. In this paper, inspired by the application potential of RMCLP, we propose a novel Structural RMCLP (called SRMCLP) method for classification problem. Unlike RMCLP, SRMCLP is sensitive to the structure of the data distribution and can construct more reasonable classifiers by exploiting these prior data distribution information within classes. The corresponding optimization problem of SRMCLP can be solved by a standard quadratic programming. The effectiveness of the proposed method is demonstrated via experiments on synthetic and available benchmark datasets.

Bayesian Active Remote Sensing Image Classification

In recent years, kernel methods, in particular support vector machines (SVMs), have been successfully introduced to remote sensing image classification. Their properties make them appropriate for dealing with a high number of image features and a low number of available labeled spectra. The introduction of alternative approaches based on (parametric) Bayesian inference has been quite scarce in the more recent years. Assuming a particular prior data distribution may lead to poor results in remote sensing problems because of the specificities and complexity of the data. In this context, the emerging field of nonparametric Bayesian methods constitutes a proper theoretical framework to tackle the remote sensing image classification problem. This paper exploits the Bayesian modeling and inference paradigm to tackle the problem of kernel-based remote sensing image classification. This Bayesian methodology is appropriate for both finite- and infinite-dimensional feature spaces. The particular problem of active learning is addressed by proposing an incremental/active learning approach based on three different approaches: 1) the maximum differential of entropies; 2) the minimum distance to decision boundary; and 3) the minimum normalized distance. Parameters are estimated by using the evidence Bayesian approach, the kernel trick, and the marginal distribution of the observations instead of the posterior distribution of the adaptive parameters. This approach allows us to deal with infinite-dimensional feature spaces. The proposed approach is tested on the challenging problem of urban monitoring from multispectral and synthetic aperture radar data and in multiclass land cover classification of hyperspectral images, in both purely supervised and active learning settings. Similar results are obtained when compared to SVMs in the supervised mode, with the advantage of providing posterior estimates for classification and automatic parameter learning. Comparison with random sampling as well as standard active learning methods such as margin sampling and entropy-query-by-bagging reveals a systematic overall accuracy gain and faster convergence with the number of queries.

Rough annealing by two-step clustering, with application to neuronal signals

To accomplish analyses on the properties of neuronal populations it is mandatory that each unit activity is identified within the overall noise background and the other unit signals merged in the same trace. The problem, addressed as a clustering one, is particularly difficult as no assumption can be made on the prior data distribution. We propose an algorithm that achieves this goal by a two-phase agglomerative hierarchical clustering. First, an inflated estimation (overly) of the number of clusters is cast down and, by a maximum entropy principle (MEP) approach, is made to collapse towards an arrangement near natural ones. In the second step consecutive partitions are created by merging, two at time previously aggregated partitions, according to similarity criteria, in order to reveal a cluster solution. The procedure makes no assumptions about data distributions and guarantees high robustness with respect to noise. An application on real data out of multiple unit recordings from spinal cord neurons of mixed gas-anaesthetized rats is presented.