Toy Dataset Research Articles

Privacy-preserving quality control of neuroimaging datasets in federated environments.

Privacy concerns for rare disease data, institutional or IRB policies, access to local computational or storage resources or download capabilities are among the reasons that may preclude analyses that pool data to a single site. A growing number of multisite projects and consortia were formed to function in the federated environment to conduct productive research under constraints of this kind. In this scenario, a quality control tool that visualizes decentralized data in its entirety via global aggregation of local computations is especially important, as it would allow the screening of samples that cannot be jointly evaluated otherwise. To solve this issue, we present two algorithms: decentralized data stochastic neighbor embedding, dSNE, and its differentially private counterpart, DP‐dSNE. We leverage publicly available datasets to simultaneously map data samples located at different sites according to their similarities. Even though the data never leaves the individual sites, dSNE does not provide any formal privacy guarantees. To overcome that, we rely on differential privacy: a formal mathematical guarantee that protects individuals from being identified as contributors to a dataset. We implement DP‐dSNE with AdaCliP, a method recently proposed to add less noise to the gradients per iteration. We introduce metrics for measuring the embedding quality and validate our algorithms on these metrics against their centralized counterpart on two toy datasets. Our validation on six multisite neuroimaging datasets shows promising results for the quality control tasks of visualization and outlier detection, highlighting the potential of our private, decentralized visualization approach.

Bayesian Gaussian process factor analysis with copula for count data

Gaussian process factor analysis models aim to capture the latent trajectories that generated observed multivariate data by factorizing the observations. These methods assume conditional independence of observations, given the factor scores and loadings. For continuous data this restrictive assumption can be resolved by using a non-diagonal covariance matrix. However, this does not work for count data. We propose a new model which pairs a negative binomial Gaussian process factor analysis with a Gaussian copula to find latent trajectories and also accounts for the residual covariance not captured by these trajectories. We provide a fully Bayesian implementation of the model and use augmented likelihood for inference with Hamiltonian Monte Carlo. We compare the proposed method to other Gaussian process factor analysis models on 12 toy data sets, finding latent qualities of NBA teams, and forecasting disease counts. The results show that the proposed method is useful for latent structure extraction and out-of-sample prediction of multivariate counts.

Clustering consumers based on product discrimination in check-all-that-apply (CATA) data

Consumers can be clustered based on their product-related check-all-that-apply (CATA) responses. We identify two paradoxes that can occur if these clusters are derived from conventional similarity coefficients. The first paradox is that clustering similar consumers can nullify within-cluster sensory differentiation of products. The second paradox is that consumers who check many attributes yet disagree can be clustered together, whereas consumers who check fewer attributes without disagreement can be split into different clusters. After illustrating these paradoxes with toy data sets, we propose “b-cluster analysis”, in which consumers are clustered according to how they differentiate products. We define performance metrics to compare cluster analysis solutions. By design, b-cluster analysis is expected to give different results than CLUSCATA, since the objective of CLUSCATA is to cluster consumers who characterize products similarly, not according to how they differentiate products. We apply b-cluster analysis to the same toy data sets and show that the identified paradoxes do not occur. Then we apply both b-cluster analysis and CLUSCATA to a real consumer data set. We find that the b-cluster analysis solutions have better within-cluster sensory differentiation, better sensory discrimination, and less redundant clusters than CLUSCATA solutions. To investigate the sensitivity of b-cluster analysis to the initial (random) cluster membership allocations, we obtained 10,000 two-cluster solutions, each initialized with a different random partitioning of consumers. The best solution, which retains the most sensory differentiation, was observed in 21.4% of the runs. As a best practice, we recommend running b-cluster analysis several times and choosing the best solution. The proposed b-cluster analysis approach can be extended to other types of sensometric data and may have applications in other fields.

Few-Shot Anomaly Detection Using Deep Generative Models for Grouped Data

There exists a great demand for automatic anomaly detection in industrial world. The anomaly has been defined as a group of samples that rarely or never appears. Given a type of products, one has to collect numerous samples and train an anomaly detector. When one diverts a model trained with old types of products with sufficient inventory to the new type, one can detect anomalies of the new type before a production line is established. However, because of the definition of the anomaly, a typical anomaly detector considers the new type of products anomalous even if it is consistent with the standard. Given the above practical demand, this study propose a novel problem setting, few-shot anomaly detection, where an anomaly detector trained in source domains is adapted to a small set of target samples without full retraining. Then, we tackle this problem using a hierarchical probabilistic model based on deep learning. Our empirical results on toy and real-world datasets demonstrate that the proposed model detects anomalies in a small set of target samples successfully.

Denoising EEG Signals for Real-World BCI Applications Using GANs.

As a measure of the brain's electrical activity, electroencephalography (EEG) is the primary signal of interest for brain-computer-interfaces (BCI). BCIs offer a communication pathway between a brain and an external device, translating thought into action with suitable processing. EEG data is the most common signal source for such technologies. However, artefacts induced in BCIs in the real-world context can severely degrade their performance relative to their in-laboratory performance. In most cases, the recorded signals are so heavily corrupted by noise that they are unusable and restrict BCI's broader applicability. To realise the use of portable BCIs capable of high-quality performance in a real-world setting, we use Generative Adversarial Networks (GANs) that can adopt both supervised and unsupervised learning approaches. Although our approach is supervised, the same model can be used for unsupervised tasks such as data augmentation/imputation in the low resource setting. Exploiting recent advancements in Generative Adversarial Networks (GAN), we construct a pipeline capable of denoising artefacts from EEG time series data. In the case of denoising data, it maps noisy EEG signals to clean EEG signals, given the nature of the respective artefact. We demonstrate the capability of our network on a toy dataset and a benchmark EEG dataset developed explicitly for deep learning denoising techniques. Our datasets consist of an artificially added mains noise (50/60 Hz) artefact dataset and an open-source EEG benchmark dataset with two artificially added artefacts. Artificially inducing myogenic and ocular artefacts for the benchmark dataset allows us to present qualitative and quantitative evidence of the GANs denoising capabilities and rank it among the current gold standard deep learning EEG denoising techniques. We show the power spectral density (PSD), signal-to-noise ratio (SNR), and other classical time series similarity measures for quantitative metrics and compare our model to those previously used in the literature. To our knowledge, this framework is the first example of a GAN capable of EEG artefact removal and generalisable to more than one artefact type. Our model has provided a competitive performance in advancing the state-of-the-art deep learning EEG denoising techniques. Furthermore, given the integration of AI into wearable technology, our method would allow for portable EEG devices with less noisy and more stable brain signals.

Low-Rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation

Learning generative probabilistic models is a core problem in machine learning, which presents significant challenges due to the curse of dimensionality. This paper proposes a joint dimensionality reduction and non-parametric density estimation framework, using a novel estimator that can explicitly capture the underlying distribution of appropriate reduced-dimension representations of the input data. The idea is to jointly design a nonlinear dimensionality reducing auto-encoder to model the training data in terms of a parsimonious set of latent random variables, and learn a canonical low-rank tensor model of the joint distribution of the latent variables in the Fourier domain. The proposed latent density model is non-parametric and universal, as opposed to the predefined prior that is assumed in variational auto-encoders. Joint optimization of the auto-encoder and the latent density estimator is pursued via a formulation which learns both by minimizing a combination of the negative log-likelihood in the latent domain and the auto-encoder reconstruction loss. We demonstrate that the proposed model achieves very promising results on toy, tabular, and image datasets on regression tasks, sampling, and anomaly detection.

Low-Rank Characteristic Tensor Density Estimation Part I: Foundations

Effective non-parametric density estimation is a key challenge in high-dimensional multivariate data analysis. In this paper,we propose a novel approach that builds upon tensor factorization tools. Any multivariate density can be represented by its characteristic function, via the Fourier transform. If the sought density is compactly supported, then its characteristic function can be approximated, within controllable error, by a finite tensor of leading Fourier coefficients, whose size de-pends on the smoothness of the underlying density. This tensor can be naturally estimated from observed realizations of the random vector of interest, via sample averaging. In order to circumvent the curse of dimensionality, we introduce a low-rank model of this characteristic tensor, which significantly improves the density estimate especially for high-dimensional data and/or in the sample-starved regime. By virtue of uniqueness of low-rank tensor decomposition, under certain conditions, our method enables learning the true data-generating distribution. We demonstrate the very promising performance of the proposed method using several measured datasets.

Scikit-Dimension: A Python Package for Intrinsic Dimension Estimation.

Dealing with uncertainty in applications of machine learning to real-life data critically depends on the knowledge of intrinsic dimensionality (ID). A number of methods have been suggested for the purpose of estimating ID, but no standard package to easily apply them one by one or all at once has been implemented in Python. This technical note introduces scikit-dimension, an open-source Python package for intrinsic dimension estimation. The scikit-dimension package provides a uniform implementation of most of the known ID estimators based on the scikit-learn application programming interface to evaluate the global and local intrinsic dimension, as well as generators of synthetic toy and benchmark datasets widespread in the literature. The package is developed with tools assessing the code quality, coverage, unit testing and continuous integration. We briefly describe the package and demonstrate its use in a large-scale (more than 500 datasets) benchmarking of methods for ID estimation for real-life and synthetic data.

Learning Ordinal Embedding from Sets.

Ordinal embedding is the task of computing a meaningful multidimensional representation of objects, for which only qualitative constraints on their distance functions are known. In particular, we consider comparisons of the form “Which object from the pair is more similar to object i?”. In this paper, we generalize this framework to the case where the ordinal constraints are not given at the level of individual points, but at the level of sets, and propose a distributional triplet embedding approach in a scalable learning framework. We show that the query complexity of our approach is on par with the single-item approach. Without having access to features of the items to be embedded, we show the applicability of our model on toy datasets for the task of reconstruction and demonstrate the validity of the obtained embeddings in experiments on synthetic and real-world datasets.

Graph compression based on transitivity for neighborhood query

In recent years, many graph compression methods have been introduced. One successful category of them is based on local decompression designed to answer neighborhood queries. These techniques mainly rely on local similarities of vertices. Besides, their performance is usually a function of graph sparsity. The proposed approach, in this paper, is a lossy compression technique used to answer neighborhood queries with a more general precondition, called transitivity. The output of this method is a sparse graph optimized to keep original adjacent vertices, in at most 2-distance from each other and vice versa. In other words, by traversing a compressed graph by depth of 2, from any desired vertex, its original adjacency list is reconstructed, with an acceptable error. This paper models an optimization problem to solve the inverse problem of finding the best compressed graph in order to minimize the reconstruction error. Then, this NP problem is approximated by a heuristic with a low degree polynomial time-complexity near to the complexity of the forward problem. The results of applying the proposed method on toy and real datasets are compared with the state of the art that improves compression ratio and performance with an acceptable query response time.

A Bayesian approach to time-varying latent strengths in pairwise comparisons.

The famous Bradley-Terry model for pairwise comparisons is widely used for ranking objects and is often applied to sports data. In this paper we extend the Bradley-Terry model by allowing time-varying latent strengths of compared objects. The time component is modelled with barycentric rational interpolation and Gaussian processes. We also allow for the inclusion of additional information in the form of outcome probabilities. Our models are evaluated and compared on toy data set and real sports data from ATP tennis matches and NBA games. We demonstrated that using Gaussian processes is advantageous compared to barycentric rational interpolation as they are more flexible to model discontinuities and are less sensitive to initial parameters settings. However, all investigated models proved to be robust to over-fitting and perform well with situations of volatile and of constant latent strengths. When using barycentric rational interpolation it has turned out that applying Bayesian approach gives better results than by using MLE. Performance of the models is further improved by incorporating the outcome probabilities.

Disentangled Representation Learning in Real-World Image Datasets via Image Segmentation Prior

We propose a novel method that can learn easy-to-interpret latent representations in real-world image datasets using a VAE-based model by splitting an image into several disjoint regions. Our method performs object-wise disentanglement by exploiting image segmentation and alpha compositing. With remarkable results obtained by unsupervised disentanglement methods for toy datasets, recent studies have tackled challenging disentanglement for real-world image datasets. However, these methods involve deviations from the standard VAE architecture, which has favorable disentanglement properties. Thus, for disentanglement in images of real-world image datasets with preservation of the VAE backbone, we designed an encoder and a decoder that embed an image into disjoint sets of latent variables corresponding to objects. The encoder includes a pre-trained image segmentation network, which allows our model to focus only on representation learning while adopting image segmentation as an inductive bias. Evaluations using real-world image datasets, CelebA and Stanford Cars, showed that our method achieves improved disentanglement and transferability.

Deep Generative Model Using Unregularized Score for Anomaly Detection With Heterogeneous Complexity.

Accurate and automated detection of anomalous samples in an image dataset can be accomplished with a probabilistic model. Such images have heterogeneous complexity, however, and a probabilistic model tends to overlook simply shaped objects with small anomalies. The reason is that a probabilistic model assigns undesirable lower likelihoods to complexly shaped objects, which are nevertheless consistent with the current set standards. This difficulty is critical, especially for a defect detection task, where the anomaly can be a small scratch or grime. To overcome this difficulty, we propose an unregularized score for deep generative models (DGMs). We found that the regularization terms of the DGMs considerably influence the anomaly score depending on the complexity of the samples. By removing these terms, we obtain an unregularized score, which we evaluated on toy datasets, two in-house manufacturing datasets, and on open manufacturing and medical datasets. The empirical results demonstrate that the unregularized score is robust to the apparent complexity of given samples and detects anomalies selectively.

A semi-supervised learning algorithm via adaptive Laplacian graph

Many semi-supervised learning methods have been developed in recent years, especially graph-based approaches, which have achieved satisfactory performance in the practical applications. There are two points that need to be noticed. Firstly, the quality of the graph directly affects the final classification accuracy. However, graph-based algorithms mostly use k-Nearest Neighbor to construct the graph. And the directly constructed graph is inaccurate due to outliers and erroneous features in the data. Secondly, the amount of labeled data is a small part of all data. It cannot be guaranteed that all categories of data are included in the labeled data and the labels of data are not totally correct in practice. To address the aforementioned problems, we propose a new graph-based semi-supervised method named ALGSSL via adaptive Laplacian graph. In the algorithm, we adaptively update the graph to reduce the sensitiveness of the construction of initial graph. Meanwhile, we use the regularization parameters to set confidence on existing labels, which can reduce the impact of the error labels on the result and discover the new category. Experiments on three toy datasets and nine benchmark datasets demonstrate the proposed method can achieve good performance.

CLoNe: automated clustering based on local density neighborhoods for application to biomolecular structural ensembles.

MotivationProteins are intrinsically dynamic entities. Flexibility sampling methods, such as molecular dynamics or those arising from integrative modeling strategies, are now commonplace and enable the study of molecular conformational landscapes in many contexts. Resulting structural ensembles increase in size as technological and algorithmic advancements take place, making their analysis increasingly demanding. In this regard, cluster analysis remains a go-to approach for their classification. However, many state-of-the-art algorithms are restricted to specific cluster properties. Combined with tedious parameter fine-tuning, cluster analysis of protein structural ensembles suffers from the lack of a generally applicable and easy to use clustering scheme.ResultsWe present CLoNe, an original Python-based clustering scheme that builds on the Density Peaks algorithm of Rodriguez and Laio. CLoNe relies on a probabilistic analysis of local density distributions derived from nearest neighbors to find relevant clusters regardless of cluster shape, size, distribution and amount. We show its capabilities on many toy datasets with properties otherwise dividing state-of-the-art approaches and improves on the original algorithm in key aspects. Applied to structural ensembles, CLoNe was able to extract meaningful conformations from membrane binding events and ligand-binding pocket opening as well as identify dominant dimerization motifs or inter-domain organization. CLoNe additionally saves clusters as individual trajectories for further analysis and provides scripts for automated use with molecular visualization software.Availability and implementation www.epfl.ch/labs/lbm/resources, github.com/LBM-EPFL/CLoNe.Supplementary information Supplementary data are available at Bioinformatics online.

New families of stable simplicial filtration functors

When quantifying the topological properties of a metric dataset using persistent homology, the first step is to produce a simplicial filtration on the data. The Čech and Vietoris-Rips filtrations are two of the workhorses of persistent homology, but over time, various other filtrations which capture different properties of data or have different computational burdens have been introduced. Towards a program of characterizing all the possible simplicial filtrations on a metric dataset, we introduce and develop the framework of valuation-induced stable filtration functors. This framework is based on the concept of curvature sets due to Gromov, and encapsulates the Vietoris-Rips and various other filtrations while simultaneously providing a model for generating families of novel filtration functors that capture diverse features present in datasets. We further extend this foundation by incorporating the notion of basepoint-dependent filtration functors and proving the associated functoriality and stability properties. This rich theoretical framework provides a unifying language for various extant simplicial filtrations, and is also a mechanism for generating arbitrarily large families of novel filtration functors with control over basepoint dependence/independence as well as the locality of the filtration. We exemplify our constructions on both toy datasets and on 3D shapes from a publicly available shape database. Our paper is accompanied by a Matlab software package incorporating an interactive platform for visualizing and testing new filtrations on datasets.

Orthogonal neighborhood preserving discriminant analysis with patch embedding for face recognition

Intuitively, all facial images of a person are located on or near a manifold in the high-dimensional image space, and the process of face recognition can be regarded as the recovery process of multiple low-dimensional manifolds. To preserve the manifold structure information of intra-class samples after dimensionality reduction, we proposed a patch-based multi-manifold orthogonal neighborhood-preserving discriminant analysis algorithm, namely ONPDA. From the perspective of path alignment, we consider the intra-class compactness, intra-class structure and inter-class separability simultaneously. Moreover, we infuse intra-class structure information described by the sample reconstruction into intra-class compactness loss, considering the compactness of two reconstruction groups instead of sample pairs in the same class. By analyzing the relationship between the projection direction and the maximum inter-class margin, we select the samples that should participate in the inter-class separability on the patch. Meanwhile, a fast orthogonalization method is performed to obtain the orthogonal projection matrix. Besides, we perform ONPDA in reproducing kernel Hilbert space which gives rise to nonlinear maps, resulting in the kernel ONPDA (KONPDA). Experimental results compared with some state-of-the-art methods on a toy dataset and several benchmark face image databases demonstrate the effectiveness of ONPDA and KONPDA.

Twin support vector machine based on adjustable large margin distribution for pattern classification

This paper researches the value of the margin distribution in binary classifier. The central idea of large margin distribution machine (LDM) is to optimize the margin distribution, such as maximizing the margin mean and minimizing the margin variance. Compared to support vector machine (SVM), LDM demonstrates the good generalization performance. In order to improve the generalization performance of twin support vector machine (TSVM), a twin support vector machine based on adjustable large margin distribution (ALD-TSVM) is proposed in this paper. Firstly, the margin distribution is redefined to construct a pair of adjustable supporting hyperplanes. Then, the redefined margin distribution is introduced onto TSVM to obtain the models of ALD-TSVM, including linear case and nonlinear case. ALD-TSVM is a general learning method which can be used in any place where TSVM and LDM can be applied. Finally, the novel method is compared with other classification algorithms by doing experiments on toy dataset, UCI datasets and image datasets. The experimental results show that ALD-TSVM obtains better classification performance.

OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization

Exploring the potential of GANs for unsupervised disentanglement learning, this paper proposes a novel GAN-based disentanglement framework with One-Hot Sampling and Orthogonal Regularization (OOGAN). While previous works mostly attempt to tackle disentanglement learning through VAE and seek to implicitly minimize the Total Correlation (TC) objective with various sorts of approximation methods, we show that GANs have a natural advantage in disentangling with an alternating latent variable (noise) sampling method that is straightforward and robust. Furthermore, we provide a brand-new perspective on designing the structure of the generator and discriminator, demonstrating that a minor structural change and an orthogonal regularization on model weights entails an improved disentanglement. Instead of experimenting on simple toy datasets, we conduct experiments on higher-resolution images and show that OOGAN greatly pushes the boundary of unsupervised disentanglement.

Discrimination-Aware Projected Matrix Factorization

Non-negative Matrix Factorization (NMF) has been one of the most popular clustering techniques in machine leaning, and involves various real-world applications. Most existing works perform matrix factorization on high-dimensional data directly. However, the intrinsic data structure is always hidden within the low-dimensional subspace. And, the redundant features within the input space may affect the final result adversely. In this paper, a new unsupervised matrix factorization method, Discrimination-aware Projected Matrix Factorization (DPMF), is proposed for data clustering. The main contributions are threefold: (1) The linear discriminant analysis is jointly incorporated into the unsupervised matrix factorization framework, so the clustering can be accomplished in the discriminant subspace. (2) The manifold regularization is introduced to perceive the geometric information, and the ${\ell _{2,1}}$ l 2 , 1 -norm is utilized to improve the robustness. (3) An efficient optimization algorithm is designed to solve the proposed problem with proved convergence. Experimental results on one toy dataset and eight real-world benchmarks show the effectiveness of the proposed method.