Mises-Fisher Distribution Research Articles

A Novel Approach to Clustering Accelerometer Data for Application in Passive Predictions of Changes in Depression Severity.

The treatment of mood disorders, which can become a lifelong process, varies widely in efficacy between individuals. Most options to monitor mood rely on subjective self-reports and clinical visits, which can be burdensome and may not portray an accurate representation of what the individual is experiencing. A passive method to monitor mood could be a useful tool for those with these disorders. Some previously proposed models utilized sensors from smartphones and wearables, such as the accelerometer. This study examined a novel approach of processing accelerometer data collected from smartphones only while participants of the open-science branch of the BiAffect study were typing. The data were modeled by von Mises-Fisher distributions and weighted networks to identify clusters relating to different typing positions unique for each participant. Longitudinal features were derived from the clustered data and used in machine learning models to predict clinically relevant changes in depression from clinical and typing measures. Model accuracy was approximately 95%, with 97% area under the ROC curve (AUC). The accelerometer features outperformed the vast majority of clinical and typing features, which suggested that this new approach to analyzing accelerometer data could contribute towards unobtrusive detection of changes in depression severity without the need for clinical input.

Similar classes latent distribution modelling-based oversampling method for imbalanced image classification

Learning an unbiased classifier from imbalanced image datasets is challenging since the classifier may be strongly biased toward the majority class. To address this issue, some generative model-based oversampling methods have been proposed. However, most of these methods pay little attention to boundary samples, which may contribute tiny to learning an unbiased classifier. In this paper, we focus on boundary samples and propose a similar classes latent distribution modelling-based oversampling method. Specifically, first, we model each class as different von Mises–Fisher distributions, thereby aligning feature learning with the class distributions. Furthermore, we develop a distance minimization loss function, which makes latent representations from similar classes close to each other. In this way, the generator can capture more shared features during training. In addition, we propose a boundary sampling strategy, which uses latent variables near the decision boundary to generate boundary samples. These samples expand the minority decision region and reshape the decision boundary. Experiments on four imbalanced image datasets show that the proposed method achieves promising performance in terms of Recall, Precision, F1-score, and G-mean.

Enhanced hyperalignment via spatial prior information.

Functional alignment between subjects is an important assumption of functional magnetic resonance imaging (fMRI) group-level analysis. However, it is often violated in practice, even after alignment to a standard anatomical template. Hyperalignment, based on sequential Procrustes orthogonal transformations, has been proposed as a method of aligning shared functional information into a common high-dimensional space and thereby improving inter-subject analysis. Though successful, current hyperalignment algorithms have a number of shortcomings, including difficulties interpreting the transformations, a lack of uniqueness of the procedure, and difficulties performing whole-brain analysis. To resolve these issues, we propose the ProMises (Procrustes von Mises-Fisher) model. We reformulate functional alignment as a statistical model and impose a prior distribution on the orthogonal parameters (the von Mises-Fisher distribution). This allows for the embedding of anatomical information into the estimation procedure by penalizing the contribution of spatially distant voxels when creating the shared functional high-dimensional space. Importantly, the transformations, aligned images, and related results are all unique. In addition, the proposed method allows for efficient whole-brain functional alignment. In simulations and application to data from four fMRI studies we find that ProMises improves inter-subject classification in terms of between-subject accuracy and interpretability compared to standard hyperalignment algorithms.

A Tractable, Transferable, and Empirically Consistent Fibrous Biomaterial Model

Stochastic modeling is a useful approach for modeling fibrous materials that attempts to recreate fibrous materials’ structure using statistical data. However, several issues remain to be resolved in the stochastic modeling of fibrous materials—for example, estimating 3D fiber orientation distributions from 2D data, achieving the desired fiber tortuosity distributions, and dealing with fiber–fiber penetration. This work proposes innovative methods to (1) create a mapping from 2D fiber orientation data to 3D fiber orientation probability distributions, and vice versa; and (2) provide a means to select parameters de novo for random walks employing the popularized von Mises–Fisher distribution given that the desired tortuosity of the path is known. The proposed methods are incorporated alongside previously developed stochastic modeling techniques to simulate fiber network structures. First, fiber orientation distributions vary significantly depending on how a fibrous material is formed, and projection distortion affects the measurement of fiber orientation distributions when reported as 2D data such as histograms or polar plots. Relationships are developed to estimate 3D fiber orientation distributions from 2D data, accounting for projection distortion and the variety of orientation distributions observed in fibrous materials. We show that without correcting for projection distortion, fiber orientation distribution parameters could have errors of up to 100%. Second, in stochastic modeling, fiber tortuosity is usually treated with random walks, but no relationship is available for choosing random walk inputs to generate a desired fiber tortuosity. Relationships are also developed to relate the input parameters of von Mises–Fisher random walks to the expected tortuosity of the generated path—a necessary link to modeling fiber tortuosity distributions tractably and with empirical consistency. Using the developed relationships, we show that modeling of tortuous fibers from a distribution could be sped up by ~1200-fold and the uncertainty of selecting appropriate parameters could be eliminated. Third, randomly placing fibers in a simulation domain inevitably results in fiber–fiber penetration, and correcting this issue requires changes to the simulated fibrous material structure through non-penetration conditions. No thorough remedy can be offered here, but we statistically quantify the effects of enforcing non-penetration conditions on the fiber shape and orientation changes as well as the overall fibrous material model. This work offers tractable and transferable methods for treating fiber orientation and tortuosity that allow for empirical consistency in the stochastic modeling of fibrous materials.

Enhancing fracture network characterization: A data-driven, outcrop-based analysis

We utilize a pixel-based fracture detection algorithm to digitize 80 published outcrop maps of different scales at different locations. The key fracture properties, including fracture lengths, orientations, intensities, topological structures, clusters, and flow, are analyzed. Our findings provide significant justifications for statistical distributions used in SDFN modelings. We find that fracture lengths follow multiple (instead of single) power-law distributions with varying exponents. Large fractures tend to have large exponents, possibly because of a small coalescence probability. Most small-scale natural fracture networks have scattered orientations, corresponding to a small κ value (κ<3) in a von Mises–Fisher distribution. Large fracture systems analyzed in this research usually have more concentrated orientations with large κ values. Fracture intensities are spatially clustered at all scales. A fractal spatial density distribution, which introduces clustered fracture positions, can better capture the spatial clustering than a uniform distribution. Natural fracture networks usually have a significant proportion of T-type nodes, which is unavailable in conventional SDFN models. Thus, a rule-based algorithm is necessary to mimic the fracture growth and form T-type nodes. Most outcrop maps show good topological connectivity. However, sealing patterns and stress impact must be considered to evaluate the hydraulic conductivity of fracture networks.

Safety Monitoring of Neural Networks Using Unsupervised Feature Learning and Novelty Estimation

Neural networks are currently suggested to be implemented in several different driving functions of autonomous vehicles. While showing promising results the drawback lies in the difficulty of safety verification and ensuring operation as intended. The aim of this paper is to increase safety when using neural networks, by proposing a monitoring framework based on novelty estimation of incoming driving data. The idea is to use unsupervised instance discrimination to learn a similarity measure across ego-vehicle camera images. By estimating a von Mises-Fisher distribution of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">expected</i> ego-camera images they can be compared with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unexpected</i> novel images. A novelty measurement is inferred through the likelihood of test frames belonging to the expected distribution. The suggested method provides competitive results to several other novelty or anomaly detection algorithms on the CIFAR-10 and CIFAR-100 datasets. It also shows promising results on real world driving scenarios by distinguishing novel driving scenes from the training data of BDD100 k. Applied on the identical training-test data split, the method is also able to predict the performance profile of a segmentation network. Finally, examples are provided on how this method can be extended to find novel segments in images.

Penalized maximum likelihood estimator for mixture of von Mises–Fisher distributions

The von Mises–Fisher distribution is one of the most widely used probability distributions to describe directional data. Finite mixtures of von Mises–Fisher distributions have found numerous applications. However, the likelihood function for the finite mixture of von Mises–Fisher distributions is unbounded and consequently the maximum likelihood estimation is not well defined. To address the problem of likelihood degeneracy, we consider a penalized maximum likelihood approach whereby a penalty function is incorporated. We prove strong consistency of the resulting estimator. An Expectation–Maximization algorithm for the penalized likelihood function is developed and experiments are performed to examine its performance.

Mixture of von Mises-Fisher distribution with sparse prototypes

Mixtures of von Mises-Fisher distributions can be used to cluster data on the unit hypersphere. This is particularly adapted for high-dimensional directional data such as texts. We propose in this article to estimate a von Mises mixture using a l1 penalized likelihood. This leads to sparse prototypes that improve clustering interpretability. We introduce an expectation–maximisation (EM) algorithm for this estimation and explore the trade-off between the sparsity term and the likelihood one with a path following algorithm. The model’s behaviour is studied on simulated data and, we show the advantages of the approach on real data benchmark. We also introduce a new data set on financial reports and exhibit the benefits of our method for exploratory analysis.

Modified Pólya-Gamma data augmentation for Bayesian analysis of directional data

In this work, we develop new data augmentation algorithms for Bayesian analysis of directional data using the von Mises-Fisher distribution in arbitrary dimensions. The approach leads to a new class of distributions, called the Modified Pólya-Gamma distribution, which we construct in detail. The proposed data augmentation strategies circumvent the need for analytic approximations to integration, numerical integration, or Metropolis-Hastings for the corresponding posterior inference. Simulations and real data examples are presented to demonstrate the applicability and to apprise the performance of the proposed procedures.

Procrustes Analysis for High-Dimensional Data

The Procrustes-based perturbation model (Goodall in J R Stat Soc Ser B Methodol 53(2):285–321, 1991) allows minimization of the Frobenius distance between matrices by similarity transformation. However, it suffers from non-identifiability, critical interpretation of the transformed matrices, and inapplicability in high-dimensional data. We provide an extension of the perturbation model focused on the high-dimensional data framework, called the ProMises (Procrustes von Mises–Fisher) model. The ill-posed and interpretability problems are solved by imposing a proper prior distribution for the orthogonal matrix parameter (i.e., the von Mises–Fisher distribution) which is a conjugate prior, resulting in a fast estimation process. Furthermore, we present the Efficient ProMises model for the high-dimensional framework, useful in neuroimaging, where the problem has much more than three dimensions. We found a great improvement in functional magnetic resonance imaging connectivity analysis because the ProMises model permits incorporation of topological brain information in the alignment’s estimation process.

Parameter estimation and model-based clustering with spherical normal distribution on the unit hypersphere

In directional statistics, the von Mises-Fisher (vMF) distribution is one of the most basic and popular probability distributions for data on the unit hypersphere. Recently, the spherical normal (SN) distribution was proposed as an intrinsic counterpart to the vMF distribution by replacing the standard Euclidean norm with the great-circle distance, which is length of the shortest path joining two points on the unit sphere. Focusing on an isotropic version of SN distribution, it is shown that maximum likelihood estimators uniquely exist under mild support conditions. Since no analytic formula are available for the estimation, efficient numerical routines are proposed for parameter estimation. The estimation is considered in a general setting where non-negative weights are assigned to observations. This leads to a more interesting contribution for model-based clustering on the unit hypersphere by finite mixture model with SN distributions. Efficiency of optimization-based estimation procedures and effectiveness of SN mixture model are validated using simulated and real data examples.

A New Method Based on the von Mises-Fisher Distribution Shows that a Minority of Liver-Localized CD8 T Cells Display Hard-To-Detect Attraction to Plasmodium-Infected Hepatocytes.

Malaria is a disease caused by Plasmodium parasites, resulting in over 200 million infections and 400,000 deaths every year. A critical step of malaria infection is when sporozoites, injected by mosquitoes, travel to the liver and form liver stages. Malaria vaccine candidates which induce large numbers of malaria-specific CD8 T cells in mice are able to eliminate all liver stages, preventing fulminant malaria. However, how CD8 T cells find all parasites in 48 h of the liver stage lifespan is not well understood. Using intravital microscopy of murine livers, we generated unique data on T cell search for malaria liver stages within a few hours after infection. To detect attraction of T cells to an infection site, we used the von Mises-Fisher distribution in 3D, similar to the 2D von Mises distribution previously used in ecology. Our results suggest that the vast majority (70–95%) of malaria-specific and non-specific liver-localized CD8 T cells did not display attraction towards the infection site, suggesting that the search for malaria liver stages occurs randomly. However, a small fraction (15–20%) displayed weak but detectable attraction towards parasites which already had been surrounded by several T cells. We found that speeds and turning angles correlated with attraction, suggesting that understanding mechanisms that determine the speed of T cell movement in the liver may improve the efficacy of future T cell-based vaccines. Stochastic simulations suggest that a small movement bias towards the parasite dramatically reduces the number of CD8 T cells needed to eliminate all malaria liver stages, but to detect such attraction by individual cells requires data from long imaging experiments which are not currently feasible. Importantly, as far as we know this is the first demonstration of how activated/memory CD8 T cells might search for the pathogen in nonlymphoid tissues a few hours after infection. We have also established a framework for how attraction of individual T cells towards a location in 3D can be rigorously evaluated.

Oblique decision tree induction by cross-entropy optimization based on the von Mises–Fisher distribution

Oblique decision trees recursively divide the feature space by using splits based on linear combinations of attributes. Compared to their univariate counterparts, which only use a single attribute per split, they are often smaller and more accurate. A common approach to learn decision trees is by iteratively introducing splits on a training set in a top–down manner, yet determining a single optimal oblique split is in general computationally intractable. Therefore, one has to rely on heuristics to find near-optimal splits. In this paper, we adapt the cross-entropy optimization method to tackle this problem. The approach is motivated geometrically by the observation that equivalent oblique splits can be interpreted as connected regions on a unit hypersphere which are defined by the samples in the training data. In each iteration, the algorithm samples multiple candidate solutions from this hypersphere using the von Mises–Fisher distribution which is parameterized by a mean direction and a concentration parameter. These parameters are then updated based on the best performing samples such that when the algorithm terminates a high probability mass is assigned to a region of near-optimal solutions. Our experimental results show that the proposed method is well-suited for the induction of compact and accurate oblique decision trees in a small amount of time.

Variational Bayesian Orthogonal Nonnegative Matrix Factorization Over the Stiefel Manifold.

Nonnegative matrix factorization (NMF) is one of the best-known multivariate data analysis techniques. The NMF uniqueness and its rank selection are two major open problems in this field. The solutions uniqueness issue can be addressed by imposing the orthogonality condition on NMF. This constraint yields sparser part-based representations and improved performance in clustering and source separation tasks. However, existing orthogonal NMF algorithms rely mainly on non-probabilistic frameworks that ignore the noise inherent in real-life data and lack variable uncertainties. Thus, in this work, we investigate a new probabilistic formulation of orthogonal NMF (ONMF). In the proposed model, we impose the orthogonality through a directional prior distribution defined on the Stiefel manifold called von Mises-Fisher distribution. This manifold consists of a set of directions that comply with the orthogonality condition that arises in many applications. Moreover, our model involves an automatic relevance determination (ARD) prior to address the model order selection issue. We devised an efficient variational Bayesian inference algorithm to solve the proposed ONMF model, which allows fast processing of large datasets. We evaluated the proposed model, called VBONMF, on the task of blind decomposition of real-world multispectral images of ancient documents. The numerical experiments demonstrate its efficiency and competitiveness compared to the state-of-the-art approaches.

A new extension of von Mises-Fisher distribution

Spherical distributions, including the von Mises-Fisher density, have received a great attention in the literature because of their usefulness to model circular data lying on the unit sphere. However, there is a paucity of research on proposing spherical densities possing multimode tuned with a single parameter. To fill in this gap, we extend von Mises-Fisher distribution to construct a new density. Moreover, some of the important statistical properties of the proposed distribution including the estimation of parameters are highlighted. To evaluate the performance of the proposed distribution, some simulation studies and analyzing three real-life examples are presented.

Nonlinear distributed state estimation on the Stiefel manifold using diffusion particle filters

Many relevant problems in engineering demand the estimation of dynamic system states that are constrained to non-Euclidean spaces. Furthermore, it is generally advantageous to implement estimation methods in a network of cooperating nodes instead of relying on a single data fusion center. With this in mind, we propose in this paper two new distributed particle filtering methods to track the states of a dynamic system that evolves on the Stiefel manifold, which arises naturally when the states are subject to nonlinear orthogonal constraints. The proposed algorithms are based on the Random Exchange and Adapt-then-Combine diffusion techniques, and perform parametric approximations using the matrix von Mises-Fisher distribution to compress information exchanged between network nodes. Estimates of the state are then determined via an empirical averaging method that approximates the centroid (Karcher mean) of the particle set. As we verify via numerical simulations, the proposed methods show improved performance compared to previous Particle and Extended Kalman filters designed for Euclidean state variables, and compared to a non-cooperative particle filtering algorithm.

Hyperspherical Variational Co-embedding for Attributed Networks

Network-based information has been widely explored and exploited in the information retrieval literature. Attributed networks, consisting of nodes, edges as well as attributes describing properties of nodes, are a basic type of network-based data, and are especially useful for many applications. Examples include user profiling in social networks and item recommendation in user-item purchase networks. Learning useful and expressive representations of entities in attributed networks can provide more effective building blocks to down-stream network-based tasks such as link prediction and attribute inference. Practically, input features of attributed networks are normalized as unit directional vectors. However, most network embedding techniques ignore the spherical nature of inputs and focus on learning representations in a Gaussian or Euclidean space, which, we hypothesize, might lead to less effective representations. To obtain more effective representations of attributed networks, we investigate the problem of mapping an attributed network with unit normalized directional features into a non-Gaussian and non-Euclidean space. Specifically, we propose a hyperspherical variational co-embedding for attributed networks (HCAN), which is based on generalized variational auto-encoders for heterogeneous data with multiple types of entities. HCAN jointly learns latent embeddings for both nodes and attributes in a unified hyperspherical space such that the affinities between nodes and attributes can be captured effectively. We argue that this is a crucial feature in many real-world applications of attributed networks. Previous Gaussian network embedding algorithms break the assumption of uninformative prior, which leads to unstable results and poor performance. In contrast, HCAN embeds nodes and attributes as von Mises-Fisher distributions, and allows one to capture the uncertainty of the inferred representations. Experimental results on eight datasets show that HCAN yields better performance in a number of applications compared with nine state-of-the-art baselines.

Resampling methods in ANOVA for data from the von Mises-Fisher distribution

An important problem in directional statistics is to test the null hypothesis of a common mean direction for several populations. The Analysis of Variance (ANOVA) test for vectorial data may be used to test the hypothesis of the equality of the mean directions for several von Mises-Fisher populations. As this test is valid only for large concentrations, we propose in this paper to apply the resampling techniques of bootstrap and permutation to the ANOVA test. We carried out an extensive simulation study in order to evaluate the performance of the ANOVA test with the resampling techniques, for several sphere dimensions and different sample sizes and we compare with the usual ANOVA test for data from von Mises-Fisher populations. The purpose of this simulation study is also to investigate whether the proposed tests are preferable to the ANOVA test, for low concentrations and small samples. Finally, we present an example with spherical data.

Classification of observations into von Mises-Fisher populations with unknown parameters

The problem of classification of an observation into von Mises-Fisher populations is considered when the concentration parameters and the mean directions are unknown. For two von Mises-Fisher distributions, we propose the restricted maximum likelihood estimators (MLEs), and Bayes estimators of the concentration parameters. The MLEs and restricted MLEs of the concentration parameters are compared in terms of risks. When the concentration parameters are ordered, we propose classification rules using the restricted MLEs and Bayes estimators of the parameters. For two populations, we also derive predictive Bayes classification rules using informative priors for the concentration parameters. We derive the likelihood ratio-based classification rule. Nonparametric rules such as k-NN rule, support vector machine classifier, and kernel density-based rule are also studied. For more than two populations, we suggest classification rules. Comparisons among the proposed rules have been carried out with respect to the expected probability of misclassification. Applications of the rules are described using directional data sets.

Gaussian Tracking With Kent-Distributed Direction-of-Arrival Measurements

This paper presents a Gaussian tracking algorithm with direction-of-arrival (DOA) measurements modelled via the Kent distribution. The key aspect of the algorithm is that the Kent distribution directly models the specific characteristics of DOA measurements in the 3-D space, and can account for different uncertainties in azimuth and elevation, which the von Mises-Fisher distribution cannot. At each update step, the algorithm performs iterated statistical linear regressions. We provide two implementations of the algorithms, one based on sigma-points and the other on analytical linearisation. The effectiveness of the approach is evaluated via numerical simulations.