N-sphere Research Articles

Session-based recommendation with temporal convolutional network to balance numerical gaps

A session-based recommendation system recommends the next possible item for users by learning the click session sequences of anonymous users. Considering the session data used is anonymous and few background information is available, it is very challenging to solve these problems. Recently, although the session-based recommendation systems based on neural network have achieved gratifying results, there are still two problems in the existing methods: (1) The value of each dimension in the embedded layer result is a non-zero mean distribution and the numerical gaps are very large. Such numerical gaps will increase the variance of the gradient, hindering the parameter optimization, and thus leading to the final prediction results inaccurate; (2) The previous models cannot effectively learn the long-term dependency information and capture the dependencies between non-adjacent items in the session sequence. To solve the above problems, we propose a Session-based Recommendation with Temporal Convolutional Network to Balance Numerical Gaps model. Specifically, we first normalize the embedded layer results, then constrain the embedded results in the unit hypersphere to reduce their impact on gradient calculation, and finally use Temporal Convolution Network (TCN) complements the multi-layer self-attention network to learn the session sequence.The TCN can obtain large enough receptive fields to fully learn session representation and the short-range item dependence that is missing due to the distraction of attention distribution by the self-attention mechanism, and the self-attention method capture the one-to-one interaction of each item, and obtain the long-term dependence of the item. We have conducted a large number of experiments on three real-world datasets. The results show that, in most cases, our proposed method outperforms the state-of-the-arts methods.

Parameter estimation and model-based clustering with spherical normal distribution on the unit hypersphere

In directional statistics, the von Mises-Fisher (vMF) distribution is one of the most basic and popular probability distributions for data on the unit hypersphere. Recently, the spherical normal (SN) distribution was proposed as an intrinsic counterpart to the vMF distribution by replacing the standard Euclidean norm with the great-circle distance, which is length of the shortest path joining two points on the unit sphere. Focusing on an isotropic version of SN distribution, it is shown that maximum likelihood estimators uniquely exist under mild support conditions. Since no analytic formula are available for the estimation, efficient numerical routines are proposed for parameter estimation. The estimation is considered in a general setting where non-negative weights are assigned to observations. This leads to a more interesting contribution for model-based clustering on the unit hypersphere by finite mixture model with SN distributions. Efficiency of optimization-based estimation procedures and effectiveness of SN mixture model are validated using simulated and real data examples.

Spherical perspective on learning with normalization layers

Normalization Layers (NLs) are widely used in modern deep-learning architectures. Despite their apparent simplicity, their effect on optimization is not yet fully understood. This paper introduces a spherical framework to study the optimization of neural networks with NLs from a geometric perspective. Concretely, the radial invariance of groups of parameters, such as filters for convolutional neural networks, allows to translate the optimization steps on the L2 unit hypersphere. This formulation and the associated geometric interpretation shed new light on the training dynamics. Firstly, the first effective learning rate expression of Adam is derived. Then the demonstration that, in the presence of NLs, performing Stochastic Gradient Descent (SGD) alone is actually equivalent to a variant of Adam constrained to the unit hypersphere, stems from the framework. Finally, this analysis outlines phenomena that previous variants of Adam act on and their importance in the optimization process are experimentally validated.

Oblique decision tree induction by cross-entropy optimization based on the von Mises–Fisher distribution

Oblique decision trees recursively divide the feature space by using splits based on linear combinations of attributes. Compared to their univariate counterparts, which only use a single attribute per split, they are often smaller and more accurate. A common approach to learn decision trees is by iteratively introducing splits on a training set in a top–down manner, yet determining a single optimal oblique split is in general computationally intractable. Therefore, one has to rely on heuristics to find near-optimal splits. In this paper, we adapt the cross-entropy optimization method to tackle this problem. The approach is motivated geometrically by the observation that equivalent oblique splits can be interpreted as connected regions on a unit hypersphere which are defined by the samples in the training data. In each iteration, the algorithm samples multiple candidate solutions from this hypersphere using the von Mises–Fisher distribution which is parameterized by a mean direction and a concentration parameter. These parameters are then updated based on the best performing samples such that when the algorithm terminates a high probability mass is assigned to a region of near-optimal solutions. Our experimental results show that the proposed method is well-suited for the induction of compact and accurate oblique decision trees in a small amount of time.

Provable properties of asymptotic safety in f(R) approximation

We study an f(R) approximation to asymptotic safety, using a family of non-adaptive cutoffs, kept general to test for universality. Matching solutions on the four-dimensional sphere and hyperboloid, we prove properties of any such global fixed point solution and its eigenoperators. For this family of cutoffs, the scaling dimension at large n of the nth eigenoperator, is λn ∝ b n ln n. The coefficient b is non-universal, a consequence of the single-metric approximation. The large R limit is universal on the hyperboloid, but not on the sphere where cutoff dependence results from certain zero modes. For right-sign conformal mode cutoff, the fixed points form at most a discrete set. The eigenoperator spectrum is quantised. They are square integrable under the Sturm-Liouville weight. For wrong sign cutoff, the fixed points form a continuum, and so do the eigenoperators unless we impose square-integrability. If we do this, we get a discrete tower of operators, infinitely many of which are relevant. These are f(R) analogues of novel operators in the conformal sector which were used recently to furnish an alternative quantisation of gravity.

Cross-entropy-based directional importance sampling with von Mises-Fisher mixture model for reliability analysis

In structural reliability analysis, the estimation of rare failure probability is a significant challenge. Directional importance sampling may address this challenge when a quasi-optimal sampling density is chosen. Directional importance sampling involves sampling on the unit hypersphere and one-dimensional reliability analysis. This paper develops a cross-entropy-based directional importance sampling (CE-DIS) for reliability analysis. The von Mises-Fisher mixture (vMFM) model is selected as sampling density to generate direction vectors on the unit hypersphere. The updating rules based on the cross-entropy method are derived for the vMFM model to find a quasi-optimal sampling density. In addition, the DBSCAN algorithm is used to choose the number of distributions in the vMFM model in case of no prior information of failure domain, and the intermediate failure event is introduced in case of not enough samples to update the distribution parameters. Several benchmark numerical examples and engineering applications are used to test the performance of the proposed CE-DIS method. The results demonstrate that the CE-DIS method significantly improves the computational efficiency compared to original directional sampling and other CE-based importance sampling methods. In addition, the CE-DIS method can be applied to different types of reliability problems, e.g., multiple design points, multiple failure modes and rare failure probability.

On a remarkable geometric-mechanical synergism based on a novel linear eigenvalue problem

The vertices of two specific eigenvectors, obtained from a novel linear eigenvalue problem, describe two curves on the surface of an N-dimensional unit hypersphere. N denotes the number of degrees of freedom in the framework of structural analysis by the Finite Element Method. The radii of curvature of these two curves are 0 and 1. They correlate with pure stretching and pure bending, respectively, of structures. The two coefficient matrices of the eigenvalue problem are the tangent stiffness matrix at the load level considered and the one at the onset of loading. The goals of this paper are to report on the numerical verification of the aforesaid geometric-mechanical synergism and to summarize current attempts of its extension to combinations of stretching and bending of structures.

Analysis of Gegenbauer kernel filtration on the hypersphere

In this study, we aim to construct explicit forms of convolution formulae for Gegenbauer kernel filtration on the surface of the unit hypersphere. Using the properties of Gegenbauer polynomials, we reformulated Gegenbauer filtration as the limit of a sequence of finite linear combinations of hyperspherical Legendre harmonics and gave proof for the completeness of the associated series. We also proved the existence of a fundamental solution of the spherical Laplace-Beltrami operator on the hypersphere using the filtration kernel. An application of the filtration on a one-dimensional Cauchy wave problem was also demonstrated.

Improvement and Assessment of a Blind Image Deblurring Algorithm Based on Independent Component Analysis

The aim of the present paper is to improve an existing blind image deblurring algorithm, based on an independent component learning paradigm, by manifold calculus. The original technique is based on an independent component analysis algorithm applied to a set of pseudo-images obtained by Gabor-filtering a blurred image and is based on an adapt-and-project paradigm. A comparison between the original technique and the improved method shows that independent component learning on the unit hypersphere by a Riemannian-gradient algorithm outperforms the adapt-and-project strategy. A comprehensive set of numerical tests evidenced the strengths and weaknesses of the discussed deblurring technique.

Single-projection reconstruction technique for positioning monodisperse spheres in 3D with a divergent x-ray beam

The measurement of the position of single-sized spheres in 3D from a single, divergent, radiographic projection is addressed in the present study with the development of a novel method. Generally speaking, the location of the shadow cast by a single sphere on a detector defines a source-detector ray; the position of the particle along this ray is identified by the strong prior knowledge of its radius and the size of the shadow. For a dense assembly of equal-sized particles whose projections overlap, a novel Fourier transform based technique is introduced to give a first 3D determination of the particle centres. The uncertainty of this measurement is calculated from synthetic data with a known noise distribution. A further refinement of this measurement is performed based on the minimisation of the projection residual. The combined approach is validated both on synthetic data, and on real radiographs of a glass bead packing. The effect of noise on the measurement uncertainty is evaluated. The technique is made available to the community in the open source python package radioSphere.

A novel iterative identification based on the optimised topology for common state monitoring in wireless sensor networks

Power consumption and data redundancy of wireless sensor networks (WSN) are widely considered for a distributed state monitoring network. For reducing the energy consumption and data amount, we propose a topology optimisation and an iterative parameter identification method for estimating the common model factors in WSN. The former method optimises the decentralised topology such that all the leaf nodes in a community connect to the head node directly. A circle topology is built to enable the remote leaf nodes to link to the head node through two adjoining relay nodes to reduce the whole communication distance and power consumption. Based on the optimised topology, an iterative identification method is proposed to minimise the information capacity by transmitting the processed results instead of raw data to reduce the data amount for calculation and storage. Then, we prove the consensus and convergence of the proposed identification method. Finally, two simulations verify the effectiveness of the proposed method and the comparative results present the data reduction for the on-board calculation, communication, and storage in the practical use of WSN.

Maximum likelihood estimation of the Fisher–Bingham distribution via efficient calculation of its normalizing constant

This paper proposes an efficient numerical integration formula to compute the normalizing constant of Fisher–Bingham distributions. This formula uses a numerical integration formula with the continuous Euler transform to a Fourier-type integral representation of the normalizing constant. As this method is fast and accurate, it can be applied to the calculation of the normalizing constant of high-dimensional Fisher–Bingham distributions. More precisely, the error decays exponentially with an increase in the integration points, and the computation cost increases linearly with the dimensions. In addition, this formula is useful for calculating the gradient and Hessian matrix of the normalizing constant. Therefore, we apply this formula to efficiently calculate the maximum likelihood estimation (MLE) of high-dimensional data. Finally, we apply the MLE to the hyperspherical variational auto-encoder (S-VAE), a deep-learning-based generative model that restricts the latent space to a unit hypersphere. We use the S-VAE trained with images of handwritten numbers to estimate the distributions of each label. This application is useful for adding new labels to the models.

Estimation of 6D Object Pose Using a 2D Bounding Box.

This paper provides an efficient way of addressing the problem of detecting or estimating the 6-Dimensional (6D) pose of objects from an RGB image. A quaternion is used to define an object′s three-dimensional pose, but the pose represented by q and the pose represented by -q are equivalent, and the L2 loss between them is very large. Therefore, we define a new quaternion pose loss function to solve this problem. Based on this, we designed a new convolutional neural network named Q-Net to estimate an object’s pose. Considering that the quaternion′s output is a unit vector, a normalization layer is added in Q-Net to hold the output of pose on a four-dimensional unit sphere. We propose a new algorithm, called the Bounding Box Equation, to obtain 3D translation quickly and effectively from 2D bounding boxes. The algorithm uses an entirely new way of assessing the 3D rotation (R) and 3D translation rotation (t) in only one RGB image. This method can upgrade any traditional 2D-box prediction algorithm to a 3D prediction model. We evaluated our model using the LineMod dataset, and experiments have shown that our methodology is more acceptable and efficient in terms of L2 loss and computational time.

Universal approximation on the hypersphere

The approximation properties of finite mixtures of location-scale distributions on Euclidean space have been well studied. It has been shown that mixtures of location-scale distributions can approximate arbitrary probability density function up to any desired level of accuracy provided the number of mixture components is sufficiently large. However, analogous results are not available for probability density functions defined on the unit hypersphere. The von-Mises-Fisher distribution, defined on the unit hypersphere Sm in plays the central role in directional statistics. We prove that any continuous probability density function on Sm can be approximated to arbitrary degrees of accuracy in sup norm by a finite mixture of von-Mises-Fisher distributions. Our proof strategy and result are also useful in studying the approximation properties of other finite mixtures of directional distributions.

Independence of shear strength with particle size dispersity still valid in polyhedral particle assemblies

A very staggering result that has been constantly highlighted in granular media is that the shear strength of granular assemblies is independent of the particle size dispersity. In other words, a packing composed of monodisperse particles has similar strength properties to those of polydisperse systems. This has been shown numerically for the simplified case of disc and polygon assemblies in 2D and spheres in 3D. In this paper, we use three-dimensional contact dynamics simulations to revisit these results for the more complex case of assemblies composed of highly polydisperse rigid polyhedra. Although non-spherical shapes induce more intricated spatial correlations than spherical shapes because of the multiple contact types (i.e., vertex-face, edge-edge, edge-face, face-face), our numerical data provide evidence that the shear strength independence as the particle size dispersity increases still holds up for assemblies of polyhedra. We explain this finding from compensation mechanisms at the micro-scale between geometrical and mechanical anisotropies developed within the assemblies.

Generation and Frame Characteristics of Predefined Evenly-Distributed Class Centroids for Pattern Classification

Predefined evenly-distributed class centroids (PEDCC) can be widely used in models and algorithms of pattern classification, such as CNN classifiers, classification autoencoders, clustering, and semi-supervised learning, etc. Its basic idea is to predefine the class centers, which are evenly-distributed on the unit hypersphere in feature space, to maximize the inter-class distance. The previous method of generating PEDCC uses an iterative algorithm based on a charge model. The generated class centers will have some errors with the theoretically evenly-distributed points, and the generation time is long. This paper takes advantage of regular polyhedron in high-dimensional space and the evenly distributed points on the n dimensional hypersphere to generate PEDCC mathematically. Then, we discussed the basic and extended characteristics of the frames formed by PEDCC, and some meaningful conclusions are obtained. Finally, the effectiveness of the new algorithm and related conclusions are proved by experiments. The mathematical analysis and experimental results of this paper can provide a theoretical tool for using PEDCC to solve the key problems in the field of pattern recognition, such as interpretable supervised/unsupervised learning, incremental learning, uncertainty analysis and so on.

Multi-Target Tracking on Riemannian Manifolds via Probabilistic Data Association

Riemannian manifolds are attracting much interest in various technical disciplines, since generated data can often be naturally represented as points on a Riemannian manifold. Due to the non-Euclidean geometry of such manifolds, usual Euclidean methods yield inferior results, thus motivating development of tools adapted or specially tailored to the true underlying geometry. In this letter we propose a method for tracking multiple targets residing on smooth manifolds via probabilistic data association. By using tools of differential geometry, such as exponential and logarithmic mapping along with the parallel transport, we extend the Euclidean multi-target tracking techniques based on probabilistic data association to systems constrained to a Riemannian manifold. The performance of the proposed method was extensively tested in experiments simulating multi-target tracking on unit hyperspheres, where we compared our approach to the von Mises-Fisher and the Kalman filters in the embedding space that projects the estimated state back to the manifold. Obtained results show that the proposed method outperforms the competitive trackers in the optimal sub-pattern assignment metric for all the tested hypersphere dimensions. Although our use case geometry is that of a unit hypersphere, our approach is by no means limited to it and can be applied to any Riemannian manifold with closed-form expressions for exponential/logarithmic maps and parallel transport along the geodesic curve. The paper code is publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

Geodesic projection of the von Mises–Fisher distribution for projection pursuit of directional data

We investigate geodesic projections of von Mises–Fisher (vMF) distributed directional data. The vMF distribution for random directions on the (p−1)-dimensional unit hypersphere Sp−1⊂Rp plays the role of multivariate normal distribution in directional statistics. For one-dimensional circle S1, the vMF distribution is called von Mises (vM) distribution. Projections onto geodesics are one of main ingredients of modeling and exploring directional data. We show that the projection of vMF distributed random directions onto any geodesic is approximately vM-distributed, albeit not exactly the same. In particular, the distribution of the geodesic-projected score is an infinite scale mixture of vM distributions. Approximations by vM distributions are given along various asymptotic scenarios including large and small concentrations (κ→∞,κ→0), high-dimensions (p→∞), and two important cases of double-asymptotics (p,κ→∞, κ∕p→c or κ∕p→λ), to support our claim: geodesic projections of the vMF are approximately vM. As one of potential applications of the result, we contemplate a projection pursuit exploration of high-dimensional directional data. We show that in a high dimensional model almost all geodesic-projections of directional data are nearly vM, thus measures of non-vM-ness are a viable candidate for projection index.

Generation of spherical vortex beams to trap large particles for enhanced axial force

Recent studies have shown the possibility to manipulate small elastic spheres in 3D with a single-sided beam. Acoustical tweezers are very appealing because they provide a fine spatial control of the motion of a single particle in space. Their main limitations are due to the weak restoring axial force and improving this force is still a challenge. We show theoretically that the spherical vortex beams can trap large particles and enhance the axial force. Indeed, the special features of these unusual beams look like the bottle beams in optics. Nevertheless, their spatial complexity presupposes that they can be produced with sufficient precision. Therefore, attention is paid to the synthesis of the spherical vortices. A method based on the inverse filter method is proposed. It allows to synthesize them with a very good precision since the theoretical force is recovered experimentally with an error smaller than 10%. Then, the spherical vortices are used to trap polyethylene beads with radii between 500 and 590µm. Experiments show that the radial trap is working while no beads have been trapped in the axial direction. This failure is analyzed in detail and is shown to be mainly due to sensitivity to the properties of the materials which influences the resonance modes of the elastic sphere. This study paves the way to the optimization of acoustical tweezers for the manipulation of large objects.

The interpoint depth for directional data

The notion of the interpoint depth is applied to spherical spaces by us-ing an appropriate angular distance function for data lying on the surfaceof the unit hypersphere. The traditional multivariate methods, indeed, arenot suitable for the analysis of directional data and this holds true also fordistance measures and related depth based methods. The interpoint depthfor directional data possesses some nice properties and can be used for highdimensional data analysis. This notion of depth is particularly useful toinvestigate local features of distribution such as multimodality and can beexploited to deal with many statistical problems. The behavior of the pro-posed depth is investigated by means of simulated data. In addition threeinteresting applications are presented.