N-sphere Research Articles

Negative-Free Self-Supervised Gaussian Embedding of Graphs

Graph Contrastive Learning (GCL) has recently emerged as a promising graph self-supervised learning framework for learning discriminative node representations without labels. The widely adopted objective function of GCL benefits from two key properties: alignment and uniformity, which align representations of positive node pairs while uniformly distributing all representations on the hypersphere. The uniformity property plays a critical role in preventing representation collapse and is achieved by pushing apart augmented views of different nodes (negative pairs). As such, existing GCL methods inherently rely on increasing the quantity and quality of negative samples, resulting in heavy computational demands, memory overhead, and potential class collision issues. In this study, we propose a negative-free objective to achieve uniformity, inspired by the fact that points distributed according to a normalized isotropic Gaussian are uniformly spread across the unit hypersphere. Therefore, we can minimize the distance between the distribution of learned representations and the isotropic Gaussian distribution to promote the uniformity of node representations. Our method also distinguishes itself from other approaches by eliminating the need for a parameterized mutual information estimator, an additional projector, asymmetric structures, and, crucially, negative samples. Extensive experiments over seven graph benchmarks demonstrate that our proposal achieves competitive performance with fewer parameters, shorter training times, and lower memory consumption compared to existing GCL methods.

Towards similar alignment and unique uniformity in collaborative filtering

Representation learning, with its desired properties, including Alignment and Uniformity, has recently emerged as an effective approach in collaborative filtering (CF) for recommender systems. Alignment measures the distance between positive pairs, and uniformity describes the distribution of all samples on the unit hypersphere. Despite the effectiveness of existing research, the challenges of insufficient alignment and uniformity bias persist in studies on the desired properties of recommender systems: (1) Sparse interaction information leads to an insufficient alignment representation. (2) Calculating the uniformity loss based on duplicate samples introduces bias, which affects the overall representation uniformity.Building on aforementioned challenges, we propose a Similar Alignment and Unique Uniformity model called SUAU. SUAU mitigates insufficient alignment by introducing additional item-similar item pairs for model training and employing a uniqueness strategy to prevent uniformity bias. More specifically, we propose a similar items generation method that identifies the most similar items based on the interaction behavior of the original items’ related users to form item-similar item pairs. The experimental results obtained on three highly sparse datasets demonstrate that the propose model achieves superior alignment and uniformity compared with state-of-the-art models, notably surpassing the best CF methods in terms of recommendation performance. We have open-sourced the source code on a publicly accessible website: https://github.com/ZzYUuuu/SUAU.

Analytical Computation of Hyper-Ellipsoidal Harmonics

The four-dimensional ellipsoid of an anisotropic hyper-structure corresponds to the four-dimensional sphere of an isotropic hyper-structure. In three dimensions, both theories for spherical and ellipsoidal harmonics have been developed by Laplace and Lamé, respectively. Nevertheless, in four dimensions, only the theory of hyper-spherical harmonics is hitherto known. This void in the literature is expected to be filled up by the present work. In fact, it is well known that the spectral decomposition of the Laplace equation in three-dimensional ellipsoidal geometry leads to the Lamé equation. This Lamé equation governs each one of the spectral functions corresponding to the three ellipsoidal coordinates, which, however, live in non-overlapping intervals. The analysis of the Lamé equation leads to four classes of Lamé functions, giving a total of 2n + 1 functions of degree n. In four dimensions, a much more elaborate procedure leads to similar results for the hyper-ellipsoidal structure. Actually, we demonstrate here that there are eight classes of the spectral hyper-Lamé equation and we provide a complete analysis for each one of them. The number of hyper-Lamé functions of degree n is (n + 1)2; that is, n2 more functions than the three-dimensional case. However, the main difficulty in the four-dimensional analysis concerns the evaluation of the three separation constants appearing during the separation process. One of them can be extracted from the corresponding theory of the hyper-sphero-conal system, but the other two constants are obtained via a much more complicated procedure than the three-dimensional case. In fact, the solution process exhibits specific nonlinearities of polynomial type, itemized for every class and every degree. An example of this procedure is demonstrated in detail in order to make the process clear. Finally, the hyper-ellipsoidal harmonics are given as the product of four identical hyper-Lamé functions, each one defined in its own domain, which are explicitly calculated and tabulated for every degree less than five.

PLoPS: Localization-aware person search with prototypical normalization

Person search involves localizing and re-identifying persons of interest captured by multiple, non-overlapping cameras. Recent approaches to person search are typically built on object detection frameworks to learn joint person representations for detection and re-identification. To this end, the features extracted from pedestrian proposals are projected onto a unit hypersphere using L2 normalization, and positive proposals that sufficiently overlap with the ground truth are equally incorporated for training by exploiting an external lookup table (LUT). We have found that (1) using the L2 normalization technique, without considering feature distributions, can degenerate the discriminative power of person representations, (2) positive proposals often depict distracting details, such as background clutter and person overlaps, and (3) person features in the LUT are not often updated during training. To address these limitations, we propose a novel framework for person search, dubbed PLoPS, using a prototypical normalization layer, ProtoNorm, that calibrates features while considering the long-tail distribution across person IDs. PLoPS also entails a localization-aware learning scheme that prioritizes better-aligned proposals w.r.t the ground truth. We further introduce a LUT calibration technique to continuously adjust the person features in the LUT. Experimental results and analysis on standard benchmarks demonstrate the effectiveness of PLoPS.

Effective Comparative Prototype Hashing for Unsupervised Domain Adaptation

Unsupervised domain adaptive hashing is a highly promising research direction within the field of retrieval. It aims to transfer valuable insights from the source domain to the target domain while maintaining high storage and retrieval efficiency. Despite its potential, this field remains relatively unexplored. Previous methods usually lead to unsatisfactory retrieval performance, as they frequently directly apply slightly modified domain adaptation algorithms to hash learning framework, or pursue domain alignment within the Hamming space characterized by limited semantic information. In this paper, we propose a simple yet effective approach named Comparative Prototype Hashing (CPH) for unsupervised domain adaptive image retrieval. We establish a domain-shared unit hypersphere space through prototype contrastive learning and then obtain the Hamming hypersphere space via mapping from the shared hypersphere. This strategy achieves a cohesive synergy between learning uniformly distributed and category conflict-averse feature representations, eliminating domain discrepancies, and facilitating hash code learning. Moreover, by leveraging dual-domain information to supervise the entire hashing model training process, we can generate hash codes that retain inter-sample similarity relationships within both domains. Experimental results validate that our CPH significantly outperforms the state-of-the-art counterparts across multiple cross-domain and single-domain retrieval tasks. Notably, on Office-Home and Office-31 datasets, CPH achieves an average performance improvement of 19.29% and 13.85% on cross-domain retrieval tasks compared to the second-best results, respectively. The source codes of our method are available at: https://github.com/christinecui/CPH.

Deep Adaptive Graph Clustering via von Mises-Fisher Distributions

Graph clustering has been a hot research topic and is widely used in many fields, such as community detection in social networks. Lots of works combining auto-encoder and graph neural networks have been applied to clustering tasks by utilizing node attributes and graph structure. These works usually assumed the inherent parameters (i.e., size and variance) of different clusters in the latent embedding space are homogeneous, and hence the assigned probability is monotonous over the Euclidean distance between node embeddings and centroids. Unfortunately, this assumption usually does not hold since the size and concentration of different clusters can be quite different, which limits the clustering accuracy. In addition, the node embeddings in deep graph clustering methods are usually L2 normalized so that it lies on the surface of a unit hyper-sphere. To solve this problem, we proposed D eep A daptive G raph C lustering via von Mises-Fisher distributions, namely DAGC. DAGC assumes the node embeddings H can be drawn from a von Mises-Fisher distribution and each cluster k is associated with cluster inherent parameters ρ k which includes cluster center μ and cluster cohesion degree κ. Then we adopt an EM-like approach (i.e., 𝒫( H | ρ ) and 𝒫( ρ | H ), respectively) to learn the embedding and cluster inherent parameters alternately. Specifically, with the node embeddings, we proposed to update the cluster centers in an attraction-repulsion manner to make the cluster centers more separable. And given the cluster inherent parameters, a likelihood-based loss is proposed to make node embeddings more concentrated around cluster centers. Thus, DAGC can simultaneously improve the intra-cluster compactness and inter-cluster heterogeneity. Finally, extensive experiments conducted on four benchmark datasets have demonstrated that the proposed DAGC consistently outperforms the state-of-the-art methods, especially on imbalanced datasets.

Averaging data on the unit hypersphere

Averaging data on the unit sphere S d (also called a unit hypersphere) is a common problem in computer vision, robotics and other fields, with applications ranging from motion planning to DNA modelling. In this paper, we introduce a new method for averaging data represented as points on the unit sphere S d−1 using the d-dimensional generalized Kuramoto model. Our method is verified on a range of benchmark data sets and compared with common data averaging algorithms. Also, we showcase the applicability of this method for solving rotation averaging problem.

Interpolation on the unit hypersphere using the n-dimensional generalized Kuramoto model

Computer graphics, robotics, and physics are one of the many domains where interpolation on the unit sphere S n (often called a unit hypersphere or unit n-sphere) plays a crucial role. In this paper, we introduce a novel approach for achieving smooth and precise interpolation on the unit sphere S n−1 using the n-dimensional generalized Kuramoto model. The proposed algorithm finds the shortest and most direct path between two points on that non-Euclidean manifold. Our simulation results demonstrate that it achieves performance comparable to that of a Spherical Linear Interpolation algorithm. Also, the paper proposes the application of our algorithm in the interpolation of rotations that are presented in the form of four-dimensional data.

Density estimation and regression analysis on hyperspheres in the presence of measurement error

AbstractThis paper studies density estimation and regression analysis with data observed on a general unit hypersphere and contaminated by measurement errors. We establish novel density and regression estimators, and study their asymptotic properties such as the rates of convergence and asymptotic normality. We also provide two types of asymptotic confidence intervals for both density and regression functions. One type is based on the asymptotic normality of their estimators and the other type is based on the empirical likelihood technique. We present practical details on the implementation of our method as well as simulation studies and real data analysis.

Contrastive Self-Supervised Learning Leads to Higher Adversarial Susceptibility

Contrastive self-supervised learning (CSL) has managed to match or surpass the performance of supervised learning in image and video classification. However, it is still largely unknown if the nature of the representations induced by the two learning paradigms is similar. We investigate this under the lens of adversarial robustness. Our analysis of the problem reveals that CSL has intrinsically higher sensitivity to perturbations over supervised learning. We identify the uniform distribution of data representation over a unit hypersphere in the CSL representation space as the key contributor to this phenomenon. We establish that this is a result of the presence of false negative pairs in the training process, which increases model sensitivity to input perturbations. Our finding is supported by extensive experiments for image and video classification using adversarial perturbations and other input corruptions. We devise a strategy to detect and remove false negative pairs that is simple, yet effective in improving model robustness with CSL training. We close up to 68% of the robustness gap between CSL and its supervised counterpart. Finally, we contribute to adversarial learning by incorporating our method in CSL. We demonstrate an average gain of about 5% over two different state-of-the-art methods in this domain.

Meta-relation assisted knowledge-aware coupled graph neural network for recommendation

Currently, the sparsity of user–item interaction data limits the performance of recommender systems. To alleviate the problems caused by data sparsity, researchers are devoted to designing more powerful graph neural network models to fuse the auxiliary information of users and items (social relationships, item dependencies, etc.). However, the lack of in-depth exploration of auxiliary information and the lack of optimization of the spatially distributed properties of the generated representations (known as embeddings) make it difficult to further improve the recommendation performance of these models. To address the above challenges, we propose a meta-relation-assisted knowledge-aware coupled graph neural network (MKCGN). Specifically, we design meta-relationship-guided graph neural networks for capturing collaborative signals from user and item domains and introduce coupled graph neural networks to inject local relationships into user/item representations. In addition, we design an auxiliary task that optimizes the distribution properties (alignment and uniformity) of the representation on the unit hypersphere to improve the quality of the representation. To evaluate the performance of MKCGN, we compare it with the state-of-the-art model on two real-world datasets with average performance improvements of 5.26% and 4.87%(using HR@N and NDCG@N as evaluation metrics), respectively. Our implemented model is available via https://github.com/Ganoder/MKCGN.

Estimating the Secret Key of Spread Spectrum Watermarking Based on Equivalent Keys

The security of spread spectrum (SS) watermarking largely depends on the difficulty of estimating its secret key. Some estimators have been proposed to estimate the secret key in the known-message attack (KMA) scenario. However, the estimation accuracies of existing estimators are not satisfactory when the number of observations is not large enough. Currently, it is still a challenging and open problem to design more effective estimators. In this paper, we propose an equivalent keys (EK)-based estimator to estimate the secret key for both the traditional and more secure SS watermarking methods. Equivalent keys form an equivalent region, which is the intersection of a unit hypersphere and a hypercone. According to the Monte Carlo simulation, we find that the secret key can be estimated by adding up the equivalent keys uniformly sampled from the equivalent region. Thus, the proposed estimator selects equivalent keys from randomly-generated vectors by exploiting the pairs of watermarked signals and their embedded messages. A theoretical analysis is performed for the proposed estimator to evaluate the estimation accuracy. Experimental results verify the theoretical analysis and show the superiority of the proposed estimator over existing estimation methods. Furthermore, this paper also shows the insecurity of the more secure SS watermarking methods in the KMA scenario from a practical perspective for the first time.

A hyperbolic divergence based nonparametric test for two‐sample multivariate distributions

AbstractTwo‐sample hypothesis testing, as a fundamental problem in statistical inference, seeks to detect the difference between two probability measures and has numerous real‐world applications. Current test procedures for multivariate two‐sample problems typically rely on angles and lengths in a Euclidean space, or lengths in a unit hypersphere after representing data with the spherical model. This article introduces a hyperbolic divergence based on hyperbolic lengths in hyperbolic geometry, as well as a subsequent nonparametric approach to testing the multivariate two‐sample problem. We investigate the properties of our test procedure and discover that our hyperbolic divergence statistic is strongly consistent and consistent against all other alternatives; we also demonstrate that its limit distribution is an infinite mixture of distributions under the null hypothesis and a normal distribution under the alternative hypothesis. To calculate the ‐value, we employ the permutation method. Furthermore, in numerical studies, we compare our method with several nonparametric procedures under various distributional assumptions and alternatives. We discover that our test procedure has some advantages when the distributions' complex correlation structures differ. Finally, we examine one real data set to show how our method can be used to test two‐sample heterogeneity.

Transquaternions

The quaternions extend the complex numbers and are used in physics and engineering. Division of quaternions by zero is not defined, which limits physical theories and engineering applications. We now introduce transquaternions, which totalise the arithmetical operations of quaternion addition, subtraction, multiplication, and both left and right division. In particular, division of quaternions by zero is allowed. The transquaternions are homeomorphic to the unit hypersphere or glome, including its interior, together with an isolated point. The 4D interior of the hypersphere is made up of the ordinary quaternions. The 3D surface of the hypersphere is made up of the infinite transquaternions, which are produced by dividing non-zero quaternions by zero. The isolated point, that lies outside the 4D space containing the hypersphere, is the transquaternion nullity, which is produced by dividing zero by zero. Transquaternions are a separable compact complete metric topological space.

Rupture Directivity of the 2021 ML 2.2 Gwangyang, Korea, Microearthquake: Toward Resolving High-Resolution Rupture Process of a Small Earthquake

Abstract Moderate-to-large earthquakes (M>6) frequently show clear rupture directivity. Recent studies revealed that a substantial percentage of small-sized earthquakes (M<∼5) display rupture directivity as well, owing to enhanced seismic monitoring. Is rupture directivity a common feature for earthquakes of all sizes? In this study, we investigated the rupture directivity of the 27 August 2021 ML 2.2 Gwangyang microearthquake that occurred at the southern tip of the Korean Peninsula by utilizing data from a recently expanded dense seismic network. The mainshock had two foreshocks and three aftershocks, which enabled us to define the fault plane from precise relative event locations and a well-constrained focal mechanism. Analysis of apparent source time functions obtained with the empirical Green’s function technique reveals that the mainshock ruptured unilaterally toward the east. A detailed analysis of the source pulse leaving the focal sphere in 3D indicates that rupture directivity is consistent with the fault geometry and slip direction. Our study demonstrates that we can resolve rupture directivity of unilaterally rupturing microearthquakes (M<∼3) given adequate azimuthal coverage, and that this will lead to an improved understanding of the seismogenic processes in regions of low seismicity worldwide.

Anisotropic Diffusion in Consensus-Based Optimization on the Sphere

In this paper we are concerned with the global minimization of a possibly non-smooth and non-convex objective function constrained on the unit hypersphere by means of a multi-agent derivative-free method. The proposed algorithm falls into the class of the recently introduced Consensus-Based Optimization. In fact, agents move on the sphere driven by a drift towards an instantaneous consensus point, which is computed as a convex combination of agent locations, weighted by the cost function according to Laplaces principle, and it represents an approximation to a global minimizer. The dynamics is further perturbed by an anisotropic random vector field to favor exploration. The main results of this paper are about the proof of convergence of the numerical scheme to global minimizers provided conditions of well-preparation of the initial datum. The proof of convergence combines a mean-field limit result with a novel asymptotic analysis, and classical convergence results of numerical methods for SDE. The main innovation with respect to previous work is the introduction of an anisotropic stochastic term, which allows us to ensure the independence of the parameters of the algorithm from the dimension and to scale the method to work in very high dimension. We present several numerical experiments, which show that the algorithm proposed in the present paper is extremely versatile and outperforms previous formulations with isotropic stochastic noise.

Neural collapse under cross-entropy loss

We consider the variational problem of cross-entropy loss with n feature vectors on a unit hypersphere in R d . We prove that when d ≥ n − 1 , the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also prove that, as n → ∞ with fixed d , the minimizing points will distribute uniformly on the hypersphere and show a connection with the frame potential of Benedetto & Fickus.

Mixture of von Mises-Fisher distribution with sparse prototypes

Mixtures of von Mises-Fisher distributions can be used to cluster data on the unit hypersphere. This is particularly adapted for high-dimensional directional data such as texts. We propose in this article to estimate a von Mises mixture using a l1 penalized likelihood. This leads to sparse prototypes that improve clustering interpretability. We introduce an expectation–maximisation (EM) algorithm for this estimation and explore the trade-off between the sparsity term and the likelihood one with a path following algorithm. The model’s behaviour is studied on simulated data and, we show the advantages of the approach on real data benchmark. We also introduce a new data set on financial reports and exhibit the benefits of our method for exploratory analysis.

Applying supervised contrastive learning for the detection of diabetic retinopathy and its severity levels from fundus images

Diabetic Retinopathy (DR) is a major complication in human eyes among the diabetic patients. Early detection of the DR can save many patients from permanent blindness. Various artificial intelligent based systems have been proposed and they outperform human analysis in accurate detection of the DR. In most of the traditional deep learning models, the cross-entropy is used as a common loss function in a single stage end-to-end training method. However, it has been recently identified that this loss function has some limitations such as poor margin leading to false results, sensitive to noisy data and hyperparameter variations. To overcome these issues, supervised contrastive learning (SCL) has been introduced. In this study, SCL method, a two-stage training method with supervised contrastive loss function was proposed for the first time to the best of authors' knowledge to identify the DR and its severity stages from fundus images (FIs) using “APTOS 2019 Blindness Detection” dataset. “Messidor-2” dataset was also used to conduct experiments for further validating the model's performance. Contrast Limited Adaptive Histogram Equalization (CLAHE) was applied for enhancing the image quality and the pre-trained Xception CNN model was deployed as the encoder with transfer learning. To interpret the SCL of the model, t-SNE method was used to visualize the embedding space (unit hyper sphere) composed of 128 D space into a 2 D space. The proposed model achieved a test accuracy of 98.36%, and AUC score of 98.50% to identify the DR (Binary classification) and a test accuracy of 84.364%, and AUC score of 93.819% for five stages grading with the APTOS 2019 dataset. Other evaluation metrics (precision, recall, F1-score) were also determined with APTOS 2019 as well as with Messidor-2 for analyzing the performance of the proposed model. It was also concluded that the proposed method achieved better performance in detecting the DR compared to the conventional CNN without SCL and other state-of-the-art methods.

From Boltzmann Equation for Granular Gases to a Modified Navier–Stokes–Fourier System

In this paper, we give an overview of the results established in Alonso (http://arxiv.org/org/abs/2008.05173, 2020) which provides the first rigorous derivation of hydrodynamic equations from the Boltzmann equation for inelastic hard spheres in 3D. In particular, we obtain a new system of hydrodynamic equations describing granular flows and prove existence of classical solutions to the aforementioned system. One of the main issue is to identify the correct relation between the restitution coefficient (which quantifies the rate of energy loss at the microscopic level) and the Knudsen number which allows us to obtain non trivial hydrodynamic behavior. In such a regime, we construct strong solutions to the inelastic Boltzmann equation, near thermal equilibrium whose role is played by the so-called homogeneous cooling state. We prove then the uniform exponential stability with respect to the Knudsen number of such solutions, using a spectral analysis of the linearized problem combined with technical a priori nonlinear estimates. Finally, we prove that such solutions converge, in a specific weak sense, towards some hydrodynamic limit that depends on time and space variables only through macroscopic quantities that satisfy a suitable modification of the incompressible Navier–Stokes–Fourier system.