Hyperbolic Neural Network Research Articles

Capacity bounds for hyperbolic neural network representations of latent tree structures

We study the representation capacity of deep hyperbolic neural networks (HNNs) with a ReLU activation function. We establish the first proof that HNNs can ɛ-isometrically embed any finite weighted tree into a hyperbolic space of dimension d at least equal to 2 with prescribed sectional curvature κ<0, for any ɛ>1 (where ɛ=1 being optimal). We establish rigorous upper bounds for the network complexity on an HNN implementing the embedding. We find that the network complexity of HNN implementing the graph representation is independent of the representation fidelity/distortion. We contrast this result against our lower bounds on distortion which any ReLU multi-layer perceptron (MLP) must exert when embedding a tree with L>2d leaves into a d-dimensional Euclidean space, which we show at least Ω(L1/d); independently of the depth, width, and (possibly discontinuous) activation function defining the MLP.

Hyperbolic Neural Network Based Preselection for Expensive Multi-Objective Optimization

Hyperbolic Neural Network Based Preselection for Expensive Multi-Objective Optimization

Q-Deformed and L-parametrized hyperbolic tangent function relied complex valued multivariate trigonometric and hyperbolic neural network approximations

Here we study the multivariate quantitative approximation of complex valued continuous functions on a box of RN , N ∈ N, by the multivariate normalized type neural network operators. We investigate also the case of approximation by iterated multilayer neural network operators. These approximations are achieved by establishing multidimen-sional Jackson type inequalities involving the multivariate moduli of continuity of the en- gaged function and its partial derivatives. Our multivariate operators are defined by using a multidimensional density function induced by a q-deformed and λ-parametrized hyper-bolic tangent function, which is a sigmoid function. The approximations are pointwise and uniform. The related feed-forward neural network are with one or multi hidden layers. The basis of our theory are the introduced multivariate Taylor formulae of trigonometric and hyperbolic type.

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

Process bottlenecks identification and its root cause analysis using fusion-based clustering and knowledge graph

Drawing the strengths of data science and machine learning, process mining has recently emerged as an effective research approach for process management and its decision support. Bottleneck identification and analysis is a key problem in process mining which is considered a critical component for process improvement. While previous studies focusing on bottlenecks have been reported, visible gaps remain. Most of these studies considered bottleneck identification from local perspectives by quantitative metrics, such as machine operation and resource requirement, which can not be applied to knowledge-intensive processes. Moreover, the root cause of such bottlenecks has not been given enough attention, which limits the impact of process optimisation. This paper proposes an approach that utilises fusion-based clustering and hyperbolic neural network-based knowledge graph embedding for bottleneck identification and root cause analysis. Firstly, a fusion-based clustering is proposed to identify bottlenecks automatically from a global perspective, where the execution frequency of each stage at different periods is calculated to reveal the abnormal stage. Secondly, a process knowledge graph representing tasks, organisations, workforce and relation features as hierarchical and logical patterns is established. Finally, a hyperbolic cluster-based community detection mechanism is researched, based on the process knowledge graph embedding trained by a hyperbolic neural network, to analyse the root cause from a process perspective. Experimental studies using real-world data collected from a multidisciplinary design project revealed the merits of the proposed approach. The execution of the proposed approach is not limited to event logs; it can automatically identify bottlenecks without local quantitative metrics and analyse the causes from a process perspective.

Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Design.

Hyperbolic neural networks have been popular in the recent past due to their ability to represent hierarchical data sets effectively and efficiently. The challenge in developing these networks lies in the nonlinearity of the embedding space namely, the Hyperbolic space. Hyperbolic space is a homogeneous Riemannian manifold of the Lorentz group which is a semi-Riemannian manifold, i.e. a manifold equipped with an indefinite metric. Most existing methods (with some exceptions) use local linearization to define a variety of operations paralleling those used in traditional deep neural networks in Euclidean spaces. In this paper, we present a novel fully hyperbolic neural network which uses the concept of projections (embeddings) followed by an intrinsic aggregation and a nonlinearity all within the hyperbolic space. The novelty here lies in the projection which is designed to project data on to a lower-dimensional embedded hyperbolic space and hence leads to a nested hyperbolic space representation independently useful for dimensionality reduction. The main theoretical contribution is that the proposed embedding is proved to be isometric and equivariant under the Lorentz transformations, which are the natural isometric transformations in hyperbolic spaces. This projection is computationally efficient since it can be expressed by simple linear operations, and, due to the aforementioned equivariance property, it allows for weight sharing. The nested hyperbolic space representation is the core component of our network and therefore, we first compare this representation - independent of the network - with other dimensionality reduction methods such as tangent PCA, principal geodesic analysis (PGA) and HoroPCA. Based on this equivariant embedding, we develop a novel fully hyperbolic graph convolutional neural network architecture to learn the parameters of the projection. Finally, we present experiments demonstrating comparative performance of our network on several publicly available data sets.

Fuzzy hyperbolic neural network with time-varying delays

In this paper, the formula of fuzzy basis functions with time-varying delays (DFBF) is firstly given. Based on the DFBF, a novel fuzzy hyperbolic neural network with time-varying delays (DFHNN) and its approximation properties are presented, respectively. By constructing an appropriate Lyapunov–Krasovskii functional, the stability of DFHNN is discussed, and some new criteria concerning the global exponential stability of DFHNN are proposed. Finally, three numerical examples are given to demonstrate the effectiveness of the obtained results.