Graph Analysis Algorithms Research Articles

Enumeration of Far-apart Pairs by Decreasing Distance for Faster Hyperbolicity Computation

Hyperbolicity is a graph parameter that indicates how much the shortest-path distance metric of a graph deviates from a tree metric. It is used in various fields such as networking, security, and bioinformatics for the classification of complex networks, the design of routing schemes, and the analysis of graph algorithms. Despite recent progress, computing the hyperbolicity of a graph remains challenging. Indeed, the best known algorithm has time complexity O(n 3.69 ) , which is prohibitive for large graphs, and the most efficient algorithms in practice have space complexity O(n 2 ) . Thus, time as well as space are bottlenecks for computing the hyperbolicity. In this article, we design a tool for enumerating all far-apart pairs of a graph by decreasing distances. A node pair (u, v) of a graph is far-apart if both v is a leaf of all shortest-path trees rooted at u and u is a leaf of all shortest-path trees rooted at v . This notion was previously used to drastically reduce the computation time for hyperbolicity in practice. However, it required the computation of the distance matrix to sort all pairs of nodes by decreasing distance, which requires an infeasible amount of memory already for medium-sized graphs. We present a new data structure that avoids this memory bottleneck in practice and for the first time enables computing the hyperbolicity of several large graphs that were far out of reach using previous algorithms. For some instances, we reduce the memory consumption by at least two orders of magnitude. Furthermore, we show that for many graphs, only a very small fraction of far-apart pairs has to be considered for the hyperbolicity computation, explaining this drastic reduction of memory. As iterating over far-apart pairs in decreasing order without storing them explicitly is a very general tool, we believe that our approach might also be relevant to other problems.

New approach for APT malware detection on the workstation based on process profile

The Advanced Persistent Threat (APT) attack is a form of dangerous, intentionally and clearly targeted attack. Currently, the APT attack trend is through the end-users and then escalating privileges in the system by spreading malware which is widely used by attackers. Therefore, the problem of early detection and warning of the APT attack malware on workstations is urgent. In this paper, we propose a new approach to APT malware detection on workstations based on the technique of analyzing and evaluating process profiles. The characteristics and principles of our proposed method are as follows: Firstly, processes are collected and aggregated into process profiles of APT malware; Secondly, these process profiles are used by Graph2Vec graph analysis algorithm to extract the characteristics of the process profile. Finally, in order to conclude about the sign of malicious APT, this paper proposes to use Long short-term memory (LSTM) and bidirectional LSTM (BiLSTM) algorithm. With the proposed approach in the paper, we have not only succeeded in building and synthesizing APT malware behavior on Workstations as a basis to improve the efficiency of predicting APT malware, but also have opened up a new approach to the task of synthesizing and analyzing anomalous behavior of malware.

Spatio-temporal pattern detection in spatio-temporal graphs

Spatio-temporal (ST) graphs have been used in many application domains to model evolving ST phenomenon. Such models represent the underlying structure of the phenomenon in terms of its entities and different types of spatial interactions between them. The reason behind using graph-based models to represent ST phenomenon is due to the existing well-established graph analysis tools and algorithms which can be directly applied to analyze the phenomenon under consideration. In this paper, considering the use case of two distinct, highly dynamic phenomena - invasive team sports, with a focus on handball and urban road traffic, we propose a spatio-temporal graph model applicable to both these phenomena. Different types of entities and spatial relations which make up these phenomena are highlighted to formalize the graph. Furthermore, the idea of graph-based pattern detection in both these phenomena is explored. Different types of ST patterns for both ST phenomena are discussed and the problem of pattern detection is formalized as the problem of subgraph isomorphism for dynamic graphs. Finally, the results of our algorithm to detect random ST patterns in random ST graphs are presented. The ideas discussed in this paper are applicable to other ST phenomena as well.

SPACE: Finding Key-Speaker in Complex Multi-Person Scenes

Intelligent image processing based on deep learning is a recently emerging artificial Intelligence method which is critical in the next generation computing systems. Key-speaker detection using image processing aims to find the speaker who plays the most important role in complex multi-person scenes. However, existing speaker detection methods just judge whether there is somebody talking. To overcome this problem, this paper proposes a novel key-speaker detection approach named Speaker Pose-Attention deteCtion modEl (SPACE). This approach extracts the space information and analyze people's attention gathering to find the key-speaker. The method consists of two main parts. The first part is our proposed Importance Detection Model (IDM) using human pose estimation and an effective evaluation strategy to find candidates of important speaker. The second part consists of Pose-Attention Graph (PAG) and link analysis algorithm. The PAG represents people's attention graph. The link analysis algorithm analyzes the connectivity of this graph to find the key-speaker. Due to the lack of available key-speaker detection datasets, a self-collected dataset containing speech and meeting scenes has been built to estimate SPACE model. The experimental results show that the proposed method can obtain the best detection accuracy compared with other existing methods.

Synchronization in graph analysis algorithms on the Partially Ordered Event‐Triggered Systems many‐core architecture

One of the key problems in designing and implementing graph analysis algorithms for distributed platforms is to find an optimal way of managing communication flows in the massively parallel processing network. Message-passing and global synchronization are powerful abstractions in this regard, especially when used in combination. This paper studies the use of a hardware-implemented refutable global barrier as a design optimization technique aimed at unifying these abstractions at the API level. The paper explores the trade-offs between the related overheads and performance factors on a message-passing prototype machine with 49,152 RISC-V threads distributed over 48 FPGAs (called the Partially Ordered Event-Triggered Systems platform). Our experiments show that some graph applications favour synchronized communication, but the effect is hard to predict in general because of the interplay between multiple hardware and software factors. A classifier model is therefore proposed and implemented to perform such a prediction based on the application graph topology parameters: graph diameter, degree of connectivity, and reconvergence metric. The presented experimental results demonstrate that the correct choice of communication mode, granted by the new model-driven approach, helps to achieve 3.22 times faster computation time on average compared to the baseline platform operation.

Application of Benford’s Law on Cryptocurrencies

The manuscript presents a study of the possibility of use of Benford’s law conformity test, a well proven tool in the accounting fraud discovery, on a new domain: the discovery of anomalies (possibly fraudulent behaviour) in the the cryptocurrency transactions. Blockchain-based currencies or cryptocurrencies have become a global phenomenon known to most people as a disruptive technology, and a new investment vehicle. However, due to their decentralized nature, regulating these markets has presented regulators with difficulties in finding a balance between nurturing innovation, and protecting consumers. The growing concerns about illicit activity have forced regulators to seek new ways of detecting, analyzing, and ultimately policing public blockchain transactions. Extensive research on machine learning, and transaction graph analysis algorithms has been done to track suspicious behaviour. However, having a macro view of a public ledger is equally important before pursuing a more fine-grained analysis. Benford’s law, the law of first digit, has been extensively used as a tool to discover accountant frauds (many other use cases exist). The basic motivation that drove our research presented in this paper was to test the applicability of the well established method to a new domain, in this case the identification of anomalous behavior using Benford’s law conformity test to the cryptocurrency domain. The research focused on transaction values in all major cryptocurrencies. A suitable time-period was identified that was long enough to present sufficiently large number of observations for Benford’s law conformity tests and was also situated long enough in the past so that the anomalies were identified and well documented. The results show that most of the cryptocurrencies that did not conform to Benford’s law had well documented anomalous incidents, the first digits of aggregated transaction values of all well known cryptocurrency projects were conforming to Benford’s law. Thus the proposed method is applicable to the new domain.

Polarity-based graph neural network for sign prediction in signed bipartite graphs

As a fundamental data structure, graphs are ubiquitous in various applications. Among all types of graphs, signed bipartite graphs contain complex structures with positive and negative links as well as bipartite settings, on which conventional graph analysis algorithms are no longer applicable. Previous works mainly focus on unipartite signed graphs or unsigned bipartite graphs separately. Several models are proposed for applications on the signed bipartite graphs by utilizing the heuristic structural information. However, these methods have limited capability to fully capture the information hidden in such graphs. In this paper, we propose the first graph neural network on signed bipartite graphs, namely Polarity-based Graph Convolutional Network (PbGCN), for sign prediction task with the help of balance theory. We introduce the novel polarity attribute to signed bipartite graphs, based on which we construct one-mode projection graphs to allow the GNNs to aggregate information between the same type nodes. Extensive experiments on five datasets demonstrate the effectiveness of our proposed techniques.

The Investigation of the Transformational Leadership Style Managers and Impact on the Financial Managerial Performance With the Modern Distributed Graph Analysis and Parallel Coordinates Algorithm in the United States

This study introduces the application of parallel coordinates to accounting information and business science, illustrating the utility of our tools to visualize and explore different types of multivariate data. Prior studies have not been able to confirm earlier findings showing the leadership research results with the distributed graph analysis for the accounting or business decision. We offer a novel demonstration of how parallel coordinates provide a practical alternative to current data-driven solutions in the business and accounting toolboxes for visualizing and exploring multivariate data, identifying causal relationships, and communicating modern business science via the advanced interactive, web-based application. Modern exploratory digital analysis of business leadership, job performance, job satisfaction requires specialize tools to identify associations among variables. The results of the modern distributed graph analysis show better understanding a transformational leadership style affects employee satisfaction and performance and how may relate to improving business management and outcomes within the company with Artificial Intelligence Business Solution.

Graph Algo Visualizer

Visual aids can be used to visualize different concepts in computer science. Algorithm recognition has been a major theme for Computer Science education for years, but it has not reached schools / teaching halls as a major educational tool. The Graph Algo Visualizer is based on the Design Analysis of Graph Algorithms. It is a Visualizer that helps to understand and evaluate the working efficiency of Graph Algorithms easily as compared to the naïve approach of understanding algorithms through code.

A Quantitative Graph-Based Approach to Monitoring Ice-Wedge Trough Dynamics in Polygonal Permafrost Landscapes

In response to increasing Arctic temperatures, ice-rich permafrost landscapes are undergoing rapid changes. In permafrost lowlands, polygonal ice wedges are especially prone to degradation. Melting of ice wedges results in deepening troughs and the transition from low-centered to high-centered ice-wedge polygons. This process has important implications for surface hydrology, as the connectivity of such troughs determines the rate of drainage for these lowland landscapes. In this study, we present a comprehensive, modular, and highly automated workflow to extract, to represent, and to analyze remotely sensed ice-wedge polygonal trough networks as a graph (i.e., network structure). With computer vision methods, we efficiently extract the trough locations as well as their geomorphometric information on trough depth and width from high-resolution digital elevation models and link these data within the graph. Further, we present and discuss the benefits of graph analysis algorithms for characterizing the erosional development of such thaw-affected landscapes. Based on our graph analysis, we show how thaw subsidence has progressed between 2009 and 2019 following burning at the Anaktuvuk River fire scar in northern Alaska, USA. We observed a considerable increase in the number of discernible troughs within the study area, while simultaneously the number of disconnected networks decreased from 54 small networks in 2009 to only six considerably larger disconnected networks in 2019. On average, the width of the troughs has increased by 13.86%, while the average depth has slightly decreased by 10.31%. Overall, our new automated approach allows for monitoring ice-wedge dynamics in unprecedented spatial detail, while simultaneously reducing the data to quantifiable geometric measures and spatial relationships.

Graph Theory Applied to Plasma Chemical Reaction Engineering

This work explores the following applications of graph theory to plasma chemical reaction engineering: assembly of a weighted directional graph with the key addition of reaction nodes, from a published set of reaction data for air; graph visualisation for probing the reaction network for potentially useful or problematic reaction pathways; running Dijkstra’s algorithm between all species nodes; further analysis of the graph for useful engineering information such as which conditions, reactions, or species could be enhanced or supressed to favour particular outcomes, e.g. targeted chemical formation. The use of reaction-nodes combined with derived parameters allowed large amounts of key information regarding the plasma chemical reaction network to be assessed simultaneously using a leading open source graph visualisation software (Gephi). A connectivity matrix of Dijkstra’s algorithm between each two species gave a measure of the relative potential of species to be created and destroyed under specific conditions. Further investigation into using the graph for key reaction engineering information led to the development of a graph analysis algorithm to quantify demand for conditions for targeted chemical formation: Optimal Condition Approaching via Reaction-In-Network Analysis (OCARINA). Predictions given by running OCARINA display significant similarities to a well-known electric field strength regime for optimal ozone production in air. Time dependent 0D simulations also showed preferential formation for O· and O3 using the respective conditions generated by the algorithm. These applications of graph theory to plasma chemical reaction engineering show potential in identifying promising simulations and experiments to devote resources.

A local density optimization method based on a graph convolutional network

Success has been obtained using a semi-supervised graph analysis method based on a graph convolutional network (GCN). However, GCN ignores some local information at each node in the graph, so that data preprocessing is incomplete and the model generated is not accurate enough. Thus, in the case of numerous unsupervised models based on graph embedding technology, local node information is important. In this paper, we apply a local analysis method based on the similar neighbor hypothesis to a GCN, and propose a local density definition; we call this method LDGCN. The LDGCN algorithm processes the input data of GCN in two methods, i.e., the unbalanced and balanced methods. Thus, the optimized input data contains detailed local node information, and then the model generated is accurate after training. We also introduce the implementation of the LDGCN algorithm through the principle of GCN, and use three mainstream datasets to verify the effectiveness of the LDGCN algorithm (i.e., the Cora, Citeseer, and Pubmed datasets). Finally, we compare the performances of several mainstream graph analysis algorithms with that of the LDGCN algorithm. Experimental results show that the LDGCN algorithm has better performance in node classification tasks.

GraVF-M

Due to the irregular nature of connections in most graph datasets, partitioning graph analysis algorithms across multiple computational nodes that do not share a common memory inevitably leads to large amounts of interconnect traffic. Previous research has shown that FPGAs can outcompete software-based graph processing in shared memory contexts, but it remains an open question if this advantage can be maintained in distributed systems. In this work, we present GraVF-M, a framework designed to ease the implementation of FPGA-based graph processing accelerators for multi-FPGA platforms with distributed memory. Based on a lightweight description of the algorithm kernel, the framework automatically generates optimized RTL code for the whole multi-FPGA design. We exploit an aspect of the programming model to present a familiar message-passing paradigm to the user, while under the hood implementing a more efficient architecture that can reduce the necessary inter-FPGA network traffic by a factor equal to the average degree of the input graph. A performance model based on a theoretical analysis of the factors influencing performance serves to evaluate the efficiency of our implementation. With a throughput of up to 5.8GTEPS (billions of traversed edges per second) on a 4-FPGA system, the designs generated by GraVF-M compare favorably to state-of-the-art frameworks from the literature and reach 94% of the projected performance limit of the system.

Exploiting Parallelism and Vectorisation in Breadth-First Search for the Intel Xeon Phi

Modern applications generate massive amounts of data that is challenging to process or analyse. Graph algorithms have emerged as a solution for the analysis of such data because they can represent the entities participating in the generation of large-scale datasets in terms of vertices and their relationships in terms of edges. Graph analysis algorithms are used for finding patterns within these relationships, aiming to extract information to be further analysed. The breadth-first search (BFS) is one of the main graph search algorithms used for graph analysis and its optimisation has been widely researched using different parallel computers. However, the parallelisation of BFS has been shown to be challenging because of its inherent characteristics, including irregular memory access patterns, data dependencies and workload imbalance, that limit its scalability. This paper investigates the optimisation of the BFS on the Xeon Phi (Knights Corner), a modern parallel architecture provided with an advanced vector processor supporting the AVX-512 instruction set, using a bespoke development framework integrated with the Graph 500 benchmark. In addition, to demonstrate portability, we show results for a direct port of the algorithms to a more recent version of the Xeon Phi (Knights Landing) and to a Skylake CPU which supports most of the AVX-512 instruction set. Optimised parallel versions of two high-level algorithms for BFS were created using vectorisation, starting with the conventional top-down BFS algorithm and, building on this, a hybrid BFS algorithm. On the KNC our best implementations result in speedups of 1.37x ( top-down ) and 1.37x ( hybrid ), for a one million vertices graph, compared to the state-of-the-art. On the KNL and Skylake, the performance is higher than on KNC. In addition, we show results of our best hybrid algorithm on real-world graphs from the SNAP datasets with speedups up to 1.3x on KNC. Performance on KNL and Skylake is again higher, demonstrating the robustness and portability of our algorithm. The hybrid BFS algorithm can be further used to speed up other graph analysis algorithms and the lessons learned from vectorisation can be applied to other algorithms targeting existing and future models of the Xeon Phi and other advanced vector architectures.

HyGen: generating random graphs with hyperbolic communities

Random graph generators are necessary tools for many network science applications. For example, the evaluation of graph analysis algorithms requires methods for generating realistic synthetic graphs. Typically random graph generators are generating graphs that satisfy certain global criteria, such as degree distribution or diameter. If the generated graph is to be used to evaluate community detection and mining algorithms, however, the generator must produce realistic community structure, as well. Recent research has shown that a clique is not necessarily a realistic community structure, necessitating the development of new graph generators. We propose HyGen, a random graph generator that leverages the recent research on non-clique-like communities to produce realistic random graphs with hyperbolic community structure, degree distribution, and clustering coefficient. Our generator can also be used to accurately model time-evolving communities.

Unsupervised Graph Anomaly Detection Algorithms Implemented in Apache Spark

The graph anomaly detection problem occurs in many application areas and can be solved by spotting outliers in unstructured collections of multi-dimensional data points, which can be obtained by graph analysis algorithms. We implement the algorithm for the small community analysis and the approximate LOF algorithm based on Locality-Sensitive Hashing, apply the algorithms to a real world graph and evaluate scalability of the algorithms. We use Apache Spark as one of the most popular Big Data frameworks.

Identifying Fake Accounts on Social Networks Based on Graph Analysis and Classification Algorithms

Social networks have become popular due to the ability to connect people around the world and share videos, photos, and communications. One of the security challenges in these networks, which have become a major concern for users, is creating fake accounts. In this paper, a new model which is based on similarity between the users’ friends’ networks was proposed in order to discover fake accounts in social networks. Similarity measures such as common friends, cosine, Jaccard, L1-measure, and weight similarity were calculated from the adjacency matrix of the corresponding graph of the social network. To evaluate the proposed model, all steps were implemented on the Twitter dataset. It was found that the Medium Gaussian SVM algorithm predicts fake accounts with high area under the curve=1 and low false positive rate=0.02.

Fuzzy Random Walkers with Second Order Bounds: An Asymmetric Analysis

Edge-fuzzy graphs constitute an essential modeling paradigm across a broad spectrum of domains ranging from artificial intelligence to computational neuroscience and social network analysis. Under this model, fundamental graph properties such as edge length and graph diameter become stochastic and as such they are consequently expressed in probabilistic terms. Thus, algorithms for fuzzy graph analysis must rely on non-deterministic design principles. One such principle is Random Walker, which is based on a virtual entity and selects either edges or, like in this case, vertices of a fuzzy graph to visit. This allows the estimation of global graph properties through a long sequence of local decisions, making it a viable strategy candidate for graph processing software relying on native graph databases such as Neo4j. As a concrete example, Chebyshev Walktrap, a heuristic fuzzy community discovery algorithm relying on second order statistics and on the teleportation of the Random Walker, is proposed and its performance, expressed in terms of community coherence and number of vertex visits, is compared to the previously proposed algorithms of Markov Walktrap, Fuzzy Walktrap, and Fuzzy Newman–Girvan. In order to facilitate this comparison, a metric based on the asymmetric metrics of Tversky index and Kullback–Leibler divergence is used.

Using domain-specific languages for analytic graph databases

Recently graph has been drawing lots of attention both as a natural data model that captures fine-grained relationships between data entities and as a tool for powerful data analysis that considers such relationships. In this paper, we present a new graph database system that integrates a robust graph storage with an efficient graph analytics engine. Primarily, our system adopts two domain-specific languages (DSLs), one for describing graph analysis algorithms and the other for graph pattern matching queries. Compared to the API-based approaches in conventional graph processing systems, the DSL-based approach provides users with more flexible and intuitive ways of expressing algorithms and queries. Moreover, the DSL-based approach has significant performance benefits as well, (1) by skipping (remote) API invocation overhead and (2) by applying high-level optimization from the compiler.

Analyzing Enterprise Storage Workloads With Graph Modeling and Clustering

Utilizing graph analysis models and algorithms to exploit complex interactions over a network of entities is emerging as an attractive network analytic technology. In this paper, we show that traditional column or row-based trace analysis may not be effective in deriving deep insights hidden in the storage traces collected over complex storage applications, such as complex spatial and temporal patterns, hotspots and their movement patterns. We propose a novel graph analytics framework, GraphLens, for mining and analyzing real world storage traces with three unique features. First, we model storage traces as heterogeneous trace graphs in order to capture multiple complex and heterogeneous factors, such as diverse spatial/temporal access information and their relationships, into a unified analytic framework. Second, we employ and develop an innovative graph clustering method that employs two levels of clustering abstractions on storage trace analysis. We discover interesting spatial access patterns and identify important temporal correlations among spatial access patterns. This enables us to better characterize important hotspots and understand hotspot movement patterns. Third, at each level of abstraction, we design a unified weighted similarity measure through an iterative dynamic weight learning algorithm. With an optimal weight assignment scheme, we can efficiently combine the correlation information for each type of storage access patterns, such as random versus sequential, read versus write, to identify interesting spatial/temporal correlations hidden in the traces. Some optimization techniques on matrix computation are proposed to further improve the efficiency of our clustering algorithm on large trace datasets. Extensive evaluation on real storage traces shows GraphLens can provide broad and deep trace analysis for better storage strategy planning and efficient data placement guidance. GraphLens can be applied to both a single PC with multiple disks and a distributed network across a cluster of compute nodes to offer a few opportunities for optimization of storage performance.