Graph Compression Research Articles

Параллельная реализация алгоритма поиска минимальных остовных деревьев с использованием центрального и графического процессоров

Solution of the finding a minimum spanning tree problem is common in various areas of research: recognition of different objects, computer vision, analysis and construction of networks (eg, telephone, electrical, computer, travel, etc.), chemistry and biology, and many others. Processing large graphs is a quite time-consuming task for the central processor (CPU), and in high demand at the present moment. The usage of Graphics processing units (GPUs) as a mean to solve general-purpose problems grows every day, because GPUs have more computing power than CPUs. This article describes the methods of compression and conversion of graphs in standard formats to increase the efficiency of their processing. The search algorithm of minimum spanning trees has been used for researching the proposed approaches. The possibility of a hybrid implementation of this algorithm has been investigated. The highest results were obtained on the large R-MAT and SSCA2 graphs.

A comprehensive reconfigurable computing approach to memory wall problem of large graph computation

The extension of the edge-streaming model with massive partitions to accelerate large graph computing in reconfigurable hardware.A two-level shuffle network architecture to reduce the on-chip memory requirement while provide high processing throughput.A compact storage designusing graph compression and a corresponding codec hardware to reduce the amount of transferred data.Up to 3.85 times improvement in terms of performance to bandwidth ratio over the state-of-the-art hardware implementation. Graph computation problems that exhibit irregular memory access patterns are known to show poor performance on multiprocessor architectures. Although recent studies use FPGA technology to tackle the memory wall problem of graph computation by adopting a massively multi-threaded architecture, the performance is still far less than optimal memory performance due to the long memory access latency. In this paper, we propose a comprehensive reconfigurable computing approach to address the memory wall problem. First, we present an extended edge-streaming model with massive partitions to provide better load balance while taking advantage of the streaming bandwidth of external memory in processing large graphs. Second, we propose a two-level shuffle network architecture to significantly reduce the on-chip memory requirement while provide high processing throughput that matches the bandwidth of the external memory. Third, we introduce a compact storage design based on graph compression schemes and propose the corresponding encoding and decoding hardware to reduce the data volume transferred between the processing engines and external memory. We validate the effectiveness of the proposed architecture by implementing three frequently-used graph algorithms on ML605 board, showing an up to 3.85 × improvement in terms of performance to bandwidth ratio over previously published FPGA-based implementations.

Using the VBARMS method in parallel computing

We introduce a new graph compression algorithm for block construction, which extends the method by Ashcraft and requires one simple to use parameter.We describe an improved block factorization computation in the VBARMS method.We conduct a parallel performance study of the VBARMS method using parallel graph partitioning.We solve turbulent three-dimensional Navier-Stokes equations on large realistic unstructured meshes. The paper describes an improved parallel MPI-based implementation of VBARMS, a variable block variant of the pARMS preconditioner proposed by Li etźal. 200314 for solving general nonsymmetric linear systems. The parallel VBARMS solver can detect automatically exact or approximate dense structures in the linear system, and exploits this information to achieve improved reliability and increased throughput during the factorization. A novel graph compression algorithm is discussed that finds these approximate dense blocks structures and requires only one simple to use parameter. A complete study of the numerical and parallel performance of parallel VBARMS is presented for the analysis of large turbulent Navier-Stokes equations on a suite of three-dimensional test cases.

An area-adaptive multi-level layout for social network visualisation

We describe an area-adaptive multi-level layout for visualising social networks. This area-adaptive layout can visualise social networks according to the display area and community structure, which could reasonably utilise the display area and enhance the community features. The whole process consists of two parts: graph multi-layered compression and top-down multi-level layout. The multi-layered compression process groups vertices to form clusters and then abstracts the clusters as new vertices to define a new graph and is repeated until the graph size falls below some threshold. Based on the compressed graph, we optimise the display area to top-down positions all vertices. We have evaluated our layout on several well-known data sets. The experimental results show that our layout outperforms the state-of-the-art methods.

Bin packing and related problems: General arc-flow formulation with graph compression

We present an exact method, based on an arc-flow formulation with side constraints, for solving bin packing and cutting stock problems—including multi-constraint variants—by simply representing all the patterns in a very compact graph. Our method includes a graph compression algorithm that usually reduces the size of the underlying graph substantially without weakening the model.Our formulation is equivalent to Gilmore and Gomory׳s, thus providing a very strong linear relaxation. However, instead of using column-generation in an iterative process, the method constructs a graph, where paths from the source to the target node represent every valid packing pattern.The same method, without any problem-specific parameterization, was used to solve a large variety of instances from several different cutting and packing problems. In this paper, we deal with vector packing, bin packing, cutting stock, cardinality constrained bin packing, cutting stock with cutting knife limitation, bin packing with conflicts, and other problems. We report computational results obtained with many benchmark test datasets, some of them showing a large advantage of this formulation with respect to the traditional ones.

A Graph Compression Based Overlapping Communities Detection Algorithm

A Graph Compression Based Overlapping Communities Detection Algorithm

Algorithm for Retrieval of Sub-community Graph from a Compressed Community Graph Using Graph Mining Techniques

Abstract Community detection is the one of the most relevant and important topics in the field of graph mining, principally for its applications in number of domains such as social or biological networks analysis to transport systems. Different community detection algorithms have been proposed earlier. This paper discusses and proposes a new technique on particular community detection and its retrieval in social community graph. Community detection constitutes a significant role for analysis of a large community graph by enabling and selecting the desired community's graph. It tries to extract a sub-community graph from a very large community graph for analysis. This paper presents both a theoretical and experimental result in this direction with certain example. We have retrieved the sub-community graph from a compressed community graph in the context of graph mining techniques. Observation concludes that the proposed technique is simpler, easier and efficient in terms of complexities.

Faster Structural Analysis of Differential-Algebraic Equations by Graph Compression

Structural Analysis of Differential Algebraic Equations is a computationally expensive task, because it relies on the solution of a highest value assignment in the bipartite structure graph of the model. Therefore the size of models that can be simulated is limited in practice by the runtime of the index reduction method. We present a technique to optimize the analysis by leveraging structural properties of hierarchical models: If a sub-component contains one more equation than protected variables, its protected variables may be removed from the structure graph by a compression transformation, thus reducing its size. A corresponding decompression operation allows to compute a highest-value assignment for the original graph from the compressed one. This can lead to drastically reduced runtime for the structural analysis of large models.

SlashBurn: Graph Compression and Mining beyond Caveman Communities

Given a real world graph, how should we lay-out its edges? How can we compress it? These questions are closely related, and the typical approach so far is to find clique-like communities, like the cavemen graph', and compress them. We show that the block-diagonal mental image of the cavemen graph' is the wrong paradigm, in full agreement with earlier results that real world graphs have no good cuts. Instead, we propose to envision graphs as a collection of hubs connecting spokes, with super-hubs connecting the hubs, and so on, recursively. Based on the idea, we propose the SLASHBURN method to recursively split a graph into hubs and spokes connected only by the hubs. We also propose techniques to select the hubs and give an ordering to the spokes, in addition to the basic SLASHBURN. We give theoretical analysis of the proposed hub selection methods. Our view point has several advantages: (a) it avoids the no good cuts' problem, (b) it gives better compression, and (c) it leads to faster execution times for matrix-vector operations, which are the back-bone of most graph processing tools. Through experiments, we show that SLASHBURN consistently outperforms other methods for all data sets, resulting in better compression and faster running time. Moreover, we show that SLASHBURN with the appropriate spokes ordering can further improve compression while hardly sacrificing the running time.

A matrix factorization approach to graph compression with partial information

We address the problem of encoding a graph of order $$\mathsf {n}$$ into a graph of order $$\mathsf {k}<\mathsf {n}$$ in a way to minimize reconstruction error. This encoding is characterized in terms of a particular factorization of the adjacency matrix of the original graph. The factorization is determined as the solution of a discrete optimization problem, which is for convenience relaxed into a continuous, but equivalent, one. Our formulation does not require to have the full graph, but it can factorize the graph also in the presence of partial information. We propose a multiplicative update rule for the optimization task resembling the ones introduced for nonnegative matrix factorization, and convergence properties are proven. Experiments are conducted to assess the effectiveness of the proposed approach.

Generalised power graph compression reveals dominant relationship patterns in complex networks

We introduce a framework for the discovery of dominant relationship patterns in complex networks, by compressing the networks into power graphs with overlapping power nodes. When paired with enrichment analysis of node classification terms, the most compressible sets of edges provide a highly informative sketch of the dominant relationship patterns that define the network. In addition, this procedure also gives rise to a novel, link-based definition of overlapping node communities in which nodes are defined by their relationships with sets of other nodes, rather than through connections within the community. We show that this completely general approach can be applied to undirected, directed, and bipartite networks, yielding valuable insights into the large-scale structure of real-world networks, including social networks and food webs. Our approach therefore provides a novel way in which network architecture can be studied, defined and classified.

Algebraic Bayesian networks secondary structure analysis features

The goal of this work is to generalize structure analysis of minimal join graphs representing algebraic Bayesian network secondary structure on common join graphs representing the same structure. A term system spreading the existing one with common join graphs is designed. New join graph properties are researched. Two lemmas characterizing an homage (product of a minimal join graph compression) as a minimal curia (product of a join graph compression) are proven. Theorem on minimal join graph proof is simplified.

Approximate Shortest Distance Computing: A Query-Dependent Local Landmark Scheme

Shortest distance query is a fundamental operation in large-scale networks. Many existing methods in the literature take a landmark embedding approach, which selects a set of graph nodes as landmarks and computes the shortest distances from each landmark to all nodes as an embedding. To answer a shortest distance query, the precomputed distances from the landmarks to the two query nodes are used to compute an approximate shortest distance based on the triangle inequality. In this paper, we analyze the factors that affect the accuracy of distance estimation in landmark embedding. In particular, we find that a globally selected, query-independent landmark set may introduce a large relative error, especially for nearby query nodes. To address this issue, we propose a query-dependent local landmark scheme, which identifies a local landmark close to both query nodes and provides more accurate distance estimation than the traditional global landmark approach. We propose efficient local landmark indexing and retrieval techniques, which achieve low offline indexing complexity and online query complexity. Two optimization techniques on graph compression and graph online search are also proposed, with the goal of further reducing index size and improving query accuracy. Furthermore, the challenge of immense graphs whose index may not fit in the memory leads us to store the embedding in relational database, so that a query of the local landmark scheme can be expressed with relational operators. Effective indexing and query optimization mechanisms are designed in this context. Our experimental results on large-scale social networks and road networks demonstrate that the local landmark scheme reduces the shortest distance estimation error significantly when compared with global landmark embedding and the state-of-the-art sketch-based embedding.

Linear-Time Compression of Bounded-Genus Graphs into Information-Theoretically Optimal Number of Bits

A $\textit{compression scheme}$ $A$ for a class $\mathbb{G}$ of graphs consists of an encoding algorithm $\textit{Encode}_A$ that computes a binary string $\textit{Code}_A(G)$ for any given graph $G$ in $\mathbb{G}$ and a decoding algorithm $\textit{Decode}_A$ that recovers $G$ from $\textit{Code}_A(G)$. A compression scheme $A$ for $\mathbb{G}$ is $\textit{optimal}$ if both $\textit{Encode}_A$ and $\textit{Decode}_A$ run in linear time and the number of bits of $\textit{Code}_A(G)$ for any $n$-node graph $G$ in $\mathbb{G}$ is information-theoretically optimal to within lower-order terms. Trees and plane triangulations were the only known nontrivial graph classes that admit optimal compression schemes. Based upon Goodrich's separator decomposition for planar graphs and Djidjev and Venkatesan's planarizers for bounded-genus graphs, we give an optimal compression scheme for any hereditary (i.e., closed under taking subgraphs) class $\mathbb{G}$ under the premise that any $n$-node graph of $\mathbb{G}$ to be encoded comes with a genus-$o(\frac{n}{\log^2 n})$ embedding. By Mohar's linear-time algorithm that embeds a bounded-genus graph on a genus-$O(1)$ surface, our result implies that any hereditary class of genus-$O(1)$ graphs admits an optimal compression scheme. For instance, our result yields the first-known optimal compression schemes for planar graphs, plane graphs, graphs embedded on genus-$1$ surfaces, graphs with genus $2$ or less, $3$-colorable directed plane graphs, $4$-outerplanar graphs, and forests with degree at most $5$. For non-hereditary graph classes, we also give a methodology for obtaining optimal compression schemes. From this methodology, we give the first known optimal compression schemes for triangulations of genus-$O(1)$ surfaces and floorplans.

Evaluation of a Hybrid Approach for Efficient Provenance Storage

Provenance is the metadata that describes the history of objects. Provenance provides new functionality in a variety of areas, including experimental documentation, debugging, search, and security. As a result, a number of groups have built systems to capture provenance. Most of these systems focus on provenance collection, a few systems focus on building applications that use the provenance, but all of these systems ignore an important aspect: efficient long-term storage of provenance. In this article, we first analyze the provenance collected from multiple workloads and characterize the properties of provenance with respect to long-term storage. We then propose a hybrid scheme that takes advantage of the graph structure of provenance data and the inherent duplication in provenance data. Our evaluation indicates that our hybrid scheme, a combination of Web graph compression (adapted for provenance) and dictionary encoding, provides the best trade-off in terms of compression ratio, compression time, and query performance when compared to other compression schemes.

Integrative Modelling of the Influence of MAPK Network on Cancer Cell Fate Decision

The Mitogen-Activated Protein Kinase (MAPK) network consists of tightly interconnected signalling pathways involved in diverse cellular processes, such as cell cycle, survival, apoptosis and differentiation. Although several studies reported the involvement of these signalling cascades in cancer deregulations, the precise mechanisms underlying their influence on the balance between cell proliferation and cell death (cell fate decision) in pathological circumstances remain elusive. Based on an extensive analysis of published data, we have built a comprehensive and generic reaction map for the MAPK signalling network, using CellDesigner software. In order to explore the MAPK responses to different stimuli and better understand their contributions to cell fate decision, we have considered the most crucial components and interactions and encoded them into a logical model, using the software GINsim. Our logical model analysis particularly focuses on urinary bladder cancer, where MAPK network deregulations have often been associated with specific phenotypes. To cope with the combinatorial explosion of the number of states, we have applied novel algorithms for model reduction and for the compression of state transition graphs, both implemented into the software GINsim. The results of systematic simulations for different signal combinations and network perturbations were found globally coherent with published data. In silico experiments further enabled us to delineate the roles of specific components, cross-talks and regulatory feedbacks in cell fate decision. Finally, tentative proliferative or anti-proliferative mechanisms can be connected with established bladder cancer deregulations, namely Epidermal Growth Factor Receptor (EGFR) over-expression and Fibroblast Growth Factor Receptor 3 (FGFR3) activating mutations.

Learning AND-OR Templates for Object Recognition and Detection

This paper presents a framework for unsupervised learning of a hierarchical reconfigurable image template--the AND-OR Template (AOT) for visual objects. The AOT includes: 1) hierarchical composition as "AND" nodes, 2) deformation and articulation of parts as geometric "OR" nodes, and 3) multiple ways of composition as structural "OR" nodes. The terminal nodes are hybrid image templates (HIT) [17] that are fully generative to the pixels. We show that both the structures and parameters of the AOT model can be learned in an unsupervised way from images using an information projection principle. The learning algorithm consists of two steps: 1) a recursive block pursuit procedure to learn the hierarchical dictionary of primitives, parts, and objects, and 2) a graph compression procedure to minimize model structure for better generalizability. We investigate the factors that influence how well the learning algorithm can identify the underlying AOT. And we propose a number of ways to evaluate the performance of the learned AOTs through both synthesized examples and real-world images. Our model advances the state of the art for object detection by improving the accuracy of template matching.

Compact representation of Web graphs with extended functionality

The representation of large subsets of the World Wide Web in the form of a directed graph has been extensively used to analyze structure, behavior, and evolution of those so-called Web graphs. However, interesting Web graphs are very large and their classical representations do not fit into the main memory of typical computers, whereas the required graph algorithms perform inefficiently on secondary memory. Compressed graph representations drastically reduce their space requirements while allowing their efficient navigation in compressed form. While the most basic navigation operation is to retrieve the successors of a node, several important Web graph algorithms require support for extended queries, such as finding the predecessors of a node, checking the presence of a link, or retrieving links between ranges of nodes. Those are seldom supported by compressed graph representations.This paper presents the k2-tree, a novel Web graph representation based on a compact tree structure that takes advantage of large empty areas of the adjacency matrix of the graph. The representation not only retrieves successors and predecessors in symmetric fashion, but also it is particularly efficient to check for specific links between nodes, or between ranges of nodes, or to list the links between ranges. Compared to the best representations in the literature supporting successor and predecessor queries, our technique offers the least space usage (1–3 bits per link) while supporting fast navigation to predecessors and successors (2–8μs per neighbor retrieved) and sharply outperforming the others on the extended queries. The representation is also of general interest and can be used to compress other kinds of graphs and data structures.

Research of Graph Compression in Information Storage

In the information storage, the image is often encountered, but the image storage and analysis are very complex, in order to more save memory space and more be used as a compressed representation of a graph, we give a new definition in the paper. We show some properties and give the lower integral sum number of some graph. DOI: http://dx.doi.org/10.11591/telkomnika.v11i8.3036

The Research of Graph Compression

With the development application of image processing, the image compression has an important role in the future area. In order to more save memory space and more be used as a compressed representation of a image, a image is mapped into a graph, we give a new graphs compressed representation.