Linked Data Structures Research Articles

DTAP: Accelerating Strongly-Typed Programs with Data Type-Aware Hardware Prefetching

Queries on linked data structures, such as trees and graphs, often suffer from frequent cache misses and significant performance loss due to dependent and random pointer-chasing memory accesses. In this paper, we propose a software-hardware co-designed solution for accelerating linked data structures implemented in strongly typed languages. The solution incorporates a compiler extension and a hardware prefetcher. The compiler extension extracts type information from the code, annotates each load instruction, and forwards the type information to the hardware prefetcher. The prefetcher leverages the type information to fetch the referred objects and identify the associated pointers in advance. By doing so, the program can find these objects in the cache when it follows the prefetched pointers, thus minimizing cache misses. In the evaluation, the proposed solution achieves an average speedup of 1.37 × over a set of memory-intensive benchmarks.

Implementing Linked Art in a Multi-Modal Database for Cross-Collection Discovery

Yale University has implemented a knowledge graph based discovery system that brings together the various art, natural history, archival, conservation and bibliographic collections using Linked Open Usable Data standards such as Linked Art and IIIF. This system comprises more than 41 million records, which would expand to more than 2 billion RDF triples, and is thus at a similar scale to Europeana. This paper presents the lessons learned from the five year effort around the usability of linked data structures across the organization, the technologies needed to make use of the knowledge in a performant way, and the appropriate design paradigms for front end applications which make the graph easily and intuitively accessible to researchers and the public, including the necessity of consistency in data modeling, that records are an essential concept to maintain through multi-modal systems, and the use of hypertext and web caches to maintain the separation between systems.

Dissidents, Rebels, and Everyday Heroes: New Perspectives on the Digital Archiving of Cultural Resistance Under State Socialism

The aim of this article is to present and study how a digital archive can shape and create new ways of producing, publishing, and studying historical sources. Based on our analysis of the COURAGE (Cultural Opposition—Understanding the CultuRal HeritAGE of Dissent in the Former Socialist Countries) Registry about cultural dissent under state socialism in Eastern Europe in the second half of the twentieth century (and focusing on collections about environmental issues in this registry), we seek to understand how different private, amateur, and professional archivists have shaped the scientific and public legacy of cultural dissent under state socialism. The COURAGE Registry conveys a unique view of the history of the Soviet Bloc, providing an assemblage of documents concerning people, groups, institutions, events, and pieces from the time. Together, they tell an alternative story of cultural opposition under socialism, shedding light on important—but until now marginalized—problems, topics, and actors. Our results have shown that cultural opposition in the Registry is understood and constructed as a wide range of forms of engagement and activities, and it is not limited to specific high-cultural or direct political products. The structure of the COURAGE Registry creates a balance among collections that are very diverse in form, and its linked data structure helps connect the information and stories compiled in it. The COURAGE Registry enables researchers to use it as a tool with which to build their own scientific narratives about dissent under socialism.

Model-guided synthesis of inductive lemmas for FOL with least fixpoints

Recursively defined linked data structures embedded in a pointer-based heap and their properties are naturally expressed in pure first-order logic with least fixpoint definitions (FO+lfp) with background theories. Such logics, unlike pure first-order logic, do not admit even complete procedures. In this paper, we undertake a novel approach for synthesizing inductive hypotheses to prove validity in this logic. The idea is to utilize several kinds of finite first-order models as counterexamples that capture the non-provability and invalidity of formulas to guide the search for inductive hypotheses. We implement our procedures and evaluate them extensively over theorems involving heap data structures that require inductive proofs and demonstrate the effectiveness of our methodology.

Life Cycle Information Models: Parameterized Linked Data Structures to Facilitate the Consistent Use of Life-Cycle Assessment in Decision Making

The recent enactment of the Buy Clean Act (2017) in California requires highway construction contractors to produce an environmental product declaration (EPD) at the point of installation for a list of all eligible construction materials. An EPD is an instrument that communicates life cycle assessment (LCA) outcomes for a product or process in compliance with the ISO Standard 14025. LCA methodology requires the use of life cycle data from interconnected industries and sectors that contribute to the supply chain of a product. This provides a challenge of consistent and reliable use of inventory data for the LCA of pavements, specifically when considering the geographic expanse of energy profiles, supply chain diversity of pavement materials, and additives. Hence, the objective of this paper is to develop life-cycle information models (LCIMs) that serve two purposes: first, allows for pavement LCA domain-specific knowledge representation, management, and sharing; and second, allows for consistent, reliable, and transparent use of life cycle inventory data. The first purpose is achieved by developing an ontological representation of key components and relationships underlying a pavement LCA to support formal parametric data structures. The second purpose is achieved by identifying life cycle inventories to be mapped to the data structures through a collaboration with the Federal LCA Commons. As LCA software tools are developed to support decision making for departments of transportation, the ontological representation of the pavement domain can serve as a specification outlining a common set of protocols and LCIM designed for reuse to support reliable decision support applications.

Steel: proof-oriented programming in a dependently typed concurrent separation logic

Steel is a language for developing and proving concurrent programs embedded in F ⋆ , a dependently typed programming language and proof assistant. Based on SteelCore, a concurrent separation logic (CSL) formalized in F ⋆ , our work focuses on exposing the proof rules of the logic in a form that enables programs and proofs to be effectively co-developed. Our main contributions include a new formulation of a Hoare logic of quintuples involving both separation logic and first-order logic, enabling efficient verification condition (VC) generation and proof discharge using a combination of tactics and SMT solving. We relate the VCs produced by our quintuple system to solving a system of associativity-commutativity (AC) unification constraints and develop tactics to (partially) solve these constraints using AC-matching modulo SMT-dischargeable equations. Our system is fully mechanized and implemented in F ⋆ . We evaluate it by developing several verified programs and libraries, including various sequential and concurrent linked data structures, proof libraries, and a library for 2-party session types. Our experience leads us to conclude that our system enables a mixture of automated and interactive proof, making it productive to build programs foundationally verified against a highly expressive, state-of-the-art CSL.

Fast Key-Value Lookups with Node Tracker

Lookup operations for in-memory databases are heavily memory bound, because they often rely on pointer-chasing linked data structure traversals. They also have many branches that are hard-to-predict due to random key lookups. In this study, we show that although cache misses are the primary bottleneck for these applications, without a method for eliminating the branch mispredictions only a small fraction of the performance benefit is achieved through prefetching alone. We propose the Node Tracker (NT), a novel programmable prefetcher/pre-execution unit that is highly effective in exploiting inter key-lookup parallelism to improve single-thread performance. We extend NT with branch outcome streaming (BOS) to reduce branch mispredictions and show that this achieves an extra 3× speedup. Finally, we evaluate the NT as a pre-execution unit and demonstrate that we can further improve the performance in both single- and multi-threaded execution modes. Our results show that, on average, NT improves single-thread performance by 4.1× when used as a prefetcher; 11.9× as a prefetcher with BOS; 14.9× as a pre-execution unit and 18.8× as a pre-execution unit with BOS. Finally, with 24 cores of the latter version, we achieve a speedup of 203× and 11× over the single-core and 24-core baselines, respectively.

PETRA

Emerging byte-addressable Non-Volatile Memories (NVMs) enable persistent memory where process state can be recovered after crashes. To enable applications to rely on persistent data, durable data structures with failure-atomic operations have been proposed. However, they lack the ability to allow users to execute a sequence of operations as transactions. Meanwhile, persistent transactional memory (PTM) has been proposed by adding durability to Software Transactional Memory (STM). However, PTM suffers from high performance overheads and low scalability due to false aborts, logging, and ordering constraints on persistence. In this article, we propose PETRA, a new approach for constructing persistent transactional linked data structures. PETRA natively supports transactions, but unlike PTM, relies on the high-level information from the data structure semantics. This gives PETRA unique advantages in the form of high performance and high scalability. Our experimental results using various benchmarks demonstrate the scalability of PETRA in all workloads and transaction sizes. PETRA outperforms the state-of-the-art PTMs by an order of magnitude in transactions of size greater than one, and demonstrates superior performance in transactions of size one.

A conversation about linked data in the library and publishing ecosystem

During the inaugural 2020 NISO+ conference, the “Ask the Experts about… Linked Data” panel included discussion of the transition of library metadata from legacy, record-based models to linked data structures. Panelists John Chapman (OCLC, Inc.) and Philip Schreur (Stanford University) were the speakers; NISO Board of Directors member Mary Sauer-Games (OCLC, Inc.) was the facilitator. The event was an open-ended conversation, with topics driven by questions and comments from the audience.

연계된 데이터로의 불규칙 메모리 접근을 위한 인-메모리 가속기

연결 리스트, 트리, 해시 테이블 같은 기본적인 자료구조는 포인터를 사용하여 인접 노드를 연결하는 선형 구조를 가지고 있다. 포인터 기반 데이터 구조 및 링크 순회 알고리즘은 프로그래밍의 필수 구성 요소이며 노드 데이터의 메모리 주소에 상관없이 노드를 동적으로 추가 또는 삭제할 수 있다. 리스트 노드는 순차적인 탐색만이 가능하여 연속으로 노드를 액세스할 때 CPU의 병렬처리 능력을 제한한다. 본 논문에서는 이를 해결하기 위해 인-메모리 가속기를 추가하여 연결 리스트의 데이터 탐색을 메모리 근접 로직에서 병렬적으로 진행할 수 있는 가속기 구조를 제안한다. Gem5 시뮬레이터를 사용하여 가속기 구조를 모델링하고, Memcached와 Graph500 벤치마크로 성능을 평가하였다. 벤치마크의 실행 시간은 baseline CPU에서 실행한 시간보다 평균 13.8% 단축되었고, 메모리 액세스 빈도 수 또한 평균 17.85% 감소하였다. TSMC 28nm 공정 라이브러리를 사용하여 게이트 수준에서 합성된 가속기 면적은 총 16172개 게이트수를 가지며, baseline CPU에서 사용한 Cortex-A53 면적대비 약 10% 정도의 추가 오버헤드가 있다.

Exploring Prefetching, Pre-Execution and Branch Outcome Streaming for In-Memory Database Lookups

Lookup operations for in-memory databases are heavily memory-bound because they often rely on pointer-chasing linked data structure traversals. They are also branch heavy with branches that are hard-to-predict due to random key lookups. In this study, we show that although cache misses are the primary bottleneck for these applications, without a method for eliminating the branch mispredictions, only a small fraction of the performance benefit is achieved through prefetching. We propose the Node Tracker (NT), a novel programmable prefetcher/pre-execution unit that is highly effective in exploiting inter key-lookup parallelism to improve single-thread performance. We extend the NT with branch outcome streaming (BOS) to reduce branch mispredictions and show that NT with BOS can achieve an extra 3x speedup. Finally, we evaluated the NT as a pre-execution unit and show that we can further improve the performance in both single- and multi-threaded execution modes.

FaultNet3D: Predicting Fault Probabilities, Strikes, and Dips With a Single Convolutional Neural Network

We simultaneously estimate fault probabilities, strikes, and dips directly from a seismic image by using a single convolutional neural network (CNN). In this method, we assume a local 3-D fault is a plane defined by a single combination of strike and dip angles. We assume the fault strikes and dips, respectively, are in the ranges of [0°, 360°) and [64°, 85°], which are divided into 577 classes corresponding to the situation of no fault and 576 different combinations of strikes and dips. We construct a 7-layer CNN to classify the fault strike and dip in a local seismic cube and obtain the classification probability at the same time. With the fault probability, strike and dip estimated at some seismic pixel, we further compute a fault cube (centered at the pixel) with fault features elongated along the fault plane. By sliding the classification window within a full seismic image, we are able to obtain a lot of overlapping fault cubes which are stacked to compute three full images of enhanced and continuous fault probabilities, strikes, and dips. To train the CNN model, we propose an effective and efficient workflow to automatically create 900 000 synthetic seismic cubes and the corresponding fault class labels. Although trained with only synthetic data sets, our CNN model can be applied to accurately estimate fault probabilities, strikes, and dips within field seismic images that are acquired at totally different surveys. With the estimated three fault images, we further construct fault cells that are represented as small 3-D squares, each square is colored by fault probability and oriented by fault strike and dip. We recursively link the fault cells by following the fault strikes and dips to finally construct fault skins, which are simple linked data structures to represent fault surfaces.

Precise reasoning with structured time, structured heaps, and collective operations

Despite decades of progress, static analysis tools still have great difficulty dealing with programs that combine arithmetic, loops, dynamic memory allocation, and linked data structures. In this paper we draw attention to two fundamental reasons for this difficulty: First, typical underlying program abstractions are low-level and inherently scalar, characterizing compound entities like data structures or results computed through iteration only indirectly. Second, to ensure termination, analyses typically project away the dimension of time, and merge information per program point, which incurs a loss in precision. As a remedy, we propose to make collective operations first-class in program analysis – inspired by Σ-notation in mathematics, and also by the success of high-level intermediate languages based on @map/reduce@ operations in program generators and aggressive optimizing compilers for domain-specific languages (DSLs). We further propose a novel structured heap abstraction that preserves a symbolic dimension of time, reflecting the program’s loop structure and thus unambiguously correlating multiple temporal points in the dynamic execution with a single point in the program text. This paper presents a formal model, based on a high-level intermediate analysis language, a practical realization in a prototype tool that analyzes C code, and an experimental evaluation that demonstrates competitive results on a series of benchmarks. Remarkably, our implementation achieves these results in a fully semantics-preserving strongest-postcondition model, which is a worst-case for analysis/verification. The underlying ideas, however, are not tied to this model and would equally apply in other settings, e.g., demand-driven invariant inference in a weakest-precondition model. Given its semantics-preserving nature, our implementation is not limited to analysis for verification, but can also check program equivalence, and translate legacy C code to high-performance DSLs.

Structuring the synthesis of heap-manipulating programs

This paper describes a deductive approach to synthesizing imperative programs with pointers from declarative specifications expressed in Separation Logic. Our synthesis algorithm takes as input a pair of assertions—a pre- and a postcondition—which describe two states of the symbolic heap, and derives a program that transforms one state into the other, guided by the shape of the heap. Our approach to program synthesis is grounded in proof theory: we introduce the novel framework of Synthetic Separation Logic (SSL), which generalises the classical notion of heap entailment P ⊢ Q to incorporate a possibility of transforming a heap satisfying an assertion P into a heap satisfying an assertion Q . A synthesized program represents a proof term for a transforming entailment statement P ↝ Q , and the synthesis procedure corresponds to a proof search. The derived programs are, thus, correct by construction, in the sense that they satisfy the ascribed pre/postconditions, and are accompanied by complete proof derivations, which can be checked independently. We have implemented a proof search engine for SSL in a form of the program synthesizer called SuSLik. For efficiency, the engine exploits properties of SSL rules, such as invertibility and commutativity of rule applications on separate heaps, to prune the space of derivations it has to consider. We explain and showcase the use of SSL on characteristic examples, describe the design of SuSLik, and report on our experience of using it to synthesize a series of benchmark programs manipulating heap-based linked data structures.

Renovating blockchain with distributed databases: An open source system

A blockchain is a decentralised linked data structure that is characterised by its inherent resistance to data modification, but it is deficient in search queries primarily due to its inferior data formatting. A distributed database is also a decentralised data structure which features quick query processing and well-designed data formatting but suffers from data reliability. In this work, we showcase ChainSQL, an open-source system developed by integrating the blockchain with the database, i.e. we present a blockchain database application platform that has the decentralised, distributed and audibility features of the blockchain and quick query processing and well-designed data structure of the distributed databases. ChainSQL features a tamper-resistant and consistent multi-active database, a reliable and cost effective data-level disaster recovery backup and an auditable transaction log mechanism. The system is presented as an operational multi-active database along with the data-level disaster recovery backup and audibility features. A comprehensive experimental evaluation is performed to demonstrate the effectiveness of the system.

Lock-Free Transactional Transformation for Linked Data Structures

Nonblocking data structures allow scalable and thread-safe access to shared data. They provide individual operations that appear to execute atomically. However, it is often desirable to execute multiple operations atomically in a transactional manner. Previous solutions, such as Software Transactional Memory (STM) and transactional boosting, manage transaction synchronization separately from the underlying data structure’s thread synchronization. Although this reduces programming effort, it leads to overhead associated with additional synchronization and the need to rollback aborted transactions. In this work, we present a new methodology for transforming high-performance lock-free linked data structures into high-performance lock-free transactional linked data structures without revamping the data structures’ original synchronization design. Our approach leverages the semantic knowledge of the data structure to eliminate the overhead of false conflicts and rollbacks. We encapsulate all operations, operands, and transaction status in a transaction descriptor, which is shared among the nodes accessed by the same transaction. We coordinate threads to help finish the remaining operations of delayed transactions based on their transaction descriptors. When a transaction fails, we recover the correct abstract state by reversely interpreting the logical status of a node. We also present an obstruction-free version of our algorithm that can be applied to dynamic execution scenarios and an example of our approach applied to a hash map. In our experimental evaluation using transactions with randomly generated operations, our lock-free transactional data structures outperform the transactional boosted ones by 70% on average. They also outperform the alternative STM-based approaches by a factor of 2 to 13 across all scenarios. More importantly, we achieve 4,700 to 915,000 times fewer spurious aborts than the alternatives.

Анализ устойчивости динамических водяных знаков

In this paper we consider previously known scheme of dynamic watermarks embedding (Ra- dix-n) that is used for preventing illegal use of software. According to the scheme a watermark is dynamic linked data structure (graph), which is created in memory during program execution. Hidden data, such as information about author, can be represented in a different type of graph structure. This data can be extracted and demonstrated in judicial proceedings. This paper declared that the above mentioned scheme was previously one of the most reliable, has a number of features that allows an attacker to detect a stage of watermark construction in the program, and therefore it can be corrupted or deleted. The author of this article shows the weakness of Radix-N scheme, which consists in the fact that we can reveal dynamic data structures of a program by using information received from some API-functions hooker which catches function calls of dynamic memory allocation. One of these data structures is the watermark. Pointers on dynamically created objects (arrays, variables, class items, etc.) of a program can be detected by content analysis of computer's RAM. Different dynamic objects in memory interconnected by pointers form dynamic data structures of a program such as lists, stacks, trees and other graphs (including the watermark). Our experiment shows that in the vast majority of cases the amount of data structure in programs is small, which increases probability of a successful attack. Also we present an algorithm for finding connected components of a graph with linear time-consuming in cases where the number of nodes is about 106. On the basis of the experimental findings the new watermarking scheme has been presented, which is resistant to the proposed attack. It is offered to use different graph structure representation of a watermark, where edges are implemented using unique signatures. Our scheme uses content encrypting of graph nodes (except signature), so an attacker is not able to reveal links between nodes and find the watermark.

Simit

With existing programming tools, writing high-performance simulation code is labor intensive and requires sacrificing readability and portability. The alternative is to prototype simulations in a high-level language like Matlab, thereby sacrificing performance. The Matlab programming model naturally describes the behavior of an entire physical system using the language of linear algebra. However, simulations also manipulate individual geometric elements, which are best represented using linked data structures like meshes. Translating between the linked data structures and linear algebra comes at significant cost, both to the programmer and to the machine. High-performance implementations avoid the cost by rephrasing the computation in terms of linked or index data structures, leaving the code complicated and monolithic, often increasing its size by an order of magnitude. In this article, we present Simit, a new language for physical simulations that lets the programmer view the system both as a linked data structure in the form of a hypergraph and as a set of global vectors, matrices, and tensors depending on what is convenient at any given time. Simit provides a novel assembly construct that makes it conceptually easy and computationally efficient to move between the two abstractions. Using the information provided by the assembly construct, the compiler generates efficient in-place computation on the graph. We demonstrate that Simit is easy to use: a Simit program is typically shorter than a Matlab program; that it is high performance: a Simit program running sequentially on a CPU performs comparably to hand-optimized simulations; and that it is portable: Simit programs can be compiled for GPUs with no change to the program, delivering 4 to 20× speedups over our optimized CPU code.

3D seismic image processing for faults

Numerous methods have been proposed to automatically extract fault surfaces from 3D seismic images, and those surfaces are often represented by meshes of triangles or quadrilaterals. However, extraction of intersecting faults is still a difficult problem that is not well addressed. Moreover, mesh data structures are more complex than the arrays used to represent seismic images, and they are more complex than necessary for subsequent processing tasks, such as that of automatically estimating fault slip vectors. We have represented a fault surface using a simpler linked data structure, in which each sample of a fault corresponded to exactly one seismic image sample, and the fault samples were linked above and below in the fault dip directions, and left and right in the fault strike directions. This linked data structure was easy to exchange between computers and facilitated subsequent image processing for faults. We then developed a method to construct complete fault surfaces without holes using this simple data structure and to extract multiple intersecting fault surfaces from 3D seismic images. Finally, we used the same structure in subsequent processing to estimate fault slip vectors and to assess the accuracy of estimated slips by unfaulting the seismic images.

Command and data management system (CDMS) of the Philae lander

The paper covers the principal requirements, design concepts and implementation of the hardware and software for the central on-board computer (CDMS) of the Philae lander in the context of the ESA Rosetta space mission, including some technical details. The focus is on the implementation of fault tolerance, autonomous operation and operational flexibility by means of specific linked data structures and code execution mechanisms that can be interpreted as a kind of object oriented model for mission sequencing.