Linear Data Structure Research Articles

GraphZeppelin : How to Find Connected Components (Even When Graphs Are Dense, Dynamic, and Massive)

Finding the connected components of a graph is a fundamental problem with uses throughout computer science and engineering. The task of computing connected components becomes more difficult when graphs are very large, or when they are dynamic, meaning the edge set changes over time subject to a stream of edge insertions and deletions. A natural approach to computing the connected components problem on a large, dynamic graph stream is to buy enough RAM to store the entire graph. However, the requirement that the graph fit in RAM is an inherent limitation of this approach and is prohibitive for very large graphs. Thus, there is an unmet need for systems that can process dense dynamic graphs, especially when those graphs are larger than available RAM. We present a new high-performance streaming graph-processing system for computing the connected components of a graph. This system, which we call GraphZeppelin , uses new linear sketching data structures ( CubeSketch ) to solve the streaming connected components problem and as a result requires space asymptotically smaller than the space required for a lossless representation of the graph. GraphZeppelin is optimized for massive dense graphs: GraphZeppelin can process millions of edge updates (both insertions and deletions) per second, even when the underlying graph is far too large to fit in available RAM. As a result GraphZeppelin vastly increases the scale of graphs that can be processed.

Linear Space Data Structures for Finite Groups with Constant Query-Time

Linear Space Data Structures for Finite Groups with Constant Query-Time

Predecessor on the Ultra-Wide Word RAM

We consider the predecessor problem on the ultra-wide word RAM model of computation, which extends the word RAM model with ultrawords consisting of w2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$w^2$$\\end{document} bits (TAMC, 2015). The model supports arithmetic and boolean operations on ultrawords, in addition to scattered memory operations that access or modify w (potentially non-contiguous) memory addresses simultaneously. The ultra-wide word RAM model captures (and idealizes) modern vector processor architectures. Our main result is a simple, linear space data structure that supports predecessor in constant time and updates in amortized, expected constant time. This improves the space of the previous constant time solution that uses space in the order of the size of the universe. Our result holds even in a weaker model where ultrawords consist of w1+ϵ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$w^{1+\\epsilon }$$\\end{document} bits for any ϵ>0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\epsilon > 0 $$\\end{document}. It is based on a new implementation of the classic x-fast trie data structure of Willard (Inform Process Lett 17(2):81–84, https://doi.org/10.1016/0020-0190(83)90075-3, 1983) combined with a new dictionary data structure that supports fast parallel lookups.

String Indexing with Compressed Patterns

Given a string S of length n , the classic string indexing problem is to preprocess S into a compact data structure that supports efficient subsequent pattern queries. In this article, we consider the basic variant where the pattern is given in compressed form and the goal is to achieve query time that is fast in terms of the compressed size of the pattern. This captures the common client-server scenario, where a client submits a query and communicates it in compressed form to a server. Instead of the server decompressing the query before processing it, we consider how to efficiently process the compressed query directly. Our main result is a novel linear space data structure that achieves near-optimal query time for patterns compressed with the classic Lempel-Ziv 1977 (LZ77) compression scheme. Along the way, we develop several data structural techniques of independent interest, including a novel data structure that compactly encodes all LZ77 compressed suffixes of a string in linear space and a general decomposition of tries that reduces the search time from logarithmic in the size of the trie to logarithmic in the length of the pattern.

Verification of mutable linear data structures and iterator-based algorithms in Dafny

We address the verification of mutable, heap-allocated abstract data types (ADTs) in Dafny, and their traversal via iterators. For this purpose, we devise a verification methodology that makes it possible to implement ADTs based on already existing ones, while maintaining proper encapsulation. Then, we apply this methodology to the specification and implementation of linear collections such as stacks, queues, deques, and lists with iterators. The approach introduced in this paper allows one to progressively refine some aspects of the specification such as iterator invalidation, so that clients of the library can reason about how structural changes to a list affect existing iterators. Finally, we extend our methodology to the verification of client code (i.e., code that makes use of the implemented ADTs) and identify the boilerplate conditions common to all methods that receive and manipulate ADTs.

Lower Bounds for Semialgebraic Range Searching and Stabbing Problems

In the semialgebraic range searching problem, we are given a set of n points in ℝ d , and we want to preprocess the points such that for any query range belonging to a family of constant complexity semialgebraic sets (Tarski cells), all the points intersecting the range can be reported or counted efficiently. When the ranges are composed of simplices, the problem is well-understood: It can be solved using S(n) space and with Q(n) query time with \(S(n)Q(n)^d = \tilde{O}(n^d),\) where the \(\tilde{O}(\cdot)\) notation hides polylogarithmic factors and this trade-off is tight (up to n o (1) factors). In particular, there exist “low space” structures that use O(n) space with O ( n 1-1/ d } ) query time [ 8 , 25 ] and “fast query” structures that use O ( n d ) space with O (log n ) query time [ 9 ]. However, for general semialgebraic ranges, only “low space” solutions are known, but the best solutions [ 7 ] match the same trade-off curve as simplex queries, with O ( n ) space and \(\tilde{O}(n^{1-1/d})\) query time. It has been conjectured that the same could be done for the “fast query” case, but this open problem has stayed unresolved. Here, we disprove this conjecture. We give the first nontrivial lower bounds for semialgebraic range searching and other related problems. More precisely, we show that any data structure for reporting the points between two concentric circles, a problem that we call 2D annulus reporting, with Q ( n ) query time must use \(S(n)=\overset{\scriptscriptstyle o}{\Omega }(n^3/Q(n)^5)\) space, where the \(\overset{\scriptscriptstyle o}{\Omega }(\cdot)\) notation hides \(n^{o(1)}\) factors, meaning, for \(Q(n)=\log ^{O(1)}n\) , \(\overset{\scriptscriptstyle o}{\Omega }(n^3)\) space must be used. In addition, we study the problem of reporting the subset of input points in a polynomial slab defined by \(\lbrace (x,y)\in \mathbb {R}^2:P(x)\le y\le P(x)+w\rbrace\) , where \(P(x)=\sum _{i=0}^\Delta a_i x^i\) is a univariate polynomial of degree Δ and \(a_0, \ldots , a_\Delta , w\) are given at the query time, a problem that we call polynomial slab reporting. For this, we show a space lower bound of \(\overset{\scriptscriptstyle o}{\Omega }(n^{\Delta +1}/Q(n)^{(\Delta +3)\Delta /2})\) , which implies that for \(Q(n)=\log ^{O(1)}n\) , we must use \(\overset{\scriptscriptstyle o}{\Omega }(n^{\Delta +1})\) space. We also consider the dual semialgebraic stabbing problems of semialgebraic range searching and present lower bounds for them. In particular, we show that in linear space, any data structure that solves 2D annulus stabbing problems must use \(\Omega (n^{2/3})\) query time. Note that this almost matches the upper bound obtained by lifting 2D annuli to 3D. Like semialgebraic range searching, we also present lower bounds for general polynomial slab stabbing problems. Again, our lower bounds are almost tight for linear size data structures in this case.

Temperature-induced response reconstruction method based on DL-AR model and attention mechanism

Data loss is unavoidable for structural health monitoring (SHM) systems due to various potential factors, such as equipment aging, radio interference. Previous studies have revealed that it is more challenging to extract structural state information from incomplete data. Therefore, it is important to develop high-precision response reconstruction methods for missing data. In this study, a hybrid deep-learning and autoregressive model with attention mechanism (DL-AR-ATT) framework is proposed to accurately reconstruct structural responses considering data correlations. The proposed model consists of the deep-learning (DL) and autoregressive (AR) components, which can simultaneously model the linear and nonlinear structure of data. To adaptively weight the contribution of the two components to the final result instead of directly summing, the study introduces a custom weighting layer into DL-AR-ATT. Moreover, the attention mechanism is introduced to eliminate redundant information caused by the data from multiple sensors as input. To verify the validity and feasibility of the proposed method, long-term SHM data from a long-span steel box girder suspension bridge and a prestressed concrete continuous box girder bridge are used. The results show that the reconstruction performance of DL-AR-ATT is superior to that of the one-dimensional convolutional neural network (1D-CNN)-based and hybrid DL and AR (DL-AR) models. Finally, the weights of the modules of DL-AR and DL-AR-ATT are visualized to verify the ability of the model to capture data correlations, and to explain how the attention and custom weighting layers improve the performance of DL-AR-ATT.

LCTree-Based Approach for Mining Frequent Items in Real-Time.

With the increase of real-time stream data, knowledge discovery from stream data becomes more and more important, which requires an efficient data structure to store transactions and scan sliding windows once to discover frequent itemsets. We present a new method named Linking Compact Tree (LCTree). We designed an algorithm by using an improved data structure to create objective tree, which can find frequent itemsets with linear complexity. Secondly, we can merge items in sliding windows by one scan with Head Linking List data structure. Third, by implementing data structure of Tail Linking List, we can locate the obsolete nodes and remove them easily. Finally, LCTree is able to find all exact frequent items in data stream with reduced time and space complexity by using such a linear data structure. Experiments on datasets with different sizes and types were conducted to compare the proposed LCTree technique with well-known frequent item mining methods including Cantree, FP-tree, DSTree, CPSTree, and Gtree. The results of experiments show presented algorithm has better performance than other methods, and also confirm that it is a promising solution for detecting frequent item sets in real time applications.

A New Vista of Performing Insertion and Deletion in Linked Lists

Insertion and deletion in linked lists. Linked lists are an example of linear data structures. They are made up of nodes which are connected to each other. The data can be accessed by sequential manner. In this paper we will create an abstract model of linked list where the creation of a linked list of n nodes will take a time complexity of O(n) and the insertion and deletion will take a time complexity of O(1) or constant time at the tail position and head position and also reduce the runtime of insertion and deletion at middle or any index in between to some extent. This abstract model of the linked list can also be used as a stack and a queue because it will contain all properties of stack and queue data structures.

Polynomial Data Structure Lower Bounds in the Group Model

Proving superlogarithmic data structure lower bounds in the static group model has been a fundamental challenge in computational geometry since the early '80s. We prove a polynomial ($n^{\Omega(1)}$) lower bound for an explicit range counting problem of $n^3$ convex polygons in $\mathbb{R}^2$ (each with $n^{\tilde{O}(1)}$ facets/semialgebraic complexity), against linear storage arithmetic data structures in the group model. Our construction and analysis are based on a combination of techniques in Diophantine approximation, pseudorandomness, and compressed sensing---in particular, on the existence and partial derandomization of optimal binary compressed sensing matrices in the polynomial sparsity regime ($k = n^{1-\delta}$). As a byproduct, this establishes a (logarithmic) separation between compressed sensing matrices and the stronger RIP property.

Markov Chain Neighborhood Sparse Preserving Graph Embedding Based on Tensor Factorization for Batch Process Monitoring

If the three-dimension data of batch process are unfolded the two-dimension data, some important information would lose, and outliers such as noise would lead to poor monitoring results. Therefore, a Markov chain neighborhood sparse preserving graph embedding algorithm based on tensor factorization (TMNSPGE) is proposed. Firstly, tensor factorization is used to directly process the three-dimension data in batch process, which can avoid the information loss. Secondly, by using the neighborhood preserving embedding algorithm and sparse manifold coding, the local linear relationship and local sparse manifold structure of data are preserved. On this basis, Markov chain analysis is introduced to construct a similar graph to make the data after dimensionality reduction have a certain probability interpretation. Finally, the statistics and control limits are determined to realize process monitoring. Numerical example and penicillin fermentation simulation process prove the effectiveness of TMNSPGE algorithm in batch process monitoring.

Orchestrating product provenance story: When IOTA ecosystem meets electronics supply chain space

Trustworthy data is the fuel for ensuring transparent traceability, precise decision-making, and cogent coordination in the Supply Chain (SC) space. However, the disparate data silos act as a trade barrier in orchestrating the provenance of the product lifecycle; starting from the raw materials to end products available for customers. Besides product traceability, the legacy SCs face several other problems including data validation, data accessibility, security, and privacy issues. In this regard, Blockchain – an advanced Distributed Ledger Technology (DLT) works well to address these challenges by linking fragmented and siloed SC events in an immutable audit trail. However, the underlying challenges with blockchain such as scalability, inability to access off-line data, vulnerability to quantum attacks, and high transaction fees necessitate a new solution to overcome the inefficiencies of the current blockchain design. In this regard, IOTA (the third generation of DLT) leverages a Directed Acyclic Graph (DAG)-based data structure in contrast to linear data structure of blockchain to address such challenges and facilitate a scalable, quantum-resistant, and miner-free solution for the Internet of Things (IoT). After realizing the crucial requirement of traceability and considering the limitations of blockchain in SC, in this work, we propose a provenance-enabled framework for the Electronics Supply Chain (ESC) through a permissioned IOTA ledger. To that end, we construct a transparent product ledger based on trade event details along with time-stamped SC processes to identify operational disruptions or counterfeiting issues. We further exploit the Masked Authenticated Messaging (MAM) protocol provided by IOTA that allows the SC players to procure distributed information while keeping confidential trade flows, ensuring restrictions on data retrieval, and facilitating the integration of fine-grained or coarse-grained data accessibility. Our experimental results show that the time required to construct secure provenance data aggregated from multiple SC entities takes 3s (on average) for a local node and 4s for a remote node, which is justifiable. Furthermore, we perform experiments on Raspberry Pi 3B to verify that the estimated energy consumption at resource-constrained devices is tolerable while implementing the proposed scheme.

A linear-space data structure for range-LCP queries in poly-logarithmic time

Let T[1,n] be a text of length n and T[i,n] be the suffix starting at position i. Also, for any two strings X and Y, let LCP(X,Y) denote their longest common prefix. The range-LCP of T w.r.t. a range [α,β], where 1≤α<β≤n isrlcp(α,β)=max⁡{|LCP(T[i,n],T[j,n])||i≠jandi,j∈[α,β]} Amir et al. [2] introduced the indexing version of this problem, where the task is to build a data structure over T, so that rlcp(α,β) for any query range [α,β] can be reported efficiently. They proposed an O(nlog1+ϵ⁡n) space structure with query time O(log⁡log⁡n), and a linear space (i.e., O(n) words) structure with query time O(δlog⁡log⁡n), where δ=β−α+1 is the length of the input range and ϵ>0 is an arbitrarily small constant. Later, Patil et al. [5] proposed another linear space structure with an improved query time of O(δlogϵ⁡δ). This poses an interesting question, whether it is possible to answer rlcp(⋅,⋅) queries in poly-logarithmic time using a linear space data structure. In this paper, we settle this question by presenting an O(n) space data structure with query time O(log1+ϵ⁡n) and construction time O(nlog⁡n).

Feature Vector Creation Using Hierarchical Data Structure for Spatial Domain Image Retrieval

Building efficient Image Features with optimal dimension and high discriminative power is important in processing the digital information hidden inside the images. Image Retrieval systems use the feature vectors to retrieve similar images based on feature vector similarity. Image Feature Vector generation is important step in CBIR. Most of the CBIR use Linear Data structure for Feature Vector. In this paper the Hierarchical data Structure: Binary Tree is used for Feature vector creation in Image Retrieval Systems. Images in Spatial Domain are considered for Feature Vector Generation. Low-level Feature: Color Descriptors are used as Image Contents, to represents the image in feature vector form. Performance of the image retrieval is tested for Color Images. It has been observed that, along with the Dimensionally reduction, the proposed approach of Feature Vector generation, improves the discriminative power of Feature Vector. For RGB colorspaces the cascaded statistical features, when stores in hierarchical data structure, this provides the facility to perform the calculations in parallel manner also capable to keep discriminative power comparable w.r.t size of feature vector.

Operation of Queue and Stack by DNA Tiles

DNA is used as self-nanomaterials to assemble into specific structures. DNA tile provides a new idea for the application of DNA tile in the field of computing. Recent years, Queue and Stack are important linear data structures which are used in various software systems widely. The implementation of DNA based queue and stack has been studied continuously for many years. In the traditional DNA computing, queue and stack are mostly realized by DNA strands displacement, restriction endonuclease and ligase were used. However, as an active material, it has a high requirement for enzyme experimental conditions. The purpose of this paper is to implement queue and stack structures using non-enzyme systems. The rule of Queue is characterized by FIFO (first in first out), which allows for insertion at one end of the list and deletion at the other. The rule of Stack is characterized by FILO(first in last out), which allows for insertion and deletion at one end of the list. We are aimed to implement Queue and Stack using self-assembly and disassembly via DNA Tiles. No enzymes are needed for the whole experiment. As an enzyme-free system, it provides a new method to implement stack and queue.

Limitations of Blockchain Technology with its Applications.

Block chain is also called as chain of blocks i.e. it is collection of technologies or collection of Technologies or a bit like a block chain sack of Logo From the sack y can collect different bricks and arrange them some particular way/different way to create a chain blocks .It is decentralized i.e. no any central control or authority assigned to it .It is simple. Easy to understand and work the system. New blocks are added at the end of the block chain .It uses data structure-link list-also called as linear data structure, stored at contiguous location in work on concept of pointers. They form a chain in continuous way using array ,doubly link list-pointers are used for fast and speedy execution ,it traverse in forward direction –move ahead, backward direction-move back, operations like addition &amp; deletion of node can be efficiently done

Scaffolded Block-based Instructional Tool for Linear Data Structures: A Constructivist Design to Ease Data Structures’ Understanding

Data Structures courses commonly introduce topics involving high levels of abstraction and complexity, requiring significant effort from instructors and apprentices to achieve positive outcomes from the teaching-learning process. Despite the multiple studies that have occurred within the Computer Science Education (CSE) community to understand the experiences novice programmers may have when learning how to program, there is still a lack of exploration and research on understanding these experiences in scenarios different from first-year Computer Science (CS) courses. Looking further from CS introductory courses, this paper presents the results of a pilot study that evaluated the interaction of a group of CS Colombian students with DStBlocks, which is a scaffolded block-based instructional technology, designed and developed to ease linear data structures understanding. The findings and results of this pilot study are favorable, corresponding to tests centered on user experience and learning impact.

Forecasting dengue epidemics using a hybrid methodology

Dengue case management is an alarmingly important global health issue. The effective allocation of resources is often difficult due to external and internal factors imposing nonlinear fluctuations in the prevalence of dengue fever. We aimed to construct an early-warning system that could accurately forecast subsequent dengue cases in three dengue endemic regions, namely San Juan, Iquitos, and the Philippines. The problem is solely regarded as a time series forecasting problem ignoring the known epidemiology of dengue fever as well as the other meteorological variables. Autoregressive integrated moving average (ARIMA) model is a popular classical time series model for linear data structures whereas with the advent of neural networks, nonlinear structures in the data set can be handled. In this paper, we propose a novel hybrid model combining ARIMA and neural network autoregressive (NNAR) model to capture both linearity and nonlinearity in the data sets. The ARIMA model filters out linear tendencies in the data and passes on the residual values to the NNAR model. The proposed hybrid approach is applied to three dengue time-series data sets and is found to give better forecasting accuracy in comparison to the state-of-the-art. The results of this study indicate that dengue cases can be accurately forecasted over a sufficient time period using the proposed hybrid methodology.

Correntropy-Induced Robust Low-Rank Hypergraph.

Hypergraph learning has been widely exploited in various image processing applications, due to its advantages in modeling the high-order information. Its efficacy highly depends on building an informative hypergraph structure to accurately and robustly formulate the underlying data correlation. However, the existing hypergraph learning methods are sensitive to non- Gaussian noise, which hurts the corresponding performance. In this paper, we present a noise-resistant hypergraph learning model, which provides superior robustness against various non- Gaussian noises. In particular, our model adopts low-rank representation to construct a hypergraph, which captures the globally linear data structure as well as preserving the grouping effect of highly-correlated data. We further introduce a correntropyinduced local metric to measure the reconstruction errors, which is particularly robust to non-Gaussian noises. Finally, the Frobenious-norm based regularization is proposed to combine with the low-rank regularizer, which enables our model to regularize the singular values of the coefficient matrix. By such, the non-zero coefficients are selected to generate a hyperedge set as well as the hyperedge weights. We have evaluated the proposed hypergraph model in the tasks of image clustering and semi-supervised image classification. Quantitatively, our scheme significantly enhances the performance of the state-of-the-art hypergraph models on several benchmark datasets.

A Linear Space Data Structure for Range LCP Queries*

A Linear Space Data Structure for Range LCP Queries*