Proposed Data Structure Research Articles

Visualizing dynamic geosciences phenomena using an octree-based view-dependent LOD strategy within virtual globes

Geoscientists build dynamic models to simulate various natural phenomena for a better understanding of our planet. Interactive visualizations of these geoscience models and their outputs through virtual globes on the Internet can help the public understand the dynamic phenomena related to the Earth more intuitively. However, challenges arise when the volume of four-dimensional data (4D), 3D in space plus time, is huge for rendering. Datasets loaded from geographically distributed data servers require synchronization between ingesting and rendering data. Also the visualization capability of display clients varies significantly in such an online visualization environment; some may not have high-end graphic cards. To enhance the efficiency of visualizing dynamic volumetric data in virtual globes, this paper proposes a systematic framework, in which an octree-based multiresolution data structure is implemented to organize time series 3D geospatial data to be used in virtual globe environments. This framework includes a view-dependent continuous level of detail (LOD) strategy formulated as a synchronized part of the virtual globe rendering process. Through the octree-based data retrieval process, the LOD strategy enables the rendering of the 4D simulation at a consistent and acceptable frame rate. To demonstrate the capabilities of this framework, data of a simulated dust storm event are rendered in World Wind, an open source virtual globe. The rendering performances with and without the octree-based LOD strategy are compared. The experimental results show that using the proposed data structure and processing strategy significantly enhances the visualization performance when rendering dynamic geospatial phenomena in virtual globes.

An efficient hardware implementation for deblocking filter of AVS decoder

Abstract In this paper, an optimized hardware implementation for deblocking filter of AVS decoder was proposed according to the arithmetic in AVS audio video coding standard. In order to reduce the bandwidth requirement of SDRAM, internal RAM is used to cache reference data for deblocking filter in the hardware design. Meanwhile, the storage structure of reference data is reasonably arranged for accelerating filtering process. Based on the proposed data structure, both the SDRAM data writing operation and the macroblock filtering operation can be done at the same time. The design is implemented on an Altera Stratix II FPGA, the synthesis and simulation results indicated that the hardware cost was low, and the design can meet the demand of the real-time video decoding.

Data structures for the most frequently used algorithm

This paper proposes three sets of data structures that could be used for the Most Frequently Used algorithm, and could serve as projects in computer science courses such as Data Structures, Operating Systems, and Algorithm Analysis. The proposed data structures are introduced first. Algorithm analysis is then provided. The paper concludes with the benefits of these data structures.

Data Structures for Extension Violations in a Query Range

Design Rule Checking (DRC) in VLSI design involves checking if a given VLSI layout satisfies a given set of rules and reporting the violations if any. We propose data structures for reporting violations of the minimum extension rule in a query window of interest.

Human motion database with a binary tree and node transition graphs

Database of human motion has been widely used for recognizing human motion and synthesizing humanoid motions. In this paper, we propose a data structure for storing and extracting human motion data and demonstrate that the database can be applied to the recognition and motion synthesis problems in robotics. We develop an efficient method for building a human motion database from a collection of continuous, multi-dimensional motion clips. The database consists of a binary tree representing the hierarchical clustering of the states observed in the motion clips, as well as node transition graphs representing the possible transitions among the nodes in the binary tree. Using databases constructed from real human motion data, we demonstrate that the proposed data structure can be used for human motion recognition, state estimation and prediction, and robot motion planning.

A Bounded and Adaptive Memory-Based Approach to Mine Frequent Patterns From Very Large Databases

Most of the existing methods to solve the problem of association rules mining (ARM) rely on special data structures to project the database (either totally or partially) in the primary memory. Traditionally, these data structures reside in the main memory and rely on the existing paging mechanism of the virtual memory manager (VMM) to handle the storage problem when they go out of the primary memory. Typically, VMM stores the overloaded data into the secondary memory based on some preassumed memory usage criteria. However, this direct and unplanned use of virtual memory results in an unpredictable behavior or thrashing, as depicted by some of the works described in the literature. This problem is tackled in this paper by presenting an ARM model capable of mining a transactional database, regardless of its size and without relying on the underlying VMM; the proposed approach could use only a bounded portion of the primary memory and this gives the opportunity to assign other parts of the main memory to other tasks with different priority. In other words, we propose a specialized memory management system which caters to the needs of the ARM model in such a way that the proposed data structure is constructed in the available allocated primary memory first. If at any point the structure grows out of the allocated memory quota, it is forced to be partially saved on secondary memory. The secondary memory version of the structure is accessed in a block-by-block basis so that both the spatial and temporal localities of the I/O access are optimized. Thus, the proposed framework takes control of the virtual memory access and hence manages the required virtual memory in an optimal way to the best benefit of the mining process to be served. Several clever data structures are used to facilitate these optimizations. Our method has the additional advantage that other tasks of different priorities may run concurrently with the main mining task with as little interference as possible because we do not rely on the default paging mechanism of the VMM. The reported test results demonstrate the applicability and effectiveness of the proposed approach.

THE VIRTUAL SUFFIX TREE

We introduce the VST (virtual suffix tree), an efficient data structure for suffix trees and suffix arrays. Starting from the suffix array, we construct the suffix tree, from which we derive the virtual suffix tree. Later, we remove the intermediate step of suffix tree construction, and build the VST directly from the suffix array. The VST provides the same functionality as the suffix tree, including suffix links, but at a much smaller space requirement. It has the same linear time construction even for large alphabets, Σ, requires O(n) space to store (n is the string length), and allows searching for a pattern of length m to be performed in O(m log |Σ|) time, the same time needed for a suffix tree. Given the VST, we show an algorithm that computes all the suffix links in linear time, independent of Σ. The VST requires less space than other recently proposed data structures for suffix trees and suffix arrays, such as the enhanced suffix array [1], and the linearized suffix tree [17]. On average, the space requirement (including that for suffix arrays and suffix links) is 13.8n bytes for the regular VST, and 12.05n bytes in its compact form.

A Novel Scalable and Storage-Efficient Architecture for High Speed Exact String Matching

String matching is a fundamental element of an important category of modern packet processing applications which involve scanning the content flowing through a network for thousands of strings at the line rate. To keep pace with high network speeds, specialized hardware-based solutions are needed which should be efficient enough to maintain scalability in terms of speed and the number of strings. In this paper, a novel architecture based upon a recently proposed data structure called the Bloomier filter is proposed which can successfully support scalability. The Bloomier filter is a compact data structure for encoding arbitrary functions, and it supports approximate evaluation queries. By eliminating the Bloomier filter's false positives in a space efficient way, a simple yet powerful exact string matching architecture is proposed that can handle several thousand strings at high rates and is amenable to on-chip realization. The proposed scheme is implemented in reconfigurable hardware and we compare it with existing solutions. The results show that the proposed approach achieves better performance compared to other existing architectures measured in terms of throughput per logic cells per character as a metric.

Predicting Missing Items in Shopping Carts

Existing research in association mining has focused mainly on how to expedite the search for frequently co-occurring groups of items in ldquoshopping cartrdquo type of transactions; less attention has been paid to methods that exploit these ldquofrequent itemsetsrdquo for prediction purposes. This paper contributes to the latter task by proposing a technique that uses partial information about the contents of a shopping cart for the prediction of what else the customer is likely to buy. Using the recently proposed data structure of itemset trees (IT-trees), we obtain, in a computationally efficient manner, all rules whose antecedents contain at least one item from the incomplete shopping cart. Then, we combine these rules by uncertainty processing techniques, including the classical Bayesian decision theory and a new algorithm based on the Dempster-Shafer (DS) theory of evidence combination.

CONSTANT TIME CLASH DETECTION IN PROTEIN FOLDING

Applications for the manipulation of molecular structures are usually computationally intensive. Problems like protein docking or ab-initio protein folding need to frequently determine if two atoms in the structure collide. Therefore, an efficient algorithm for this problem, usually referred as clash detection, can greatly improve the application efficiency. This work focus mainly on the ab-initio protein folding problem. A naive approach for the clash problem, the most commonly-used by molecular structure programs, consists in calculating the distance between every pair of atoms. We propose an efficient data structure that uses a three-dimensional array to store the atoms' position. We compare the proposed data structure with one of the best known general data structures for this type of problems (SAT tree) and with the naive approach. While the naive approach takes linear time to the number of atoms to verify if a new atom clashes with any previously-set atoms, the proposed data structure takes constant time to perform the same verification. The SAT tree takes logarithmic time for the same task. The results show that the proposed data structure surpasses the other techniques for any protein size. The proposed data structure takes near half of the time of the SAT data structure and close to a fifth of the time of the naive approach for the larger proteins. We believe that this data structure can improve the existing molecular structure applications by decreasing the computational cost needed for clash detection. The data structure presented in this work can be used for any protein structure clash verification, as long as the atoms that need to be checked are kept in the 3D array. This data structure is particulary useful when manipulating large sets of atoms, for example, in applications like loop prediction, structure refinement of large proteins, and protein docking.

A versatile hierarchical mesh for subdivision surfaces

Most of the needs that occur in applications using subdivision surfaces are covered by our hierarchical mesh data structure. We tested the proposed data structure in some implementations for adaptive visualizations where a highly efficient data structure is essential. Primarily, this means our mesh must support a fast hierarchical refinement of the object with a fast access to the vertices in the different subdivision levels. Only the parts of the object with a contribution to the light transport are computed and stored up to an adequate subdivision depth. A rapid and stable navigation on the mesh must be possible, and storing of ancillary information is required. The modular concept of our mesh enables us to exchange the rules of the subdivision scheme easily. With our hierarchical mesh, the rules for Catmull-Clark, Loop and ESubs, which is another subdivison scheme, were easy to implement.

DMP-tree: A dynamic M-way prefix tree data structure for strings matching

We propose DMP-tree, a dynamic M-way prefix tree, data structure for the string matching problem in general and prefix matching in particular. DMP-tree has been initially devised for fast and efficiently handling prefix matching which constitutes the building block of some applications in the computer realm and related area. It is assumed there are strings of an alphabet Σ which are ordered. The data strings can have different lengths and some of them can be prefixes of others. Two well known applications of prefix matching are layers 3 and 4 switching in TCP/IP protocols. In layer 3 switching, routers forward an IP packet by checking its destination address and finding the longest matching prefix from a database. In layer 4 switching, the source and destination addresses are used to classify packets for differentiated service and Quality of Services (QoS). DMP-tree is a superset of B-tree. When none of the data strings are a prefix of each other, DMP-tree is the same as B-tree. In DMP-tree, no data string can be in a higher level than another data string which is its prefix. This requires a special procedure for node splitting. Indeed, node splitting differentiates DMP-tree from B-tree. The proposed data structure is simple, well defined, easy to implement in hardware or software and efficient in terms of memory usages and search time compared to other data structures proposed for prefix matching. We have implemented DMP-tree and the experimental results for simulated IP prefixes from the {0, 1} character set show an average search time of Log M N for a large number of N, number of data elements, when the internal node branching factor M is big enough (⩾5).

An efficient data structure for network anomaly detection

AbstractDespite the rapid advance in networking technologies, detection of network anomalies at high‐speed switches/routers is still far from maturity. To push the frontier, two major technologies need to be addressed. The first one is efficient feature‐extraction algorithms/hardware that can match a line rate in the order of Gb/second; the second one is fast and effective anomaly detection schemes. In this paper, we focus on design of efficient data structure and algorithms for feature extraction. Specifically, we propose a novel data structure that extracts the so‐called two‐directional (2D) matching features, which are shown to be effective indicators of network anomalies. Our key idea is to use a Bloom filter array (BFA) to trade‐off a small amount of accuracy in feature extraction, for much less space and time complexity, so that our data structure can catch up with a line rate in the order of Gb/second. Different from the existing work, our data structure has the following properties: (1) it dynamic Bloom filter, (2) combination of a it sliding window with Bloom filter, and (3) using an insertion–removal pair to enhance Bloom filter with a removal operation. Our analysis and simulation demonstrate that the proposed data structure has a better space/time trade‐off than conventional algorithms. For example, for a fixed time complexity, the conventional algorithm (i.e., hash table [1—8]) requires a memory of 1.01 Gbits while our data structure requires a memory of only 62.9 Mbits, at the cost of losing 1% accuracy in feature extraction. Copyright © 2008 John Wiley & Sons, Ltd.

Dynamic Segment Trees for Ranges and Prefixes

In this paper, we develop a segment tree data structure for solving dynamic table lookup problems. The proposed dynamic segment tree (DST) uses all of the distinct end points of ranges as the keys based on a new range end point scheme. The new end point scheme generates fewer end points than the traditional end point scheme. DST is implemented as a balanced binary search tree augmented with a range set in each node. The performance of accessing and updating the ranges stored in each node is improved by an efficient range set data structure that combines the priority queue and the interval tree. Based on the proposed data structures, the time complexities of search, insertion, and deletion in a set of N arbitrary ranges are O(log N), O(log N times log Max), and O(Max times log N times log Max), respectively, where Max is the maximum number of ranges covering any address. In practical routing tables, Max is a small constant (six for the routing tables we tested). The memory requirement for DST is O(N log N). The experimental results using real Internet Protocol version 4 (IPv4) routing tables show that both the DST and prefix binary tree on binary tree (PBOB) by Lu et al. (2004) perform much better than the multiway range tree (MRT) by Warkhede et al. (2004) and prefix in B-tree (PIBT) by Lu et al. (2005) in terms of update speed and memory consumption, but DST performs much better than PBOB and a little slower than MRT and PIBT in terms of search speed

Product Development Cost Estimation in Mass Customization

The ultimate goal of mass customization is to achieve economies of both scope and scale. This goal implies a conflict between customization and economy of scale (or mass production) in the traditional manufacturing paradigm. However, recent developments in computer and Internet communication technologies, concurrent engineering, and modular design methodology provide concepts, methods and technology infrastructure for realizing mass customization. One of the findings from numerous research efforts on mass customization is the use of e-commerce technologies to manage a product development chain that links customers, suppliers and manufacturers together to approach concurrently customized products in a short time and at the low cost level of mass production, i.e., mass customization. To ensure the success of mass customization in a product development chain, a rapid, automatic yet accurate cost estimate and control system is needed. This paper presents a cost index structure, which is a novel data structure, together with two novel cost estimate methods, viz. the generative cost estimate method and the variant cost estimate method, for the development of a semi or fully automatic computer-aided cost estimate and control system in mass customization. Finally, an industrial case is reported in the paper to illustrate the principles and feasibility of the proposed data structure, methods and system framework

The empty bin: A data structure for spatial search of time‐varying data

AbstractAn extension of the classic bin data structure with linked list storage is proposed and tested. This ‘empty‐bin’ data structure is particularly well suited for problems where spatial search is required for items that: (a) do not change significantly in numbers; (b) have similar size; and (c) move a fraction of the bin size in every timestep. This ‘empty‐bin’ data structure can be updated and rebalanced efficiently on shared‐memory parallel machines. Several examples demonstrate the effectiveness of the proposed data structure. Copyright © 2006 John Wiley & Sons, Ltd.

High-Volume Continuous XPath Querying in XML Message Brokers

The core technical issue in XML message brokers, which play a key role in exchanging information in ubiquitous environments, is processing a large set of continuous XPath queries over incoming XML streams. In this paper, a new system as an epochal solution for this issue is proposed. The system is designed to minimize the runtime workload of continuous query processing by transforming XPath expressions and XML streams into newly proposed data structures and matching them efficiently. Also, system performances are estimated both in terms of space and time, and verified through a variety of experimental studies, showing that the proposed system is practically linear-scalable and stable in terms of processing a set of XPath queries in a continuous and timely fashion.

Dynamic count filters

Bloom filters are not able to handle deletes and inserts on multisets over time. This is important in many situations when streamed data evolve rapidly and change patterns frequently. Counting Bloom Filters (CBF) have been proposed to overcome this limitation and allow for the dynamic evolution of Bloom filters. The only dynamic approach to a compact and efficient representation of CBF are the Spectral Bloom Filters (SBF).In this paper we propose the Dynamic Count Filters (DCF) as a new dynamic and space-time efficient representation of CBF. Although DCF does not make a compact use of memory, it shows to be faster and more space efficient than any previous proposal. Results show that the proposed data structure is more efficient independently of the incoming data characteristics.

P2P를 이용한 배포 서버의 부하 분산

분산 정보 시스템의 온-라인 유지보수를 위해서는 네트워크 내의 참여 노드들에게 파일을 배포하는 작업이 필수적이다. 이때 파일 배포에 대한 사용자들의 요청이 단기간에 집중되면, 배포 서버는 과부하 상태에 빠지며, 이를 플래시 크라우드(Flash Crowds)라 부른다. 플래시 크라우드를 회피하기 위한 일반적인 해결책은 하드웨어의 용량을 증설하는 것이다. 본 논문에서는 추가 비용의 발생 없이 P2P 기반의 소프트웨어적 해결책을 제안한다. 제안된 해결책에서 네트워크의 노드들은 인접한 노드들을 중심으로 서브네트워크들로 구성된다. 각 서브네트워크 내에서 배포 파일의 복사본은 노드들 상호간에 전송될 수 있어 배포 서버의 부하를 분산시킨다. 효율성을 높이기 위해 배포 대상 파일들은 하나의 패키지로 묶여지고 전송에 앞서 패키지는 동일한 크기를 갖는 다수의 세그먼트들로 분할된다. 정상 상태에서 배포 서버는 노드가 요청한 패키지를 세그먼트 단위로 전송한다. 그러나 배포 서버의 과부하 상태에서 노드가 필요한 세그먼트가 이미 서브네트워크 내에 존재할 경우, 서브네트워크 내의 노드는 필요한 세그먼트를 인접 노드로부터 전송받을 수 있다. 본 논문에서는 이를 처리하기 위한 자료구조와 알고리즘을 제안하고 시뮬레이션을 통해 성능 개선을 확인하였다. To perform on-line maintenance for Distributed Information System, it is indispensable to disseminate files to participant nodes in the network. When users' requests for file deployment occur simultaneously in a short period a deployment server falls into overload phase, which is often called Flash Crowds. h common solution to avoid Flash Crowds is to increase hardware capacity. In this paper, we propose a software solution based on P2P, which does not cost any additional expense. In the proposed solution, nodes in the network are grouped into subnetworks one of which is composed of only neighboring nodes. In each subnetwork, copies of deployment files can be transferred to each other. Consequently, it brings about the effect of load balancing in deployment server. To raise the effectiveness, target files for deployment are packed into one package. Before being transferred, each package is divided into multiple equal-sized segments. A deployment server in a normal phase transmits a package requested from nodes in segment units. However a deployment server is overloaded, if segments already exist in the subnetwork participant nodes in the subnetwork receive necessary segments from neighboring nodes. In this paper, we propose data structures and algorithm for this approach and show performance improvement through simulation.

An efficient method of Huffman decoding for MPEG-2 AAC and its performance analysis

This paper presents a new method for Huffman decoding specially designed for the MPEG-2 AAC audio. The method significantly enhances the processing efficiency of the conventional Huffman decoding realized with the ordinary binary tree search method. A data structure of one-dimensional array is newly designed based on the numerical interpretation of the incoming bit stream and its utilization for the offset oriented nodes allocation. The Huffman tree implemented with the proposed data structure allows the direct computation of the branching location, eliminating the need for the pipeline-violating compare and jump instructions. The experimental results show the average performance enhancement of 67% and 285%, compared to those of the conventional binary tree search method and the sequential search method, respectively. The proposed method also shows slightly better processing efficiency, while requiring much less memory space, compared even with those up-to-date efficient search methods of Hashemian and its variants.