Heap Data Research Articles

Bounded scheduling two-component jobs simultaneously or hierarchically

This paper studies the multi-objective problem of scheduling n two-component jobs on a fabrication machine with bounded batch capacity in terms of the number of jobs. Each job consists of two components, one is common among all jobs (called standard component) and the other is unique to itself (called specific component). The machine processes all components so that it might become a capacitated bottleneck machine. Standard components have equal processing times. They are fabricated in batches sequentially on the machine, and a setup time is required to fabricate a batch. Each batch contains the standard components of at most b(b < n) jobs. Specific components are manufactured individually, therefore their associated setup times could be absorbed into their processing times. Standard components in a batch are not available until all these components are completed (meaning that the processing time of a batch is equal to the total processing time of the standard components in it), whereas a specific component is available on completion of its processing. A job is completed when both its two components are completed and available. For simultaneous optimization of makespan and maximum lateness, an O(n3)-time algorithm is obtained. Through a careful use of the heap data structure, the time bound can be improved to O(n2 log n). For hierarchical optimization of two maximum lateness objectives and makespan, an O(n5)-time algorithm is obtained, which can be improved to run in time O(n4).

On-the-Fly Static Analysis via Dynamic Bidirected Dyck Reachability

Dyck reachability is a principled, graph-based formulation of a plethora of static analyses. Bidirected graphs are used for capturing dataflow through mutable heap data, and are usual formalisms of demand-driven points-to and alias analyses. The best (offline) algorithm runs in O ( m + n · α( n )) time, where n is the number of nodes and m is the number of edges in the flow graph, which becomes O ( n 2 ) in the worst case. In the everyday practice of program analysis, the analyzed code is subject to continuous change, with source code being added and removed. On-the-fly static analysis under such continuous updates gives rise to dynamic Dyck reachability, where reachability queries run on a dynamically changing graph, following program updates. Naturally, executing the offline algorithm in this online setting is inadequate, as the time required to process a single update is prohibitively large. In this work we develop a novel dynamic algorithm for bidirected Dyck reachability that has O ( n · α( n )) worst-case performance per update, thus beating the O ( n 2 ) bound, and is also optimal in certain settings. We also implement our algorithm and evaluate its performance on on-the-fly data-dependence and alias analyses, and compare it with two best known alternatives, namely (i) the optimal offline algorithm, and (ii) a fully dynamic Datalog solver. Our experiments show that our dynamic algorithm is consistently, and by far, the top performing algorithm, exhibiting speedups in the order of 1000X. The running time of each update is almost always unnoticeable to the human eye, making it ideal for the on-the-fly analysis setting.

Development and evaluations of an approach with full utilization of point cloud for measuring the angle of repose

The angle of repose (AOR) is an indicator of the fluidity of granular materials. The uncertainty for determining the material AOR includes the deviation in the global shape of repeatedly generated heaps and measurement uncertainty for each heap. A suitable method of measuring the AOR helps to decrease the latter. For this aim, a measurement method with full utilization of a three-dimensional (3D) point cloud was developed. About the approach, there was a sector-shaped domain for batch detection in a cylindrical coordinate system. The progressively rotating domain could calculate the slope AORs. Then the average of total slope AORs was determined as the heap AOR. A series of simulated heaps with varying particle shapes, heap sizes, particle sizes, and known AORs were measured by the approach for evaluating its performance. The heap 3D data from a publicly available dataset was used, which contained fresh corn grains, lignite, wood chips, and limestone with 4 replicate heaps for each material. The two other published 3D methods for calculating the AORs were used as comparison items for the developed method. Factors of measurement uncertainty depended on these three methods. And the first heap data of each material was configured with a gradient of 6 for this factor. The coefficient of variation (CV) of the measurements by each method for 6 gradients of a heap was used to assess its measurement uncertainty. The replicate heaps of each material were measured using these methods. The evaluation of the measurement uncertainty showed that the CV of the measurements by the developed method for 6 gradients of a heap was significantly lower than it was by the two other methods. This article gives and evaluates a method for measuring the AOR, and it can decrease the measurement uncertainty.

Model-guided synthesis of inductive lemmas for FOL with least fixpoints

Recursively defined linked data structures embedded in a pointer-based heap and their properties are naturally expressed in pure first-order logic with least fixpoint definitions (FO+lfp) with background theories. Such logics, unlike pure first-order logic, do not admit even complete procedures. In this paper, we undertake a novel approach for synthesizing inductive hypotheses to prove validity in this logic. The idea is to utilize several kinds of finite first-order models as counterexamples that capture the non-provability and invalidity of formulas to guide the search for inductive hypotheses. We implement our procedures and evaluate them extensively over theorems involving heap data structures that require inductive proofs and demonstrate the effectiveness of our methodology.

Reduced Memory Viterbi Decoding for Hardware-accelerated Speech Recognition

Large Vocabulary Continuous Speech Recognition systems require Viterbi searching through a large state space to find the most probable sequence of phonemes that led to a given sound sample. This needs storing and updating of a large Active State List (ASL) in the on-chip memory (OCM) at regular intervals (called frames), which poses a major performance bottleneck for speech decoding. Most works use hash tables for OCM storage while beam-width pruning to restrict the ASL size. To achieve a decent accuracy and performance, a large OCM, numerous acoustic probability computations, and DRAM accesses are incurred. We propose to use a binary search tree for ASL storage and a max heap data structure to track the worst cost state and efficiently replace it when a better state is found. With this approach, the ASL size can be reduced from over 32K to 512 with minimal impact on recognition accuracy for a 7,000-word vocabulary model. This, combined with a caching technique for acoustic scores, reduced the DRAM data accessed by 31 \( \times \) and the acoustic probability computations by 26 \( \times \) . The approach has also been implemented in hardware on a Xilinx Zynq FPGA at 200 MHz using the Vivado SDS compiler. We study the tradeoffs among the amount of OCM used, word error rate, and decoding speed to show the effectiveness of the approach. The resulting implementation is capable of running faster than real time with 91% lesser block-RAMs.

Entropy conservation for comparison-based algorithms

Comparison-based algorithms are algorithms for which the execution of each operation is solely based on the outcome of a series of comparisons between elements [5]. Examples include most sorting algorithms, search algorithms, and more general algorithms such as Heapify which constructs a heap data structure from an input list [5]. Comparison-based computations can be naturally represented via the following computational model[12]: (a) model data structures as partially-ordered finite sets; (b) model data on these by topological sorts (aka linear extensions); (c) considering computation states as finite multisets of such data; (d) represent computations by their induced transformations on states.In this view, an abstract specification of a sorting algorithm has input state given by any possible permutation of a finite set of elements (represented, according to (a) and (b), by a discrete partially-ordered set together with its topological sorts given by all permutations) and output state a sorted list of elements (represented, again according to (a) and (b), by a linearly-ordered finite set with its unique topological sort).Entropy is a measure of “randomness” or “disorder.” Based on the computational model we introduce an entropy conservation result for comparison-based sorting algorithms: “quantitative order gained is proportional to positional order lost.” Intuitively, the result bears some relation to the messy office argument advocating a chaotic office where nothing is in the right place yet each item's place is known to the owner, over the case where each item is stored in the right order and yet the owner can no longer locate the items. Formally, we generalise the result to the class of data structures representable via series-parallel partial orders–a well-known computationally tractable class [6,7]. The resulting “denotational” version of entropy conservation will be extended in follow-up work to an “operational” version for a core part of our computational model.

Postmortem accurate IR-level state recovery for deployed concurrent programs

Debugging failures of deployed concurrent software is important for quality assurance. However, such failures are difficult to debug because their behavior is non-deterministic and limited information can be obtained with conventional means. Reverse debuggers such as REPT [11] assists with debugging by recovering data values before the failure. This is achieved by using a hardware-tracer to log control-flow information, then using the information and a conventional coredump to recover data values via reverse-execution at machine-level. REPT's algorithm for data value recovery is reliable and fast. But the implementation cost is high because of its dependence on architecture. Applying REPT to more abstract IR (Intermediate Representation) level instructions to counter this yielded limited results with low accuracy compared to the original x86_64 implementation. The main reason for this is that the stack layout is abstracted at IR-level. In this paper, we present STRAB (State Recovery at Abstract-level), a collection of our proposed methods to solve these problems. STRAB works in two phases. First, the data values in the coredump are lifted from machine-level to IR-level using rich debug information (DWARF3) and a novel technique we call mid-recovery lifting, the latter helping to recover more heap data values at IR-level. Second, our novel hybrid memory location resolution reduces the accuracy loss due to the abstracted stack layout at IR-level. Experimental results on a variety of real-world concurrent programs show that STRAB has significantly higher accuracy compared to REPT at IR-level (+40% on average) with only minor slowdowns (x2.7 on average), while also achieving architecture-independence.

Gradual verification of recursive heap data structures

Current static verification techniques do not provide good support for incrementality, making it difficult for developers to focus on specifying and verifying the properties and components that are most important. Dynamic verification approaches support incrementality, but cannot provide static guarantees. To bridge this gap, prior work proposed gradual verification, which supports incrementality by allowing every assertion to be complete, partial, or omitted, and provides sound verification that smoothly scales from dynamic to static checking. The prior approach to gradual verification, however, was limited to programs without recursive data structures. This paper extends gradual verification to programs that manipulate recursive, mutable data structures on the heap. We address several technical challenges, such as semantically connecting iso- and equi-recursive interpretations of abstract predicates, and supporting gradual verification of heap ownership. This work thus lays the foundation for future tools that work on realistic programs and support verification within an engineering process in which cost-benefit trade-offs can be made.

Igloo: soundly linking compositional refinement and separation logic for distributed system verification

Lighthouse projects like CompCert, seL4, IronFleet, and DeepSpec have demonstrated that full system verification is feasible by establishing a refinement between an abstract system specification and an executable implementation. Existing approaches however impose severe restrictions on the abstract system specifications due to their limited expressiveness or versatility, or on the executable code due to their use of suboptimal code extraction or inexpressive program logics. We propose a novel methodology that combines the compositional refinement of event-based models of distributed systems with the verification of full-fledged program code using expressive separation logics, which support features of realistic programming languages like heap data structures and concurrency. Our main technical contribution is a formal framework that soundly relates event-based system models to program specifications in separation logics. This enables protocol development tools to soundly interoperate with program verifiers to establish a refinement between the model and the code. We formalized our framework, Igloo, in Isabelle/HOL. We report on three case studies, a leader election protocol, a replication protocol, and a security protocol, for which we refine formal requirements into program specifications that we implement in Java and Python and prove correct using the VeriFast and Nagini tools.

FlowCFL: generalized type-based reachability analysis: graph reduction and equivalence of CFL-based and type-based reachability

Reachability analysis is a fundamental program analysis with a wide variety of applications. We present FlowCFL, a type-based reachability analysis that accounts for mutable heap data. The underlying semantics of FlowCFL is Context-Free-Language (CFL)-reachability. We make three contributions. First, we define a dynamic semantics that captures the notion of flow commonly used in reachability analysis. Second, we establish correctness of CFL-reachability over graphs with inverse edges (inverse edges are necessary for the handling of mutable heap data). Our approach combines CFL-reachability with reference immutability to avoid the addition of certain inverse edges, which results in graph reduction and precision improvement. The key contribution of our work is the formal account of correctness, which extends to the case when inverse edges are removed. Third, we present a type-based reachability analysis and establish equivalence between a certain CFL-reachability analysis and the type-based analysis, thus proving correctness of the type-based analysis.

Heap-based optimizer inspired by corporate rank hierarchy for global optimization

In an organization, a group of people working for a common goal may not achieve their goal unless they organize themselves in a hierarchy called Corporate Rank Hierarchy (CRH). This principle motivates us to map the concept of CRH to propose a new algorithm for optimization that logically arranges the search agents in a hierarchy based on their fitness. The proposed algorithm is named as heap-based optimizer (HBO) because it utilizes the heap data structure to map the concept of CRH. The mathematical model of HBO is built on three pillars: the interaction between the subordinates and their immediate boss, the interaction between the colleagues, and self-contribution of the employees. The proposed algorithm is benchmarked with 97 diverse test functions including 29 CEC-BC-2017 functions with very challenging landscapes against 7 highly-cited optimization algorithms including the winner of CEC-BC-2017 (EBO-CMAR). In the first two experiments, the exploitative and explorative behavior of HBO is evaluated by using 24 unimodal and 44 multimodal functions, respectively. It is shown through experiments and Friedman mean rank test that HBO outperforms and secures 1st rank. In the third experiment, we use 29 CEC-BC-2017 benchmark functions. According to Friedman mean rank test HBO attains 2nd position after EBO-CMAR; however, the difference in ranks of HBO and EBO-CMAR is shown to be statistically insignificant by using Bonferroni method based multiple comparison test. Moreover, it is shown through the Friedman test that the overall rank of HBO is 1st for all 97 benchmarks. In the fourth and the last experiment, the applicability on real-world problems is demonstrated by solving 3 constrained mechanical engineering optimization problems. The performance is shown to be superior or equivalent to the other algorithms, which have been used in the literature. The source code of HBO is publicly available athttps://github.com/qamar-askari/HBO.

A Fibonacci Heap Based Ensemble Model for Stock Price Prediction

The prediction of stock price with high accuracy and consistency is a major challenge. A lot of research has been done to predict the stock price. However, there is a scope of improvement in accuracy and consistency of the prediction. This article proposes a Fibonacci heap based ensemble model that predicts the closing price of a stock for the next trading day. The proposed model is a combination of different (time series and regression) models. The model computes the median of significant prediction models that are organized in a Fibonacci heap data structure. The performance of this model is tested with the datasets of well-known eleven stocks. The results prove the effectiveness of the proposed model over other traditional prediction models with respect to consistency and accuracy

A Selectable Sloppy Heap

We study the selection problem, namely that of computing the ith order statistic of n given elements. Here we offer a data structure called selectable sloppy heap that handles a dynamic version in which upon request (i) a new element is inserted or (ii) an element of a prescribed quantile group is deleted from the data structure. Each operation is executed in constant time—and is thus independent of n (the number of elements stored in the data structure)—provided that the number of quantile groups is fixed. This is the first result of this kind accommodating both insertion and deletion in constant time. As such, our data structure outperforms the soft heap data structure of Chazelle (which only offers constant amortized complexity for a fixed error rate 0 < ε ≤ 1 / 2 ) in applications such as dynamic percentile maintenance. The design demonstrates how slowing down a certain computation can speed up the data structure. The method described here is likely to have further impact in the field of data structure design in extending asymptotic amortized upper bounds to same formula asymptotic worst-case bounds.

Data Mining for Clustering Revenue Plan Expense Area (APBD) by using K-Means Algorithm

APBD is a systematic detailed list of receipts, expenditures and local spending within a certain period ( 1 year ) arranged in Permendagri No. 16 of 2006, so that the data APBD can be used as guidelines for governments and local expenditures in carrying out activities to raise revenue to maintain economic stability and to avoid inflation and deflation. Government financial institutions in areas such as DPKA kota Bima, experienced difficulties in identifying the relevance of each archive data on a APBD that so much, that results in a data warehouse, in addition to the administration, APBD in the government of Kota Bima have not been effective. To minimize the difficulty in identifying heap data archive APBD, then the data warehouse can be used to produce a knowledge that by using the techniques of Data Mining ( DM ), the method used is clustering and forecasting, clusterisasi performed using the K-Means Algorithm while for forecasting with multiple linear regression. With this method intended to classify and identify the data in the budget that have certain characteristics in common, and can predict the value of APBD in the future.

Improved Algorithm for Sorting via Heap Data Structure

Improved Algorithm for Sorting via Heap Data Structure

Qualitative and quantitative assessment of 3D-scanned bulk solid heap data

According to angle of repose theory, non-cohesive bulk solid heaps will form a perfect conical heap if carefully poured. However, there are indications that global shape deviations from an ideal cone exist in experimentally generated heaps. For example, rounded heap tips and concave or convex heap shapes have been reported in the literature, but no studies exist concerning local shape deviations from spatial data. The main aim of this study was to compare the 3D-scanned spatial data from experimentally generated heaps of eight bulk materials with their respective counterparts according to angle of repose/continuum theory and to examine the differences both qualitatively and quantitatively. The results showed that mapping the differences between 3D scans and ideal cones onto a two-dimensional developed view of the ideal cone's lateral surface is a meaningful method for visualizing the heaps' real shapes and for illustrating global as well as local shape deviations. Furthermore, the averaged shapes and respective variances on the experimental heaps' circumferences were examined. Global shape deviations could be identified for lignite, milkpowder, limestone and coke. An important conclusion was that simple consideration of the averaged surface lines does not enable identification of rotationally asymmetric heaps; however, it can be useful for indicating global shape deviations. Large surface line variance implies considerable rotational asymmetry, however the converse conclusion is false.

Bipartite binomial heaps

We describe a heap data structure that supports Minimum, Insert, and Borrow at O (1) worst-case cost, Delete at O (lg n ) worst-case cost including at most lg n + O (1) element comparisons, and Union at O (lg n ) worst-case cost including at most lg n + O (lglg n ) element comparisons, where n denotes the (total) number of elements stored in the data structure(s) prior to the operation. As the resulting data structure consists of two components that are different variants of binomial heaps, we call it a bipartite binomial heap. Compared to its counterpart, a multipartite binomial heap, the new structure is simpler and mergeable, still retaining the efficiency of the other operations.

Scalable Bandwidth Shaping Scheme via Adaptively Managed Parallel Heaps in Manycore-Based Network Processors

Scalability of network processor-based routers heavily depends on limitations imposed by memory accesses and associated power consumption. Bandwidth shaping of a flow is a key function, which requires a token bucket per output queue and abuses memory bandwidth. As the number of output queues increases, managing token buckets becomes prohibitively expensive and limits scalability. In this work, we propose a scalable software-based token bucket management scheme that can reduce memory accesses and power consumption significantly. To satisfy real-time and low-cost constraints, we propose novel parallel heap data structures running on a manycore-based network processor. By using cache locking, the performance of heap processing is enhanced significantly and is more predictable. In addition, we quantitatively analyze the performance and memory footprint of the proposed software scheme using stochastic modeling and the Lyapunov central limit theorem. Finally, the proposed scheme provides an adaptive method to limit the size of heaps in the case of oversubscribed queues, which can successfully isolate the queues showing unideal behavior. The proposed scheme reduces memory accesses by up to three orders of magnitude for one million queues sharing a 100Gbps interface of the router while maintaining stability under stressful scenarios.

Minimizing Context Migration in Mobile Code Offload

Mobile Cloud Computing (MCC) is of particular importance to address the conflict between the increasing complexity of user applications and the limited lifespan of mobile device's battery, by offloading the computational workloads from local devices to the remote cloud. Current offloading schemes either require the programmer's annotations, which restricts its wide application; or transmits too much unnecessary data, resulting bandwidth, and energy waste. In this paper, we propose a novel method-level offloading methodology to offload local computational workload with as least data transmission as possible. Our basic idea is to identify the contexts which are necessary to the method execution by parsing application binaries in advance and applying this parsing result to selectively migrate heap data while allowing successful method execution remotely. To further improve the efficiency of such offline parsing of application binaries, our scheme also conducts one-time parsing to all the mobile OS libraries and reuses these parsing results for different user applications. We have implemented our design over the Dalvik Virtual Machine of Android OS. Our experiments and evaluation against applications downloaded from Google Play show that our approach can save data transmission significantly comparing to existing schemes.

Heap Abstractions for Static Analysis

Heap data is potentially unbounded and seemingly arbitrary. Hence, unlike stack and static data, heap data cannot be abstracted in terms of a fixed set of program variables. This makes it an interesting topic of study and there is an abundance of literature employing heap abstractions. Although most studies have addressed similar concerns, insights gained in one description of heap abstraction may not directly carry over to some other description. In our search of a unified theme, we view heap abstraction as consisting of two steps: (a) heap modelling , which is the process of representing a heap memory (i.e., an unbounded set of concrete locations) as a heap model (i.e., an unbounded set of abstract locations), and (b) summarization , which is the process of bounding the heap model by merging multiple abstract locations into summary locations. We classify the heap models as storeless, store based, and hybrid. We describe various summarization techniques based on k -limiting, allocation sites, patterns, variables, other generic instrumentation predicates, and higher-order logics. This approach allows us to compare the insights of a large number of seemingly dissimilar heap abstractions and also paves the way for creating new abstractions by mix and match of models and summarization techniques.