Storage Allocation Research Articles

Closure to “Drought Storage Allocation Rules for Surface Reservoir Systems” by Jay R. Lund

Closure to “Drought Storage Allocation Rules for Surface Reservoir Systems” by Jay R. Lund

Discussion of “Drought Storage Allocation Rules for Surface Reservoir Systems” by J. R. Lund

Discussion of “Drought Storage Allocation Rules for Surface Reservoir Systems” by J. R. Lund

Analyzing the feasibility of building a new mass storage system on distributed resources

The average PC now contains a large and increasing amount of storage with an ever greater amount left unused. We believe there is an opportunity for organizations to harness the vast unused storage capacity on their PCs to create a very large, low-cost, shared storage system. What is needed is the proper storage system architecture and software to exploit and manage the unused portions of existing PC storage devices across an organization and make it reliably accessible to users and applications. We call our vision of such a storage system Storage@desk (SD). This paper describes our first step towards the realization of SD—a study of machine and storage characteristics and usage in a model organization. We studied 729 PCs in an academic institution for 91 days, monitoring the configuration, load and usage of the major machine subsystems, i.e. disk, memory, CPU and network. To further analyze the availability characteristics of storage in an SD system, we performed a trace-driven simulation of some basic storage allocation strategies. This paper presents the results of our data collection efforts, our analysis of the data, our simulation results and our conclusion that an SD system is indeed feasible and holds promise as a cost-effective way to create massive storage systems. Copyright © 2007 John Wiley & Sons, Ltd.

A constant-time dynamic storage allocator for real-time systems

Dynamic memory allocation has been used for decades. However, it has seldom been used in real-time systems since the worst case of spatial and temporal requirements for allocation and deallocation operations is either unbounded or bounded but with a very large bound. In this paper, a new allocator called TLSF (Two Level Segregated Fit) is presented. TLSF is designed and implemented to accommodate real-time constraints. The proposed allocator exhibits time-bounded behaviour, O(1), and maintains a very good execution time. This paper describes in detail the data structures and functions provided by TLSF. We also compare TLSF with a representative set of allocators regarding their temporal cost and fragmentation. Although the paper is mainly focused on timing analysis, a brief study and comparative analysis of fragmentation incurred by the allocators has been also included in order to provide a global view of the behaviour of the allocators. The temporal and spatial results showed that TLSF is also a fast allocator and produces a fragmentation close to that caused by the best existing allocators.

Resource Allocation in Bounded Degree Trees

We study the bandwidth allocation problem (bap) in bounded degree trees. In this problem we are given a tree and a set of connection requests. Each request consists of a path in the tree, a bandwidth requirement, and a weight. Our goal is to find a maximum weight subset S of requests such that, for every edge e, the total bandwidth of requests in S whose path contains e is at most 1. We also consider the storage allocation problem (sap), in which it is also required that every request in the solution is given the same contiguous portion of the resource in every edge in its path. We present a deterministic approximation algorithm for bap in bounded degree trees with ratio . Our algorithm is based on a novel application of the local ratio technique in which the available bandwidth is divided into narrow strips and requests with very small bandwidths are allocated in these strips. We also present a randomized (2+e)-approximation algorithm for bap in bounded degree trees. The best previously known ratio for bap in general trees is 5. We present a reduction from sap to bap that works for instances where the tree is a line and the bandwidths are very small. It follows that there exists a deterministic 2.582-approximation algorithm and a randomized (2+e)-approximation algorithm for sap in the line. The best previously known ratio for this problem is 7.

Implementation of a constant‐time dynamic storage allocator

AbstractThis paper describes the design criteria and implementation details of a dynamic storage allocator for real‐time systems. The main requirements that have to be considered when designing a new allocator are concerned with temporal and spatial constraints. The proposed algorithm, called TLSF (two‐level segregated fit), has an asymptotic constant cost, O(1), maintaining a fast response time (less than 200 processor instructions on a x86 processor) and a low level of memory usage (low fragmentation). TLSF uses two levels of segregated lists to arrange free memory blocks and an incomplete search policy. This policy is implemented with word‐size bitmaps and logical processor instructions. Therefore, TLSF can be categorized as a good‐fit allocator. The incomplete search policy is shown also to be a good policy in terms of fragmentation. The fragmentation caused by TLSF is slightly smaller (better) than that caused by best fit (which is one of the best allocators regarding memory fragmentation). In order to evaluate the proposed allocator, three analyses are presented in this paper. The first one is based on worst‐case scenarios. The second one provides a detailed consideration of the execution cost of the internal operations of the allocator and its fragmentation. The third analysis is a comparison with other well‐known allocators from the temporal (number of cycles and processor instructions) and spatial (fragmentation) points of view. In order to compare them, a task model has been presented. Copyright © 2007 John Wiley & Sons, Ltd.

A step towards unifying schedule and storage optimization

We present a unified mathematical framework for analyzing the tradeoffs between parallelism and storage allocation within a parallelizing compiler. Using this framework, we show how to find a good storage mapping for a given schedule, a good schedule for a given storage mapping, and a good storage mapping that is valid for all legal (one-dimensional affine) schedules. We consider storage mappings that collapse one dimension of a multidimensional array, and programs that are in a single assignment form and accept a one-dimensional affine schedule. Our method combines affine scheduling techniques with occupancy vector analysis and incorporates general affine dependences across statements and loop nests. We formulate the constraints imposed by the data dependences and storage mappings as a set of linear inequalities, and apply numerical programming techniques to solve for the shortest occupancy vector. We consider our method to be a first step towards automating a procedure that finds the optimal tradeoff between parallelism and storage space.

Exploiting parallelism in configurable architectures through custom array mapping

Configurable architectures offer the unique opportunity of customising the storage allocation to meet specific applications' needs. A compiler approach to map the arrays of a loop-based computation to internal memories of a configurable architecture with the objective of minimising the overall execution time is described. An algorithm that considers the data access patterns of the arrays along the critical path of the computation as well as the available storage and memory bandwidth is presented. Experimental results are presented which demonstrate the application of this approach for a set of kernel codes when targeting a field-programmable gate-array. The results reveal that the proposed algorithm outperforms the naive and custom data layout techniques by an average of 33% and 15% in terms of execution time, while taking into account the available hardware resources.

A pointer logic and certifying compiler

Proof-Carrying Code brings two big challenges to the research field of programming languages. One is to seek more expressive logics or type systems to specify or reason about the properties of low-level or high-level programs. The other is to study the technology of certifying compilation in which the compiler generates proofs for programs with annotations. This paper presents our progress in the above two aspects. A pointer logic was designed for PointerC (a C-like programming language) in our research. As an extension of Hoare logic, our pointer logic expresses the change of pointer information for each statement in its inference rules to support program verification. Meanwhile, based on the ideas from CAP (Certified Assembly Programming) and SCAP (Stack-based Certified Assembly Programming), a reasoning framework was built to verify the properties of object code in a Hoare style. And a certifying compiler prototype for PointerC was implemented based on this framework. The main contribution of this paper is the design of the pointer logic and the implementation of the certifying compiler prototype. In our certifying compiler, the source language contains rich pointer types and operations and also supports dynamic storage allocation and deallocation.

A simple direct solution to a storage allocation model

In this note we consider a two-dimensional difference equation, that arises in models of storage allocation. We us a discrete version of the classic method of separation of variables, and obtain an explicit form of the solution, as either a contour integral or an infinite series.

Simultaneous Optimization of Storage Allocation and Routing Problems for Belt-conveyor Transportation

Raw materials for industrial steel-making plants need to be stored in material yards in order to satisfy various production requirements. The stored materials are supplied to production plants by belt-conveyors with multiple alternative transportation routes. The simultaneous optimization of the storage allocation and transportation routing for the raw materials yard is required for achieving production that is flexible to changes in demand. In this paper, we propose a Lagrangian decomposition and coordination approach for storage allocation and routing for a belt-conveyor transportation system to handle demand changes. The overall problem is decomposed into an allocation planning subproblem and a transportation routing subproblem. A near-optimal solution for allocation planning and transportation routing is derived by repeating each optimization of subproblems and multiplier updates. The effectiveness of the proposed approach is confirmed in numerical experiments.

Multiperiod optimization for the design and planning of multiproduct batch plants

This paper presents a general multiperiod optimization model, which simultaneously solves the design and planning decisions in multiproduct batch plants. Therefore, the trade-offs between both problems are taken into account as well as variations due to seasonal effects, demand patterns, etc. From the design point of view, the model is formulated considering batch and semicontinuous units, the allocation of intermediate storage, and structural decisions. Following the usual procurement policy, equipment is provided using discrete sizes. From the planning point of view, the formulation takes into account both products and raw materials inventories, product demands and raw materials supplies that vary seasonally in a multiperiod approach. The objective is the maximization of an economic function, which considers incomes, and both investment and operation costs. A plant that produces five oleoresins in seven stages is used to illustrate this approach.

An approach to virtual allocation in storage systems

This article presents virtual allocation , a scheme for flexible storage allocation. Virtual allocation separates storage allocation from the file system. It employs an allocate-on-write strategy which lets applications fit into the actual usage of storage space, without regard to the configured file system size. This improves flexibility by allowing storage space to be shared across different file systems. This article presents the design of virtual allocation and its evaluation through benchmarks. To illustrate our approach, we implemented a prototype system on PCs running Linux. We present the results from the prototype implementation and its evaluation.

Load-balance policy in two level-cluster file system

In this paper, we explored a load-balancing algorithm in a cluster file system contains two levels of metadata-server, primary-level server quickly distributestasks to second-level servers depending on the closest load-balancing information. At the same time, we explored a method which accurately reflect I/O traffic and storage of storage-node; computing the heat-value of file, according to which we realized a more logical storage allocation. According to the experiment result, we conclude that this new algorithm shortens the executing time of tasks and improves the system performance compared with other load algorithm.

Drought Storage Allocation Rules for Surface Reservoir Systems

This technical note develops a simple drought storage allocation rule to minimize evaporative and seepage water losses from a system of reservoirs. Such a rule might have value during a prolonged drought, when the value of lost water is likely to be particularly high. Typically, concentrating water storage in one or a few reservoirs reduces overall water losses compared to “balancing” storage among reservoirs. Paradoxically, concentrating storage during a drought tends to minimize reservoir surface area available for recreation, increasing recreation losses even as it minimizes water losses.

Efficient in-memory extensible inverted file

The growing amount of on-line data demands efficient parallel and distributed indexing mechanisms to manage large resource requirements and unpredictable system failures. Parallel and distributed indices built using commodity hardware like personal computers (PCs) can substantially save cost because PCs are produced in bulk, achieving the scale of economy. However, PCs have limited amount of random access memory (RAM) and the effective utilization of RAM for in-memory inversion is crucial. This paper presents an analytical investigation and an empirical evaluation of storage-efficient inmemory extensible inverted files, which are represented by fixed- or variable-sized linked list nodes. The size of these linked list nodes is determined by minimizing the storage wastes or maximizing storage utilization under different conditions, which lead to different storage allocation schemes. Minimizing storage wastes also reduces the number of address indirections (i.e., chaining). We evaluated our storage allocation schemes using a number of reference collections. We found that the arrival rate scheme is the best in terms of both storage utilization and the mean number of chainings per term. The final storage utilization can be over 90% in our evaluation if there is a sufficient number of documents indexed. The mean number of chainings is not large (less than 2.6 for all the reference collections). We have also showed that our best storage allocation scheme can be used for our extensible compressed inverted file. The final storage utilization of the extensible compressed inverted file can be over 90% in our evaluation provided that there is a sufficient number of documents indexed. The proposed storage allocation schemes can also be used by compressed extensible inverted files with word positions

Message analysis for concurrent programs using message passing

We describe an analysis-driven storage allocation scheme for concurrent systems that use message passing with copying semantics. The basic principle is that in such a system, data which is not part of any message does not need to be allocated in a shared data area. This allows for the deallocation of thread-specific data without requiring global synchronization and often without even triggering garbage collection. On the other hand, data that is part of a message should preferably be allocated on a shared area since this allows for fast ( O (1)) interprocess communication that does not require actual copying. In the context of a dynamically typed, higher-order concurrent functional language, we present a static message analysis which guides the allocation. As shown by our performance evaluation, conducted using a production-quality language implementation, the analysis is effective enough to discover most data which is to be used as a message, and to allow the allocation scheme to combine the best performance characteristics of both a process-centric and a communal memory architecture.

An optimization model for storage yard management in transshipment hubs

This paper studies a yard storage allocation problem in a transshipment hub where there is a great number of loading and unloading activities. The primary challenge is to efficiently shift containers between the vessels and the storage area so that reshuffling and traffic congestion is minimized. In particular, to reduce reshuffling, a consignment strategy is used. This strategy groups unloaded containers according to their destination vessel. To reduce traffic congestion, a new workload balancing protocol is proposed. A mixed integer-programming model is then formulated to determine the minimum number of yard cranes to deploy and the location where unloaded containers should be stored. The model is solved using CPLEX. Due to the size and complexity of this model two heuristics are also developed. The first is a sequential method while the second is a column generation method. A bound is developed that allows the quality of the solution to be judged. Lastly, a numerical investigation is provided and demonstrates that the algorithms perform adequately on most cases considered.

A new node organization mechanism of a campus Grid and performance analysis

AbstractThis paper proposes a new node organization mechanism in the storage service provided by the Campus Grid Service Environment. The advantage of this architecture is its ability to aggregate nodes in the order of millions; the corresponding storage space aggregated can attain a petabyte level, which is a significant improvement in comparison with other campus Grids. Furthermore, this architecture is scalable and it can allow more nodes to join the storage pool to increase the storage space. The multilevel linked lists were first developed in order to organize the storage resources that are contributed by the nodes in the campus network. Subsequently, the algorithms were presented to analyze the mechanisms of the nodes: joining and leaving; storage resource scheduling and allocation; and unallocated storage space management. The performance of this system was investigated by employing a stochastic fluid model as the storage space changes with time as a result of the variation of the number of nodes. With this analytical model, the function between the storage allocation probability and the number of nodes was obtained. If more nodes join the campus Grid, the aggregated storage space will provide more available storage resources. If the storage resources allocation rate is less than the storage resources release rate, the available storage resources will increase and vice versa. Copyright © 2006 John Wiley & Sons, Ltd.

Transgenic modification of gai or rgl1 causes dwarfing and alters gibberellins, root growth, and metabolite profiles in Populus

In Arabidopsis and other plants, gibberellin (GA)-regulated responses are mediated by proteins including GAI, RGA and RGL1-3 that contain a functional DELLA domain. Through transgenic modification, we found that DELLA-less versions of GAI (gai) and RGL1 (rgl1) in a Populus tree have profound, dominant effects on phenotype, producing pleiotropic changes in morphology and metabolic profiles. Shoots were dwarfed, likely via constitutive repression of GA-induced elongation, whereas root growth was promoted two- to threefold in vitro. Applied GA(3 )inhibited adventitious root production in wild-type poplar, but gai/rgl1 poplars were unaffected by the inhibition. The concentrations of bioactive GA(1) and GA(4) in leaves of gai- and rgl1-expressing plants increased 12- to 64-fold, while the C(19) precursors of GA(1) (GA(53), GA(44) and GA(19)) decreased three- to ninefold, consistent with feedback regulation of GA 20-oxidase in the transgenic plants. The transgenic modifications elicited significant metabolic changes. In roots, metabolic profiling suggested increased respiration as a possible mechanism of the increased root growth. In leaves, we found metabolite changes suggesting reduced carbon flux through the lignin biosynthetic pathway and a shift towards allocation of secondary storage and defense metabolites, including various phenols, phenolic glucosides, and phenolic acid conjugates.