Cost-based Optimization Research Articles

Combining possibilistic linear programming and fuzzy AHP for solving the multi-objective capacitated multi-facility location problem

The capacitated multi-facility location problem is a complex and imprecise decision-making problem which contains both quantitative and qualitative factors. In the literature, many objectives for optimizing many types of logistics networks are described: (i) minimization objectives such as cost, inventory, transportation time, environmental impact, financial risk and (ii) maximization objectives such as profit, customer satisfaction, and flexibility and robustness. However, only a few papers have considered quantitative and qualitative factors together with imprecise methodologies. Unlike traditional cost-based optimization techniques, the approach proposed here evaluates these factors together while considering various viewpoints. Decision-makers must deal both factors together to model complex structure of real-world applications. In this paper, a two-phase possibilistic linear programming approach and a fuzzy analytical hierarchical process approach have been combined to optimize two objective functions (“minimum cost” and “maximum qualitative factors benefit”) in a four-stage (suppliers, plants, distribution centers, customers) supply chain network in the presence of vagueness. The results and findings of this method are illustrated with a numerical example, and the advantages of this methodology are discussed in the conclusion.

Why it is time for a HyPE

GPU acceleration is a promising approach to speed up query processing of database systems by using low cost graphic processors as coprocessors. Two major trends have emerged in this area: (1) The development of frameworks for scheduling tasks in heterogeneous CPU/GPU platforms, which is mainly in the context of coprocessing for applications and does not consider specifics of database-query processing and optimization. (2) The acceleration of database operations using efficient GPU algorithms, which typically cannot be applied easily on other database systems, because of their analytical-algorithm-specific cost models. One major challenge is how to combine traditional database query processing with GPU coprocessing techniques and efficient database operation scheduling in a GPU-aware query optimizer. In this thesis, we develop a hybrid query processing engine, which extends the traditional physical optimization process to generate hybrid query plans and to perform a cost-based optimization in a way that the advantages of CPUs and GPUs are combined. Furthermore, we aim at a portable solution between different GPU-accelerated database management systems to maximize applicability. Preliminary results indicate great potential.

Mixed-width aisle configurations for order picking in distribution centers

Space required for the order picking area and labor required to perform the picking activity are two significant costs for a distribution center (DC). Traditionally, DCs employ either entirely wide or entirely narrow aisles in their picking systems. Wide aisles allow pickers to pass each other, which reduces blocking, and requires fewer pickers than their narrow-aisle counterpart for the same throughput. However, the amount of space required for wide-aisle configurations is high. Narrow aisles utilize less space than wide aisles, but are less efficient because of the increased likelihood of congestion experienced by pickers. We propose a variation to the traditional orthogonal aisle designs where both wide and narrow aisles are mixed within the configuration, with a view that mixed-width aisles may provide a compromise between space and labor. To analyze these new mixed-width aisle configurations, we develop analytical models for space and travel time considering randomized storage and traversal routing policies. Through a cost-based optimization model, we identify system parameters for which mixed-width aisle configurations are optimal. Experimental results indicate that annual cost savings of up to $48,000 can be realized over systems with pure wide or narrow aisle configurations.

Optimal Integration of Distributed Energy Storage Devices in Smart Grids

Energy storage is traditionally well established in the form of large scale pumped-hydro systems, but nowadays is finding increased attraction in medium and smaller scale systems. Such expansion is entirely complementary to the forecasted wider integration of intermittent renewable resources in future electrical distribution systems (Smart Grids). This paper is intended to offer a useful tool for analyzing potential advantages of distributed energy storages in Smart Grids with reference to both different possible conceivable regulatory schemes and services to be provided. The Smart Grid Operator is assumed to have the ownership and operation of the energy storage systems, and a new cost-based optimization strategy for their optimal placement, sizing and control is proposed. The need to quantify benefits of both the Smart Grid where the energy storage devices are included and the external interconnected grid is explored. Numerical applications to a Medium Voltage test Smart Grid show the advantages of using storage systems related to different options in terms of incentives and services to be provided.

Cost-Based Optimization of Service Compositions

For providers of composite services, preventing cases of SLA violations is crucial. Previous work has established runtime adaptation of compositions as a promising tool to achieve SLA conformance. However, to get a realistic and complete view of the decision process of service providers, the costs of adaptation need to be taken into account. In this paper, we formalize the problem of finding the optimal set of adaptations, which minimizes the total costs arising from SLA violations and the adaptations to prevent them. We present possible algorithms to solve this complex optimization problem, and detail an end-to-end system based on our earlier work on the PREvent (prediction and prevention based on event monitoring) framework, which clearly indicates the usefulness of our model. We discuss experimental results that show how the application of our approach leads to reduced costs for the service provider, and explain the circumstances in which different algorithms lead to more or less satisfactory results.

Integrated Emission Management for Cost Optimal EGR-SCR Balancing in Diesels

The potential of a cost-based optimization method is experimentally demonstrated on a Euro-VI heavy-duty diesel engine. Based on the actual engine-aftertreatment state, this model-based Integrated Emission Management (IEM) strategy minimizes operational (fuel and AdBlue) costs within emission constraints by real-time computation of optimal air management set points. This work discusses the control design in detail. By EGR-SCR balancing, fuel consumption and operational costs are reduced by 2.1% and 1.5%, respectively, compared to the baseline strategy over the hot World Harmonized Transient Cycle. Due to its adaptation characteristics, the IEM strategy is robust for varying conditions during real-world operation.

Distributed time-aware provenance

The ability to reason about changes in a distributed system's state enables network administrators to better diagnose protocol misconfigurations, detect intrusions, and pinpoint performance bottlenecks. We propose a novel provenance model called Distributed Time-aware Provenance (DTaP) that aids forensics and debugging in distributed systems by explicitly representing time, distributed state, and state changes. Using a distributed Datalog abstraction for modeling distributed protocols, we prove that the DTaP model provides a sound and complete representation that correctly captures dependencies among events in a distributed system. We additionally introduce DistTape, an implementation of the DTaP model that uses novel distributed storage structures, query processing, and cost-based optimization techniques to efficiently query time-aware provenance in a distributed setting. Using two example systems (declarative network routing and Hadoop MapReduce), we demonstrate that DistTape can efficiently maintain and query time-aware provenance at low communication and computation cost.

Sensitivity Analysis of Server Virtualized System Availability

Server virtualization is a technology used in many enterprise systems to reduce operation and acquisition costs, and increase the availability of their critical services. Virtualized systems may be even more complex than traditional nonvirtualized systems; thus, the quantitative assessment of system availability is even more difficult. In this paper, we propose a sensitivity analysis approach to find the parameters that deserve more attention for improving the availability of systems. Our analysis is based on Markov reward models, and suggests that host failure rate is the most important parameter when the measure of interest is the system mean time to failure. For capacity oriented availability, the failure rate of applications was found to be another major concern. The results of both analyses were cross-validated by varying each parameter in isolation, and checking the corresponding change in the measure of interest. A cost-based optimization method helps to highlight the parameter that should have higher priority in system enhancement.

Reassessing Top-Down Join Enumeration

Finding an optimal execution order of join operations is a crucial task in every cost-based query optimizer. Since there are many possible join trees for a given query, the overhead of the join (tree) enumeration algorithm per valid join tree should be minimal. In the case of a clique-shaped query graph, the best known top-down algorithm has a complexity of \Theta (n^2) per join tree, where n is the number of relations. In this paper, we present an algorithm that has an according O(1) complexity in this case. We show experimentally that this more theoretical result has indeed a high impact on the performance in other nonclique settings. This is especially true for cyclic query graphs. Further, we evaluate the performance of our new algorithm and compare it with the best top-down and bottom-up algorithms described in the literature.

Optimization of the combustion of SOME (soybean oil methyl ester), B5, B10, B20 and petrodiesel in a semi industrial boiler

This paper characterizes combustion of petrodiesel and prevalent SOME blends (B100, B5, B10 and B20), which were produced through an alkali-based transesterification in PUT, in a semi industrial boiler by determining its efficiency, exhaust emissions and costs. First, the influence of oxygen content of stock gases (by volume percent) upon regular emissions, thermal efficiency and costs (calculated by a cost function) was studied in order to investigate the performance and feasibility of using biodiesel in the boiler. In the next level, a multi-objective optimization method, named cost-based optimization, was represented and optimized the combustion of the fuels.The overall efficiency of the boiler obtained with the different fuels is comparable for the same operating points. Soy bean methyl ester and the blends emitted lower emissions than petrodiesel. Results demonstrate that the use of pure biodiesel is not economically feasible in the boiler. The total costs of the boiler obtained with the blends and petrodiesel are quite competitive. B5 is introduced to be the best fuel for using in the boiler. Results also reveal an optimum (“compromise”) in total cost of all the fuels in relation to the oxygen concentration of stock gases, which promises a cost saving in the boiler performance.

Stubby

There is a growing trend of performing analysis on large datasets using workflows composed of MapReduce jobs connected through producer-consumer relationships based on data. This trend has spurred the development of a number of interfaces---ranging from program-based to query-based interfaces---for generating MapReduce workflows. Studies have shown that the gap in performance can be quite large between optimized and unoptimized workflows. However, automatic cost-based optimization of MapReduce workflows remains a challenge due to the multitude of interfaces, large size of the execution plan space, and the frequent unavailability of all types of information needed for optimization. We introduce a comprehensive plan space for MapReduce workflows generated by popular workflow generators. We then propose Stubby , a cost-based optimizer that searches selectively through the subspace of the full plan space that can be enumerated correctly and costed based on the information available in any given setting. Stubby enumerates the plan space based on plan-to-plan transformations and an efficient search algorithm. Stubby is designed to be extensible to new interfaces and new types of optimizations, which is a desirable feature given how rapidly MapReduce systems are evolving. Stubby's efficiency and effectiveness have been evaluated using representative workflows from many domains.

Optimisation of the design of infrastructure components subject to progressive deterioration and extreme loads

The construction of new infrastructure requires solutions that maximise the benefit while meeting acceptable safety levels. The solution to this problem can be obtained through a structural cost-based optimisation. This paper integrates a life-cycle cost optimisation model developed by Rackwitz (2000) with a structural deterioration model that combines the action of progressive degradation (e.g. corrosion, fatigue) and sudden events (e.g. earthquakes). The structural condition at a given time is measured in terms of the system's remaining life, which can be measured in practice through any structural performance indicator (e.g. ductility). It is assumed that sudden events conform to a compound point process with shock sizes and interarrival times that are independent and identically distributed random variables. Deterministic and random progressive deterioration models are also included in the formulation. Randomness in progressive deterioration is modelled as a point process with known arrival times and random shock sizes. Structural reliability is evaluated against prescribed design and operation thresholds that can be used to establish limit states or intervention policies. The entire model is illustrated with an example that compares several life-cycle models showing the importance of taking into account the damage history in evaluating the performance of infrastructure systems.

DBToaster

Applications ranging from algorithmic trading to scientific data analysis require realtime analytics based on views over databases that change at very high rates. Such views have to be kept fresh at low maintenance cost and latencies. At the same time, these views have to support classical SQL, rather than window semantics, to enable applications that combine current with aged or historical data. In this paper, we present viewlet transforms , a recursive finite differencing technique applied to queries. The viewlet transform materializes a query and a set of its higher-order deltas as views. These views support each other's incremental maintenance, leading to a reduced overall view maintenance cost. The viewlet transform of a query admits efficient evaluation, the elimination of certain expensive query operations, and aggressive parallelization. We develop viewlet transforms into a workable query execution technique, present a heuristic and cost-based optimization framework, and report on experiments with a prototype dynamic data management system that combines viewlet transforms with an optimizing compilation technique. The system supports tens of thousands of complete view refreshes a second for a wide range of queries.

Optimizing XML twig queries with full-text predicates

Efficient query processing has been a critical issue for XML repositories. In this paper, we consider the XML query which can be represented as a query tree with twig patterns, and also consists of full-text constraints. Previously, the structure-first approach and the keyword-first approach have been proposed to process such kind of queries. The main focus of this paper is constructing an integrated system to support these two approaches and find the best execution plan. To achieve this goal, we first analyze the components of these two approaches and design a set of operators. We then derive the corresponding cost model and rewriting rules to perform costbased optimization. We also propose several heuristic rules by observing the behaviors of the two approaches. Via an extensive experimental study, we demonstrate that our cost-based system and heuristic system are both effective.

Performance Life Cost-Based Maintenance Strategy Optimization for Reinforced Concrete Girder Bridges

This paper presents a maintenance strategy optimization method for RC girder bridges by considering the performance indicators, service life and life cycle maintenance cost, as criteria. The condition and reliability indices are defined as performance indicators. The deteriorated processes of performance indicators with and without maintenance actions are described as multilinear models. The life cycle maintenance planning optimization of deteriorating bridges is formulated as a multiobjective problem to be solved by an improved nondominated sorting genetic algorithm with controlled elitism. The condition index, reliability index, service life, and life cycle maintenance cost of the bridge are considered as four separate objective functions. A simply supported RC girder bridge is analyzed as an application example to demonstrate the usefulness of the proposed procedure.

Efficient Substructure Searching of Large Chemical Libraries: The ABCD Chemical Cartridge

Efficient substructure searching is a key requirement for any chemical information management system. In this paper, we describe the substructure search capabilities of ABCD, an integrated drug discovery informatics platform developed at Johnson & Johnson Pharmaceutical Research & Development, L.L.C. The solution consists of several algorithmic components: 1) a pattern mapping algorithm for solving the subgraph isomorphism problem, 2) an indexing scheme that enables very fast substructure searches on large structure files, 3) the incorporation of that indexing scheme into an Oracle cartridge to enable querying large relational databases through SQL, and 4) a cost estimation scheme that allows the Oracle cost-based optimizer to generate a good execution plan when a substructure search is combined with additional constraints in a single SQL query. The algorithm was tested on a public database comprising nearly 1 million molecules using 4,629 substructure queries, the vast majority of which were submitted by discovery scientists over the last 2.5 years of user acceptance testing of ABCD. 80.7% of these queries were completed in less than a second and 96.8% in less than ten seconds on a single CPU, while on eight processing cores these numbers increased to 93.2% and 99.7%, respectively. The slower queries involved extremely generic patterns that returned the entire database as screening hits and required extensive atom-by-atom verification.

MapReduce programming and cost-based optimization?

MapReduce has emerged as a viable competitor to database systems in big data analytics. MapReduce programs are being written for a wide variety of application domains including business data processing, text analysis, natural language processing, Web graph and social network analysis, and computational science. However, MapReduce systems lack a feature that has been key to the historical success of database systems, namely, cost-based optimization. A major challenge here is that, to the MapReduce system, a program consists of black-box map and reduce functions written in some programming language like C++, Java, Python, or Ruby. Starfish is a self-tuning system for big data analytics that includes, to our knowledge, the first Cost-based Optimizer for simple to arbitrarily complex MapReduce programs. Starfish also includes a Profiler to collect detailed statistical information from unmodified MapReduce programs, and a What-if Engine for fine-grained cost estimation. This demonstration will present the profiling, what-if analysis, and cost-based optimization of MapReduce programs in Starfish. We will show how (nonexpert) users can employ the Starfish Visualizer to (a) get a deep understanding of a MapReduce program's behavior during execution, (b) ask hypothetical questions on how the program's behavior will change when parameter settings, cluster resources, or input data properties change, and (c) ultimately optimize the program.

Profiling, what-if analysis, and cost-based optimization of MapReduce programs

MapReduce has emerged as a viable competitor to database systems in big data analytics. MapReduce programs are being written for a wide variety of application domains including business data processing, text analysis, natural language processing, Web graph and social network analysis, and computational science. However, MapReduce systems lack a feature that has been key to the historical success of database systems, namely, cost-based optimization. A major challenge here is that, to the MapReduce system, a program consists of black-box map and reduce functions written in some programming language like C++, Java, Python, or Ruby. We introduce, to our knowledge, the first Cost-based Optimizer for simple to arbitrarily complex MapReduce programs. We focus on the optimization opportunities presented by the large space of configuration parameters for these programs. We also introduce a Profiler to collect detailed statistical information from unmodified MapReduce programs, and a What-if Engine for fine-grained cost estimation. All components have been prototyped for the popular Hadoop MapReduce system. The effectiveness of each component is demonstrated through a comprehensive evaluation using representative MapReduce programs from various application domains.

Which sort orders are interesting?

Sort orders play an important role in query evaluation. Algorithms that rely on sorting are widely used to implement joins, grouping, duplicate elimination and other set operations. The notion of interesting orders has allowed query optimizers to consider plans that could be locally sub-optimal, but produce ordered output beneficial for other operators, and thus be part of a globally optimal plan. However, the number of interesting orders for most operators is factorial in the number of attributes involved. Optimizer implementations use heuristics to prune the number of interesting orders, but the quality of the heuristics is unclear. Increasingly complex decision support queries and increasing use of query-covering indices, which provide multiple alternative sort orders for relations, motivate us to better address the problem of choosing interesting orders. We show that even a simplified version of the problem is NP-hard and provide a 1/2-benefit approximation algorithm for a special case of the problem. We then present principled heuristics for the general case of choosing interesting orders. We have implemented the proposed techniques in a Volcano-style cost-based optimizer, and our performance study shows significant improvements in estimated cost. We also executed our plans on a widely used commercial database system, and on PostgreSQL, and found that actual execution times for our plans were significantly better than for plans generated by those systems in several cases.

Cost-Based Topology Optimization of Embedded Ethernet Networks

During past decades, Ethernet progressively became the most widely used Local Area Network (LAN) technology. Apart from LAN installations, Ethernet also became attractive for other application areas such as industrial control, automotive, and avionics. In traditional LAN design, the objective is to minimize the network deployment cost. However, in embedded networks, additional constraints and ambient conditions add to the complexity of the problem. In this paper, the authors propose a Simulated Annealing (SA) algorithm to optimize the physical topology of an embedded Ethernet network. The various constraints and ambient conditions are modeled by a cost map. For networks with small number of nodes and/or switches, the authors were able to find the optimal solutions using adapted algorithms. These solutions will serve as a lower bound for the solutions obtained via the SA algorithm. However, the adapted algorithms are time consuming and application specific. The paper shows that the SA algorithm can be applied in all cases and finds (near-) optimal solutions.