Data Placement Research Articles

HERMES: data placement and schema optimization for enterprise knowledge bases

HERMES: data placement and schema optimization for enterprise knowledge bases

Persistent Mapping of Sensor Data for Medium-Term Autonomy.

For vehicles to operate in unmapped areas with some degree of autonomy, it would be useful to aggregate and store processed sensor data so that it can be used later. In this paper, a tool that records and optimizes the placement of costmap data on a persistent map is presented. The optimization takes several factors into account, including local vehicle odometry, GPS signals when available, local map consistency, deformation of map regions, and proprioceptive GPS offset error. Results illustrating the creation of maps from previously unseen regions (a 100 m × 880 m test track and a 1.2 km dirt trail) are presented, with and without GPS signals available during the creation of the maps. Finally, two examples of the use of these maps are given. First, a path is planned along roads that have been seen exactly once during the mapping phase. Secondly, the map is used for vehicle localization in the absence of GPS signals.

Orchestrating data placement and query execution in heterogeneous CPU-GPU DBMS

There has been a growing interest in using GPU to accelerate data analytics due to its massive parallelism and high memory bandwidth. The main constraint of using GPU for data analytics is the limited capacity of GPU memory. Heterogeneous CPU-GPU query execution is a compelling approach to mitigate the limited GPU memory capacity and PCIe bandwidth. However, the design space of heterogeneous CPU-GPU query execution has not been fully explored. We aim to improve state-of-the-art CPU-GPU data analytics engine by optimizing data placement and heterogeneous query execution. First, we introduce a semantic-aware fine-grained caching policy which takes into account various aspects of the workload such as query semantics, data correlation, and query frequency when determining data placement between CPU and GPU. Second, we introduce a heterogeneous query executor which can fully exploit data in both CPU and GPU and coordinate query execution at a fine granularity. We integrate both solutions in Mordred, our novel hybrid CPU-GPU data analytics engine. Evaluation on the Star Schema Benchmark shows that the semantic-aware caching policy can outperform the best traditional caching policy by up to 3x. Compared to existing GPU DBMSs, Mordred can outperform by an order of magnitude.

DNA Pre-Alignment Filter Using Processing Near Racetrack Memory

Recent DNA <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pre-alignment</i> filter designs employ DRAM for storing the reference genome and its associated meta-data. However, DRAM incurs increasingly high energy consumption of background and refresh energy as devices scale. To overcome this problem, this paper explores a design with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">racetrack memory</i> (RTM)–an emerging non-volatile memory that promises higher storage density, faster access latency, and lower energy consumption. Multi-bit storage cells in RTM are inherently sequential and thus require data placement strategies to mitigate the performance and energy impacts of shifting during data accesses. We propose a near-memory pre-alignment filter with a novel data mapping and several shift reduction strategies designed explicitly for RTM. On a set of four input genomes from the 1000 Genome Project, our approach improves performance and energy efficiency by 68% and 52%, respectively, compared to the state-of-the-art DRAM-based architecture.

Cost modelling for optimal data placement in heterogeneous main memory

The cost of DRAM contributes significantly to the operating costs of in-memory database management systems (IMDBMS). Persistent memory (PMEM) is an alternative type of byte-addressable memory that offers --- in addition to persistence --- higher capacities than DRAM at a lower price with the disadvantage of increased latencies and reduced bandwidth. This paper evaluates PMEM as a cheaper alternative to DRAM for storing table base data, which can make up a significant fraction of an IMDBMS' total memory footprint. Using a prototype implementation in the SAP HANA IMDBMS, we find that placing all table data in PMEM can reduce query performance in analytical benchmarks by more than a factor of two, while transactional workloads are less affected. To quantify the performance impact of placing individual data structures in PMEM, we propose a cost model based on a lightweight workload characterization. Using this model, we show how to place data pareto-optimally in the heterogeneous memory. Our evaluation demonstrates the accuracy of the model and shows that it is possible to place more than 75% of table data in PMEM while keeping performance within 10% of the DRAM baseline for two analytical benchmarks.

Big Data Processing in Geo-Distributed Data Centers Using Cost Minimization Methods

Abstract: The explosive growth of demands on huge process imposes a crucial burden on computation, storage, and communication in data centers, that so incurs wide operational expenditure to data center suppliers. Therefore, Price decrease has become Associate in Nursing rising issue for the longer term huge data era. All totally different from customary cloud services, one in each of the foremost choices of huge data services is that the tight coupling between data and computation as computation tasks is also conducted provided that the corresponding data is obtainable. As a result, 3 factors, i.e., task assignment, data placement and data movement, deeply the operational expenditure of data centers. Throughout this paper, we tend to tend to unit of measurement meant to see the worth decrease downside via a joint improvement of these three factors for big data services in geo-distributed data centers. To elucidate the task completion time with the thought of every data transmission and computation, we tend to tend to propose a two-dimensional stochastic process and derive the standard task completion time in closed-form. Moreover, we tend to tend to model the matter as a mixed-integer non-linear programming (MINLP) and propose economical resolution to correct it. The high efficiency of our proposal is valid by thorough simulation primarily based studies Keywords— Nonlinear Programming, Big Data, Computer Centres, Integer Programming, Minimisation, Mixed Integer Nonlinear Programming, Cost Minimization, Data Centers, Big Data Services, Operational Expenditure, Task Assignment, Big Data

H2RUN: An efficient vendor lock‐in solution for multi‐cloud environment using horse herd Runge Kutta based data placement optimization

AbstractVendor lock‐in is a scenario where the cost of switching to another vendor is mainly high and for this reason, the consumer is mainly struck with a single cloud vendor. The vendor lock‐in mainly implies the situation of an organization where it is locked into a service or product of a single Cloud Service Providers (CSP) to prevent the disruption of the business operation or due to an insufficient workforce. Due to its high availability, portability, and other benefits, various researchers frequently adopt the multi‐cloud strategy to prevent vendor lock‐in. However, the multi‐cloud architecture also comes with major drawbacks such as cost, multiple CSP management, and the tradeoff between cost and availability. The organizations are mainly considerate about achieving the tradeoff between the cost and availability. To overcome this problem, we have proposed an H2RUN (Horse Herd Runge Kutta) optimization algorithm to achieve a tradeoff between the cost and availability using erasure coding. The proposed H2RUN optimization algorithm selects the optimal CSP and erasure coding parameters that enhance the availability of the CSP with minimal cost. The efficiency of the proposed methodology can be evaluated using the experiments conducted on different real‐time CSPs using various metrics. Based on the results achieved, the H2RUN optimizer minimizes the cost up to 48% and availability up to 58%.

Bi-Objective Optimizing for Data-Intensive ‎Scientific Workflow Scheduling in Cloud ‎Computing

Cloud Computing is increasingly recognized as a new way to use on-demand, computing, ‎storage and network services in a transparent and efficient way. Cloud Computing environment ‎consists of large customers requesting for cloud resources. Nowadays, task scheduling problem ‎and data placement are the current research topic in cloud computing. In this work, a new ‎technique for workflow scheduling and data placement are proposed based on genetic ‎algorithm to fulfill a final bi-objective goal such as minimizing total workflow response time and ‎cost of their execution. the scheduling of scientific workflows is considered to be an NP-complete ‎problem, i.e. a problem not solvable within polynomial time with current resources. The ‎performance of this proposed algorithm has been evaluated using CloudSim toolkit, Simulation ‎results show the effectiveness of the proposed algorithm in comparison with well-known ‎algorithms such as genetic algorithm with Random data placement. ‎

TDOPS: Time series based deduplication and optimal data placement strategy for IIoT in cloud environment

As a result of the advancements in Industry 4.0, the amount of data collected within industries are continuously expanding to achieve an innovative environment within the industry by maximizing asset usage. Meanwhile, the redundancy rate is increasing in cloud storage, which has an impact on data storage and analysis. To lower the rate of redundancy, the proposed system comprises a Time series-based deduplication technique. In the Time series-based deduplication technique, the Adaptive Multi-Pattern Boyer Moore Horspool (AM-BMH) algorithm, and Merkle tree were used to produce time-series data. Another significant challenge is that the geographically distributed cloud system has encountered that the data placement methodology with high-priced transportation costs for data transmission. To overcome this issue, an optimal data placement strategy using Modified Distribution is proposed. Thus the proposed Time Series-based Deduplication and Optimal Data Placement Strategy (TDOPS) is found to be effective when compared with the existing system. The various parameters like space reduction, efficient retrieval, data transportation costs, and data transmission time are taken into the account in the cloud environment for an evaluation. The proposed scheme saves 98 percent of storage space, 55 percent computation overhead, and improves 60% of cloud storage efficacy.

Fog-based caching mechanism for IoT data in information centric network using prioritization

In-Network caching is one of the most prominent features of Information-Centric Networks (ICN). This feature, designed to reduce the content delivery time, can improve data availability, especially during the sleeping mode of the original data producer. However, the fact that caching mechanism is simultaneously performed with data forwarding, limited processing power, and limited memory capacity of routers has posed challenges in fully profiting from this feature. The synchronization of caching mechanism with data forwarding limits the speed of decision-making in content placement policies. These challenges as well as the limitations of ICN routers have not been taken into consideration by the existing strategies for content placement of IoT data. In this paper, the restrictions for content placement policies in ICN-based IoT are outlined, and a data prioritization-based approach that prevents ICN nodes from filling their Content Store (CS) with inappropriate and inferior data is proposed to improve cache efficiency. Besides, fog computing is used as a middle layer between IoT devices and ICN networks to resolve the speed limits of decision-making and improve cache performance. In the fog layer, prioritization is performed based on the popularity and freshness of data object, and, thus, low-priority data are removed from the caching mechanism of ICN nodes. As the result shows the total number of cached data is reduced. Freshness is an essential property of multiple IoT data that significantly affects caching performance. That's why we prioritized data based on popularity and freshness. Eventually, the simulation results performed by the NDNsim simulator show that decreasing the number of data objects could improve the cache mechanism efficiency in terms of both delay and hit ratio.

Do Not Forget the “How” along with the “What”: Improving the Transparency of Sustainability Reports

Summary Considerable resources are invested in producing sustainability reports, yet few organizations reap the transparency benefits they promise. This article explores the way ten leading global fashion companies use a combination of data visualization and placement, stakeholder-driven interactive content, and multi-media and immersive content to build the trust necessary to improve their reporting and transparency. While few organizations have the resources of the global fashion giants, this article proposes a four-stage framework that guides managers through a step-change of systematic and targeted improvement.

Confinement forest‐based enhanced min‐min and max‐min technique for secure multicloud task scheduling

AbstractTask scheduling is the most important issue in the cloud computing environment since its implementation is largely dependent on it. However, due to its weaknesses in energy consumption for resource usage, high computational complexity, and memory needs, Virtual Machine (VM) cannot manage large amounts of workload. Furthermore, by reducing the number of active hosts' data assignments while utilizing the resources, a security risk occurs. To address these challenges, a Grey‐wolf with Deadline Restrictions approach was developed to allocate tasks and resources in less time while maintaining fixed interval constraints. Moreover, the Cognizing Anomaly detection method is utilized to find even the most recent abnormalities in the data. Furthermore, using the Confinement Forest‐based augmented Min‐Min and Max‐Min approach, the task scheduling process provides an efficient workload balancing mechanism by restricting makespan, energy consumption, load balancing, and fault tolerance. In contrast, enabling a security‐aware multicharacteristic‐based heuristic technique is employed to provide secure placement of data in the appropriate location. The proposed technique outperforms existing methods in terms of makespan and memory consumption.

Optimizing data placement and size configuration for morphable NVM based SPM in embedded multicore systems

Embedded multicore systems are widely designed to meet the high-performance requirement. Meanwhile, many embedded multicore systems are equipped with multiple scratchpad memories (SPM) because of their advantages in power efficiency and small area. Whereas, the traditional SRAM-based SPM is limited by its leakage power and capacity. Compared with SRAM, emerging non-volatile memory (NVM) has lower power consumption. Moreover, some morphable NVMs like resistive memory can build a “morphable NVM” to get a balance between capacity and performance, in which a memory cell can be converted between high-density MLC (multi-level cell) and low-latency SLC (single-level cell).Considering the features of NVM, this paper proposes a dynamic data placement and size configuration technique to fully utilize the benefits from morphable NVM based SPM in embedded multicore systems. The core idea is to dynamically convert the memory mode of NVM in each SPM with different workloads, and store the associated data into an optimal storage location for minimizing the total cost of memory access. Therefore, combining with the access pattern of embedded systems, an Integer Linear Programming (ILP) model is first explored for obtaining the optimal data placement and size configuration of SLC/MLC of SPMs. Then, our polynomial-time algorithm (CDA) is provided to obtain a near-optimal result. Finally, our experiments based on the gem5 simulator exhibit the proposed techniques can achieve the performance improvement and the reduction of energy consumption compared with the baseline scheme.

PENERAPAN LAYANAN BIMBINGAN KONSELING PAUD

Counseling guidance services in children in PAUD as assistance provided to children in developing optimal aspects of development with the aim that children become responsible and independent in solving problems. The stages of the implementation of counseling guidance services in early childhood include planning, implementation, evaluation, and follow-up. This type of research is a type of field research or case study. And it's qualitatively deskritive. Data collection techniques are carried out by interviewing to find out the school's point of view on the implementation of counseling guidance services in early childhood education while additional data in the form of observation and documentation. The result of this study is the implementation of counseling guidance services in early childhood education n has been going well, which is illustrated from the planning that includes the identification of learners' problems through the child's personal data form. Implementation includes data collection, information, extension, placement and devaluation services. Evaluation includes conformity between the program and implementation, compliance with the program, obstacles, responses of children, teachers and parents and changes in progress in the provision of guidance also running as a whole. Follow-up includes other methods performed if an action that has been performed has not been successful. Counseling guidance services include nonprofessional counseling guidance services, because those who become counseling guidance teachers are classroom teachers appointed by the principal as BK teachers.

Adaptive data placement in the Fog infrastructure of IoT applications with dynamic changes

Time-sensitive Internet of Things (IoT) applications need to provide instant responses to end users. Seeking to reduce the latency of IoT applications is challenging task. Due to the dynamic changes that can occur over time such as failure of IoT devices or integration of new services and devices, the IoT application constantly requires new storage resources. To ensure the high performance of an IoT application, it is common today to place its produced data in the Fog infrastructure while satisfying its requirements over time. The main challenge in this regard is to ensure optimal data placement in this heterogeneous and distributed infrastructure with low latency. In fact, with continuous changes, new data assignments to target storage nodes can lead to data migration and increased latency. In this paper, we propose an adaptive approach to place the IoT data with multiple replicas in the Fog infrastructure while estimating and optimizing the overall latency. To cope with the IoT application changes, we establish a ℓ-backtracking greedy algorithm called adaptive multiple data replicas placement of IoT applications with external event changes in the Fog infrastructure (iFogDPE). It is a stable many-to-many matching algorithm that minimizes the write, read and migration latency using a few number of data replicas. Then, it assigns each data consumer with its appropriate data replica. To model and to simulate the IoT application and Fog infrastructure enactment as well as to test and to evaluate our solution we used iFogSim simulator. Our experimental results highlight the efficiency of our approach in reducing the storage and access latency with fewer data replicas while maintaining low data migration latency over time.

Building a novel physical design of a distributed big data warehouse over a Hadoop cluster to enhance OLAP cube query performance

Improving OLAP (Online Analytical Processing) query performance in a distributed system on top of Hadoop is a challenging task. An OLAP Cube query comprises several relational operations, such as selection, join, and group-by aggregation. It is well-known that star join and group-by aggregation are the most costly operations in a Hadoop database system. These operations indeed increase network traffic and may overflow memory; to overcome these difficulties, numerous partitioning and data load balancing techniques have been proposed in the literature. However, some issues remain questionable, such as decreasing the Spark stages and the network I/O for an OLAP query being executed on a distributed system. In a precedent work, we proposed a novel data placement strategy for a big data warehouse over a Hadoop cluster. This data warehouse schema enhances the projection, selection, and star-join operations of an OLAP query, such that the system’s query-optimizer can perform a star join process locally, in only one spark stage without a shuffle phase. Also, the system can skip loading unnecessary data blocks when executing the predicates. In this paper, we extend our previous work with further technical details and experiments, and we propose a new dynamic approach to improve the group-by aggregation. To evaluate our approach, we conduct some experiments on a cluster with 15 nodes. Experimental results show that our method outperforms existing approaches in terms of OLAP query evaluation time.

Random Versus Copyset Placement: Data-Loss Models for Proactive-Tolerance Replica-Based Data Storage

In a distributed storage system, proactive fault tolerance is used to provide an extra layer of data protection over traditional reactive fault tolerance. The placement of replicas across storage servers in a replication-based system affects both rebuild times and the vulnerability of data to device failures, and therefore influences system reliability. The effects of proactive fault tolerance and data placement on the reliability of distributed storage systems are intricate and understudied. This article proposes reliability equations for predicting the number of data loss in proactive replication-based systems using random and copyset placement schemes, over a period of time. The reliability equations represent the effect of proactive fault tolerance, disk-operation failures, server-operation failures, rebuild bandwidths, and replica placement schemes upon the systems reliability, and take time-based Weibull distributions to model failure and repair processes. Moreover, a Monte-Carlo based method is designed to simulate the operation of proactive distributed storage systems. The results of the reliability equations are in good accord with those of the simulations, which verifies the correctness of the equations. The proposed equations can help system designers readily optimize tradeoffs and compare schemes, facilitating distributed storage systems design.

Improving the accessibility of NUMA‐aware C++ application development based on the PGASUS framework

AbstractCertain workloads such as in‐memory databases are inherently hard to scale‐out and rely on cache‐coherent scale‐up non‐uniform memory access (NUMA) systems to keep up with the ever‐increasing demand for compute resources. However, many parallel programming frameworks such as OpenMP do not make efficient use of large scale‐up NUMA systems as they do not consider data locality sufficiently. In this work, we present PGASUS, a C++ framework for NUMA‐aware application development that provides integrated facilities for NUMA‐aware task parallelism and data placement. The framework is based on an extensive review of parallel programming languages and frameworks to incorporate the best practices of the field. In a comprehensive evaluation, we demonstrate that PGASUS provides average performance improvements of and peak performance improvements of up to across a wide range of workloads.

Cost and Latency Optimized Edge Computing Platform

Latency-critical applications, e.g., automated and assisted driving services, can now be deployed in fog or edge computing environments, offloading energy-consuming tasks from end devices. Besides the proximity, though, the edge computing platform must provide the necessary operation techniques in order to avoid added delays by all means. In this paper, we propose an integrated edge platform that comprises orchestration methods with such objectives, in terms of handling the deployment of both functions and data. We show how the integration of the function orchestration solution with the adaptive data placement of a distributed key–value store can lead to decreased end-to-end latency even when the mobility of end devices creates a dynamic set of requirements. Along with the necessary monitoring features, the proposed edge platform is capable of serving the nomad users of novel applications with low latency requirements. We showcase this capability in several scenarios, in which we articulate the end-to-end latency performance of our platform by comparing delay measurements with the benchmark of a Redis-based setup lacking the adaptive nature of data orchestration. Our results prove that the stringent delay requisites necessitate the close integration that we present in this paper: functions and data must be orchestrated in sync in order to fully exploit the potential that the proximity of edge resources enables.

Dynamic data replication and placement strategy in geographically distributed data centers

AbstractWith the evolution of geographically distributed data centers in the Cloud Computing landscape along with the amount of data being processed in these data centers, which is growing at an exponential rate, processing massive data applications become an important topic. Since a given task may require many datasets for its execution and the datasets are spread over several different data centers, finding an efficient way to manage the datasets storage across nodes of a Cloud system is a difficult problem. In fact, the execution time of a task might be influenced by the cost of data transfers, which mainly depends on two criterias. The first one is the initial placement of the input datasets during the build‐time phase, while the second is the replication of the datasets during the runtime phase. The replication is explicitly considered when datasets are being migrated over the data centers in order to make them locally available wherever needed. Data placement and data replication are important challenges in Cloud Computing. Nevertheless, many studies focus on data placement or data replication exclusively. In this paper, a combination of a data placement strategy followed by a dynamic data replication management strategy is proposed, with the purpose of reducing the associated cost of all data transfers between the (distant) data centers. Our proposed data placement approach considers the main characteristics of a data center such as storage capacity and read/write speeds to efficiently store the datasets, while our dynamic data replication management approach considers three parameters: the number of replicas in the system, the dependency between datasets and tasks and the storage capacity of data centers. The decision of when and whether to keep or to delete replicas is determined by the fulfillment of those three parameters. Our approach estimates the total execution time of the tasks as well as the monetary cost, considering the data transfers activity. Our experiments are conducted using Cloudsim simulator. The obtained results show that our proposed strategies produce an efficient data management by reducing the overheads of the data transfers, compared to both a data placement without replication (by 76%) and the selected data replication approach from Kouidri et al. (by 52%), and by improving the financial cost.