Performance Modeling Of Applications Research Articles

Sonnet: A control-theoretic approach for resource allocation in cluster management

Cluster users expect to minimize the resource costs while ensuring target performance for different applications. It is particularly difficult to reach such a goal, because the applications are diverse with dynamic load changes, and interference exists between them. In addition, the performance of the applications depends on heterogeneous resources with different costs. However, existing works either use simplistic and generalized heuristics that disregard resource-specific characteristics or need suspending service to get expert knowledge to optimize the resource cost for a brand-new application or runtime, which fails to optimize the resource allocation finely.In this paper, we propose Sonnet, a control-theoretic approach to perform efficient resource allocation. Sonnet can efficiently optimize the cost of resources while satisfying the SLO by quickly establishing new application performance models through only online profiling and without affecting service. Experiments on Docker Swarm using various open-source benchmarks demonstrate that Sonnet can decrease the SLO violation rate by 91% while reducing resource costs up to 47% compared with the state-of-the-arts.

Combining multitask and transfer learning with deep Gaussian processes for autotuning-based performance engineering

We combine deep Gaussian processes (DGPs) with multitask and transfer learning for the performance modeling and optimization of HPC applications. Deep Gaussian processes merge the uncertainty quantification advantage of Gaussian processes (GPs) with the predictive power of deep learning. Multitask and transfer learning allow for improved learning efficiency when several similar tasks are to be learned simultaneously and when previous learned models are sought to help in the learning of new tasks, respectively. A comparison with state-of-the-art autotuners shows the advantage of our approach on two application problems. In this article, we combine DGPs with multitask and transfer learning to allow for both an improved tuning of an application parameters on problems of interest but also the prediction of parameters on any potential problem the application might encounter.

Research of LoRaWAN productivity performance models for building IoT networks

The study of the LoRaWAN method applicationperformance models intheIoT networks, which is currently being intensively developed, improved and is an important component of the information society. LoRa is a new long-range, low-power wireless technology that is key to building IoT networks around the world. Unlike other wireless technologies, the signal range and autonomy are enormous. Unlike GSM networks, it does not require bulky equipment with a high level of radiation.Itcan be easily used in places with mass construction without harm to human health. Two main scenarios for modeling performance improvement were investigated. Based on the results of the research, it is concluded that doubling the bandwidth effectively doubles the baud rate, and increasing the bandwidth reduces the sensitivity of the receiver, while increasing the propagation factor increases the sensitivity of the receiver. It is shown that by slightly changing the ACK procedure, it is possible to significantly improve the system performance in terms of packet delivery factor, system capacity, and energyefficiency. Conversely, it is determined that other system parameters are already well configured.

Developing Network Slurry Seal Performance Models Using Pavement Management Systems Data

As a popular pavement preservation treatment, a slurry seal is typically included in agencies’ maintenance and rehabilitation toolboxes, in decision trees of pavement management systems (PMS), or both. However, performance prediction models for slurry seals are mostly anecdotal or at undertaken at project level. The goal of this paper is to illustrate a method of developing data-driven performance models for slurry seal applications for use at a network level. A total of 537,891 pavement condition records were collected from 18 different agencies in the United States using the Metropolitan Transportation Commission’s PMS program, StreetSaver®. Of these records, 1,195 slurry seal projects met the requirements to be utilized in the development of slurry seal performance models. The development of slurry seal performance models includes grouping project-level modeling results into functional classes, and testing and validating the slurry seal performance models with test cases as well as on actual data produced by agencies. The performance models for slurry seals placed on an asphalt concrete pavement vary depending mainly on the existing pavement’s performance, the timing of the slurry seal application, the functional class, and surface type. The data-driven slurry seal models improved the accuracy of predicting slurry seal performance using PMS. The models could also be used to develop beneficial area curves, which are useful in estimating the cost-effectiveness of slurry seals constructed at various pavement condition index values and in identifying the optimal timings for slurry seal applications at a network level.

Guaranteeing Performance SLAs of Cloud Applications Under Resource Storms

In modern data centers, enterprise cloud instances run not only foreground applications like web and databases, but also different background services (e.g., backup, virus/compliance scan, batch) to manage the cloud instances securely and improve the overall resource utilization. These background services often incur resource storms that suddenly consume a lot of shared resources on cloud instances. The resource storms significantly degrade the performance of foreground applications by interfering in the preemption of the shared resources, resulting in frequent SLA violations. However, stock OS schedulers are not designed to handle these situations, and prior works are insufficient to address such resource storms under highly dynamic cloud workloads. This work presents Orchestra, a cloud-specific framework for controlling multiple applications in the user space, aiming at meeting corresponding SLAs. Orchestra takes an online approach with lightweight monitoring and performance models for both applications on the fly. It optimizes the resource allocations to meet corresponding SLAs. We evaluate the performance of Orchestra on a production cloud with a diverse range of SLAs. Orchestra guarantees the foreground application's performance SLAs at all times. At the same time, Orchestra maintains the background's performance by minimizing its performance penalty with proper allocation of the shared resources.

Erratum to: A Hybrid Machine Learning Approach for Performance Modeling of Cloud-Based Big Data Applications

Erratum to: A Hybrid Machine Learning Approach for Performance Modeling of Cloud-Based Big Data Applications

A Hybrid Machine Learning Approach for Performance Modeling of Cloud-Based Big Data Applications

Abstract Nowadays, Apache Hadoop and Apache Spark are two of the most prominent distributed solutions for processing big data applications on the market. Since in many cases these frameworks are adopted to support business critical activities, it is often important to predict with fair confidence the execution time of submitted applications, for instance when service-level agreements are established with end-users. In this work, we propose and validate a hybrid approach for the performance prediction of big data applications running on clouds, which exploits both analytical modeling and machine learning (ML) techniques and it is able to achieve a good accuracy without too many time consuming and costly experiments on a real setup. The experimental results show how the proposed approach attains improvement in accuracy, number of experiments to be run on the operational system and cost over applying ML techniques without any support from analytical models. Moreover, we compare our approach with Ernest, an ML-based technique proposed in the literature by the Spark inventors. Experiments show that Ernest can accurately estimate the performance in interpolating scenarios while it fails to predict the performance when configurations with increasing number of cores are considered. Finally, a comparison with a similar hybrid approach proposed in the literature demonstrates how our approach significantly reduce prediction errors especially when few experiments on the real system are performed.

Online Performance Modeling and Prediction for Single-VM Applications in Multi-Tenant Clouds

Clouds have been adopted widely by many organizations for their supports of flexible resource demands and low cost, which is normally achieved through sharing the underlying hardware among multiple cloud tenants. However, such sharing with the changes in resource contentions in virtual machines (VMs) can result in large variations for the performance of cloud applications, which makes it difficult for ordinary cloud users to estimate the run-time performance of their applications. In this article, we propose online learning methodologies for performance modeling and prediction of applications that run repetitively on multi-tenant clouds (such as on-line data analytic tasks). Here, a few micro-benchmarks are utilized to probe the in-situ perceivable performance of CPU, memory and I/O components of the target VM. Then, based on such profiling information and in-place measured application’s performance, the predictive models can be derived with either Regression or Neural-Network techniques. In particular, to address the changes in the intensity of resource contentions of a VM over time and its effects on the target application, we proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">periodic model retraining</i> where the sliding-window technique was exploited to control the frequency and historical data used for model retraining. Moreover, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">progressive modeling</i> approach has been devised where the Regression and Neural-Network models are gradually updated for better adaptation to recent changes in resource contention. With 17 representative applications from PARSEC, NAS Parallel and CloudSuite benchmarks being considered, we have extensively evaluated the proposed online schemes for the prediction accuracy of the resulting models and associated overheads on both a private and public clouds. The evaluation results show that, even on the private cloud with high and radically changed resource contention, the average prediction errors of the considered models can be less than 20 percent with periodic retraining. The prediction errors generally decrease with higher retraining frequencies and more historical data points but incurring higher run-time overheads. Furthermore, with the neural-network progressive models, the average prediction errors can be reduced by about 7 percent with much reduced run-time overheads (up to 265 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> ) on the private cloud. For public clouds with less resource contentions, the average prediction errors can be less than 4 percent for the considered models with our proposed online schemes.

Automated Performance Modeling of HPC Applications Using Machine Learning

Automated performance modeling and performance prediction of parallel programs are highly valuable in many use cases, such as in guiding task management and job scheduling, offering insights of application behaviors, and assisting resource requirement estimation. The performance of parallel programs is affected by numerous factors, including but not limited to hardware, applications, algorithms, and input parameters, thus an accurate performance prediction is often a challenging and daunting task. In this article, we focus on automatically predicting the execution time of parallel programs (more specifically, MPI programs) with different inputs, at different scales, and without domain knowledge. We model the correlation between the execution time and domain-independent runtime features. These features include values of variables, counters of branches, loops, and MPI communications. Through automatically instrumenting an MPI program, each execution of the program will output a feature vector and its corresponding execution time. After collecting data from executions with different inputs, a random forest machine learning approach is used to build an empirical performance model, which can predict the execution time of the program given a new input. A transfer learning method is used to reuse an existing performance model and improve the prediction accuracy on a new platform that lacks historical execution data. Our experiments and analyses of three parallel applications, Graph500, GalaxSee, and SMG2000, on three different systems confirm that our method performs well, with less than 20 percent prediction error on average.

Performance Modeling and Workflow Scheduling of Microservice-Based Applications in Clouds

Microservice has been increasingly recognized as a promising architectural style for constructing large-scale cloud-based applications within and across organizational boundaries. This microservice-based architecture greatly increases application scalability, but meanwhile incurs an expensive performance overhead, which calls for a careful design of performance modeling and task scheduling. However, these problems have thus far remained largely unexplored. In this paper, we develop a performance modeling and prediction method for independent microservices, design a three-layer performance model for microservice-based applications, formulate a Microservice-based Application Workflow Scheduling problem for minimum end-to-end delay under a user-specified Budget Constraint (MAWS-BC), and propose a heuristic microservice scheduling algorithm. The performance modeling and prediction method are validated and justified by experimental results generated through a well-known microservice benchmark on disparate computing nodes, and the performance superiority of the proposed scheduling solution is illustrated by extensive simulation results in comparison with existing algorithms.

Analytical composite performance models for Big Data applications

Recent years witnessed a steep rise in data generation and, consequently, the widespread adoption of software solutions able to support data-intensive applications. Many companies currently engage in data-intensive processes, however, fully embracing a data-driven paradigm is still cumbersome, and establishing a production-ready and fine-tuned deployment is time-consuming. This situation calls for innovative models and techniques to streamline the process of deployment configuration for Big Data applications. Moreover, many companies are using Cloud deployed clusters, which represent a cost-effective alternative to installation on premises. Accurate and fast prediction of the execution time of a Big Data application helps improving design time decisions, reduces Cloud over allocation charges, and assists budget management. In this paper, analytical models based on Stochastic Activity Networks (SANs) are proposed to accurately model the execution of Hadoop, Tez and Spark applications, i.e., the most referred frameworks to support Big Data analyses. The proposed SANs model these applications together with the underlying cluster in order to accurately estimate the execution time. We evaluate the accuracy of the proposed models over the TPC-DS industry benchmark across different configurations. Results obtained by numerically solving the SAN models show an average error of 4.5%, 5.8%, and 2.7% in estimating the execution time of MR, Tez, and Spark applications, respectively, against the data gathered from the experiments demonstrating higher accuracy compared with the state-of-the-art. Moreover, the time required to solve the proposed models is lower than the simulation time of the previously presented approaches in this area.

A gray-box performance model for Apache Spark

Apache Spark is a powerful open source data processing platform. It is getting more and more popular with the growing need of processing massive amounts of data. A performance prediction model not only helps administrators to have a better understanding of system behavior, but also is useful in performance tuning. However, considering the complex application processing mechanism of Spark, it is not an easy job to model the relationship between system performance and configuration settings.In this paper, we present a gray-box performance model for Spark applications based on machine learning algorithms. Given a specific Spark application, the size of its input data and some key system parameters, this performance model is able to forecast its execution time according to history information. To achieve better accuracy, our model takes basic hardware information and the resource allocation strategy of Spark into consideration.In our experiments, result shows our gray-box model is better than typical black-box approaches in most of the cases. We consider this model is helpful for further researches on Apache Spark.

Mind Your State for Your State of Mind

Applications have had an interesting evolution as they have moved into the distributed and scalable world. Similarly, storage and its cousin databases have changed side by side with applications. Many times, the semantics, performance, and failure models of storage and applications do a subtle dance as they change in support of changing business requirements and environmental challenges. Adding scale to the mix has really stirred things up. This article looks at some of these issues and their impact on systems.

ElaClo: A framework for optimizing software application topology in the cloud environment

Abstract Application architectures in the cloud employ elastic components, and achieve lower operating costs without sacrificing quality. Software architects strive to provide efficient services by deciding on software topology: a set of structural architectural decisions. For a given application, there can be numerous software topology alternatives creating the need for automated optimization methods. Current optimization approaches rely on experts providing application performance models built upfront, based on their experience and the requirements provided. While such techniques are effective and valuable, they require additional maintenance effort as the software evolves. This paper introduces ElaClo, a framework for optimizing application topologies in a cloud environment. ElaClo’s main contribution is in providing optimization in the software assembly phase from automatically extracted application models. ElaClo provides workload generation, monitoring, topology management, elasticity mechanisms, and algorithms to support the optimization process. We have implemented ElaClo as an expert tool and evaluated it on a real-life cloud application from the retailing business domain. ElaClo was used to select optimal topologies with regards to service response time objectives and infrastructure costs. The efficiency of the optimization process and the quality of optimization results were validated quantitatively on a set of optimization runs. Results demonstrate the effectiveness of the suggested framework in yielding optimal topologies.

Performance modeling of big data applications in the cloud centers

Cloud computing has evolved as an efficient paradigm to process big data applications. Performance evaluation of cloud center is a necessary prerequisite to guarantee quality of service. However, it is a challenge task to effectively analyze the performance of cloud service due to the complexity of cloud resources and the diversity of big data applications. In this paper, we leverage queuing theory and probabilistic statistics to propose a performance evaluation model for cloud center under big data application arrivals. In this model, the tasks (i.e., big data applications) are with Poisson arrivals, each task is divided into lots of parallel subtasks, and the number of subtasks follows a general distribution. The model allows to calculate the important performance indicators such as mean number of subtasks in the system, the probability that a task obtains immediate service, task waiting time and blocking probability. The model can also be used to predict the time cost of performing application. Finally, we use the simulations and benchmarking running WordCount and TeraSort applications on a Hadoop platform to demonstrate the utility of the model.

Performance Modeling of Multi-tiered Web Applications with Varying Service Demands

Multi-tiered transactional web applications are frequently used in IT enterprise based systems. Due to their inherent distributed nature, pre-deployment testing for high-availability and varying concurrency are important for post-deployment performance. Accurate performance modeling of such applications can help estimate values for future deployment variations as well as validate experimental results. In order to theoretically model performance of multi-tiered applications, we use queuing networks and Mean Value Analysis (MVA) models. While MVA has been shown to work well with closed queuing networks, there are particular limitations in cases where the service demands vary with concurrency. This variation of service demands for various resources (CPU, Disk, Network) is demonstrated through multiple experiments. This is further contrived by the use of multi-server queues in multi-core CPUs, that are not traditionally captured in MVA. We compare performance of a multi-server MVA model alongside actual performance testing measurements and demonstrate this deviation. Using spline interpolation of collected service demands, we show that a modified version of the MVA algorithm (called MVASD ) that accepts an array of service demands, can provide superior estimates of maximum throughput and response time. Results are demonstrated over multi-tier vehicle insurance registration and e-commerce web applications. The mean deviations of predicted throughput and response time are shown to be less the 3% and 9% , respectively. Additionally, we analyze the effect of spline interpolation of service demands as a function of throughput on the prediction results. Using Chebyshev Nodes , the tradeoff between the number of test points and the spline interpolation/prediction accuracy is also studied.Â

Ensemble: A Tool for Performance Modeling of Applications in Cloud Data Centers

We introduce Ensemble, a runtime framework and associated tools for building application performance models on-the-fly. These dynamic performance models can be used to support complex, highly dimensional resource allocation, and/or what-if performance inquiry in modern heterogeneous environments, such as data centers and Clouds. Ensemble combines simple, partially specified, and lower-dimensionality models to provide good initial approximations for higher dimensionality application performance models. We evaluated Ensemble on industry-standard and scientific applications. The results show that Ensemble provides accurate, fast, and flexible performance models even in the presence of significant environment variability.

Evolutionary approaches to signal decomposition in an application service management system

The increased demand for autonomous control in enterprise information systems has generated interest on efficient global search methods for multivariate datasets in order to search for original elements in time-series patterns, and build causal models of systems interactions, utilization dependencies, and performance characteristics. In this context, activity signals deconvolution is a necessary step to achieve effective adaptive control in Application Service Management. The paper investigates the potential of population-based metaheuristic algorithms, particularly variants of particle swarm, genetic algorithms and differential evolution methods, for activity signals deconvolution when the application performance model is unknown a priori. In our approach, the Application Service Management System is treated as a black- or grey-box, and the activity signals deconvolution is formulated as a search problem, decomposing time-series that outline relations between action signals and utilization-execution time of resources. Experiments are conducted using a queue-based computing system model as a test-bed under different load conditions and search configurations. Special attention was put on high-dimensional scenarios, testing effectiveness for large-scale multivariate data analyses that can obtain a near-optimal signal decomposition solution in a short time. The experimental results reveal benefits, qualities and drawbacks of the various metaheuristic strategies selected for a given signal deconvolution problem, and confirm the potential of evolutionary-type search to effectively explore the search space even in high-dimensional cases. The approach and the algorithms investigated can be useful in support of human administrators, or in enhancing the effectiveness of feature extraction schemes that feed decision blocks of autonomous controllers.

Online root-cause performance analysis of parallel applications

We present a technique for automated application performance modeling and analysis.A parallel application is modeled by communication and computational activities.The analysis discovers causal dependencies between problems and infers root causes.It compares concurrent execution flows correlating problems in causal relationship.Results of root-cause analysis for different MPI applications are encouraging. The evolution of hardware is improving at an incredible rate. However, the advances in parallel software have been hampered for many reasons. Developing an efficient parallel application is still not an easy task. Applications rarely achieve good performances immediately and, therefore, careful performance analysis and optimization are crucial. These tasks are difficult and require a thorough understanding of the programs behavior. In this paper, we propose a systematic approach to online root-cause performance analysis. The automated analysis uses an online model to quickly identify the most important performance problems, and correlates them with application source code. Our technique is able to discover causal dependencies among the problems, infer their root causes and explain them to developers. In all of the scenarios we performed, this online modelling and analysis approach allowed us to understand the behavior of the applications, evaluate the performance and locate problem causes without specific knowledge of application internals.

Parallel performance modeling of irregular applications in cell-centered finite volume methods over unstructured tetrahedral meshes

Finite volume methods are widely used numerical strategies for solving partial differential equations. This paper aims at obtaining a quantitative understanding of the achievable performance of the cell-centered finite volume method on 3D unstructured tetrahedral meshes, using traditional multicore CPUs as well as modern GPUs. By using an optimized implementation and a synthetic connectivity matrix that exhibits a perfect structure of equal-sized blocks lying on the main diagonal, we can closely relate the achievable computing performance to the size of these diagonal blocks. Moreover, we have derived a theoretical model for identifying characteristic levels of the attainable performance as a function of hardware parameters, based on which a realistic upper limit of the performance can be predicted accurately. For real-world tetrahedral meshes, the key to high performance lies in a reordering of the tetrahedra, such that the resulting connectivity matrix resembles a block diagonal form where the optimal size of the blocks depends on the hardware. Numerical experiments confirm that the achieved performance is close to the practically attainable maximum and it reaches 75% of the theoretical upper limit, independent of the actual tetrahedral mesh considered. From this, we develop a general model capable of identifying bottleneck performance of a system’s memory hierarchy in irregular applications.