Execution Time Prediction Research Articles

Regression-Based Prediction for Task-Based Program Performance

As multicore systems evolve by increasing the number of parallel execution units, parallel programming models have been released to exploit parallelism in the applications. Task-based programming model uses task abstractions to specify parallel tasks and schedules tasks onto processors at runtime. In order to increase the efficiency and get the highest performance, it is required to identify which runtime configuration is needed and how processor cores must be shared among tasks. Exploring design space for all possible scheduling and runtime options, especially for large input data, becomes infeasible and requires statistical modeling. Regression-based modeling determines the effects of multiple factors on a response variable, and makes predictions based on statistical analysis. In this work, we propose a regression-based modeling approach to predict the task-based program performance for different scheduling parameters with variable data size. We execute a set of task-based programs by varying the runtime parameters, and conduct a systematic measurement for influencing factors on execution time. Our approach uses executions with different configurations for a set of input data, and derives different regression models to predict execution time for larger input data. Our results show that regression models provide accurate predictions for validation inputs with mean error rate as low as 6.3%, and 14% on average among four task-based programs.

Location-Based Collaborative Filtering for Web Service Recommendation

Background: In many modern applications, information filtering is now used that exposes users to a collection of data. In such systems, the users are provided with recommended items’ list they might prefer or predict the rate that they might prefer for the items. So that, the users might be select the items that are preferred in that list. Objective: In web service recommendation based on Quality of Service (QoS), predicting QoS value will greatly help people to select the appropriate web service and discover new services. Methods: The effective method or technique for this would be Collaborative Filtering (CF). CF will greatly help in service selection and web service recommendation. It is the more general way of information filtering among the large data sets. In the narrower sense, it is the method of making predictions about a user’s interest by collecting taste information from many users. Results: It is easy to build and also much more effective for recommendations by predicting missing QoS values for the users. It also addresses the scalability problem since the recommendations are based on like-minded users using PCC or in clusters using KNN rather than in large data sources. Conclusion: In this paper, location-aware collaborative filtering is used to recommend the services. The proposed system compares the prediction outcomes and execution time with existing algorithms.

A Shared Memory Parallel Block Streaming Model for Irregular Applications

Due to worsening machine balance, a lightweight irregular application can utilize only a small fraction of the peak computational capacity on modern processors. Performance of such an application is also unpredictable due to the scattered data accesses. Even though architectural features such as a cache system, hardware prefetchers etc., that reduce the cost of irregular access, are commonly found in most of the modern processors, their design parameters differ widely from one processor to another. Therefore, a performance improving programming technique still needs extensive tuning to gain maximum benefit on a target processor. In this scenario, achieving portable performance becomes difficult. This work proposes a block streaming machine model and hypothesizes that an algorithm based on the model has predictable execution time. To enable adaptation of this model for irregular applications, we also provide algorithmic transformations that can be used to replace the scattered accesses with streaming accesses in a cost predictable way. Further, we experimentally demonstrate usefulness of the model and the transformations for static lightweight irregular computations such as those performed by a numerical partial differential equation solver on modern multicore processors.

Using Machine Learning Ensemble Methods to Predict Execution Time of e-Science Workflows in Heterogeneous Distributed Systems

Effective planning and optimized execution of the e-Science workflows in distributed systems, such as the Grid, need predictions of execution times of the workflows. However, predicting the execution times of e-Science workflows in heterogeneous distributed systems is a challenging job due to the complex structure of workflows, variations due to input problem-sizes, and heterogeneous and dynamic nature of the shared resources. To this end, we propose two novel workflow execution time-prediction methods based on the machine learning ensemble models. In this paper, we showcase our approach for two different real Grid environments. Our approach can effectively predict the execution time of the scientific workflow applications in the Grid for various problem sizes, Grid sites, and runtime environments. We characterized the workflow performance in the Grid using the attributes that define structure of workflow as well as the execution environment. Contrary to common ensembles, our ensemble systems employed three strong learners, which balance the weaknesses of each other by their strengths to model the workflow execution times. The proposed methods have been thoroughly evaluated for three real-world e-science workflow applications. The experimental results demonstrated that our proposed multi-model ensemble models can significantly decrease the prediction error (by 50%, on average) as compared with methods based on the radial basis function neural network, local learning, and performance templates. The proposed methods can also be applied with similar effectiveness and without any major modification for other heterogeneous distributed environments, such as the Cloud.

A Hardware and Software Task-Scheduling Framework Based on CPU+FPGA Heterogeneous Architecture in Edge Computing

Real-time performance is the primary requirement for edge computing systems. However, with the surge in data volume and the growing demand for computing power, a computing framework consisting solely of CPUs is no longer competent. As a result, CPU+ heterogeneous architecture using accelerators to improve edge computing systems' computing capacity has received great attention. The type of accelerators determines the performance of the edge computing system largely. The accelerators include Graphics Processing Unit (GPU), Application Specific Integrated Circuit (ASIC) and Field Programmable Gate Array (FPGA). FPGAs with its reconfigurability and high energy efficiency are widely used in many edge computing scenarios. Nontheless, the performance depends also on the scheduling efficiency between software tasks on CPUs and hardware tasks on FPGAs. Unfortunately, the existing strategies have not fully exploited the differences between hardware and software tasks, thus resulting in low scheduling efficiency. This paper proposes a task scheduling framework on the Dynamic Partial Reconfiguration (DPR) platform. We take full account of the characteristics of task switching overhead and predictable execution time of hardware tasks in DPR, and reduce the number of task-switching times and active tasks in the system, thus improving the scheduling efficiency. We conduct a set of experiments on the Zynq platform to verify the proposed framework. Experimental results demonstrate that when the execution time of the accelerator exceeds the reconfiguration cost by an order of magnitude, the efficiencies of all the cases are more than 98%, and the efficiencies can reach 90%-98% in the same order of magnitude.

D-Simplexed: Adaptive Delaunay Triangulation for Performance Modeling and Prediction on Big Data Analytics

Big Data processing systems (e.g., Spark) have a number of resource configuration parameters, such as memory size, CPU allocation, and the number of running nodes. Regular users and even expert administrators struggle to understand the mutual relation between different parameter configurations and the overall performance of the system. In this paper, we address this challenge by proposing a performance prediction framework, called d-Simplexed, to build performance models with varied configurable parameters on Spark. We take inspiration from the field of Computational Geometry to construct a d-dimensional mesh using Delaunay Triangulation over a selected set of features. From this mesh, we predict execution time for various feature configurations. To minimize the time and resources in building a bootstrap model with a large number of configuration values, we propose an adaptive sampling technique to allow us to collect as few training points as required. Our evaluation on a cluster of computers using WordCount, PageRank, Kmeans, and Join workloads in HiBench benchmarking suites shows that we can achieve less than 5% error rate for estimation accuracy by sampling less than 1% of data.

Improving the MXFT scheduling algorithm for a cloud computing context

In this paper, the Max-Min Fast Track (MXFT) scheduling algorithm is improved and compared against a selection of popular algorithms. The improved versions of MXFT are called Min-Min Max-Min Fast Track (MMMXFT) and Clustering Min-Min Max-Min Fast Track (CMMMXFT). The key difference is using Min-Min for the fast track. Experimentation revealed that despite Min-Min's characteristic of prioritising small tasks at the expense of overall makespan, the overall makespan was not adversely affected and the benefits of prioritising small tasks were identified in MMMXFT. Experiments were conducted by using a simulator with the exception of one real-world experiment. The real-world experiment identified challenges faced by algorithms which rely on accurate execution time prediction.

Improving the MXFT scheduling algorithm for a cloud computing context

In this paper, the Max-Min Fast Track (MXFT) scheduling algorithm is improved and compared against a selection of popular algorithms. The improved versions of MXFT are called Min-Min Max-Min Fast Track (MMMXFT) and Clustering Min-Min Max-Min Fast Track (CMMMXFT). The key difference is using Min-Min for the fast track. Experimentation revealed that despite Min-Min's characteristic of prioritising small tasks at the expense of overall makespan, the overall makespan was not adversely affected and the benefits of prioritising small tasks were identified in MMMXFT. Experiments were conducted by using a simulator with the exception of one real-world experiment. The real-world experiment identified challenges faced by algorithms which rely on accurate execution time prediction.

A Systematic Review of Modifications and Validation Methods for the Extension of the Keystroke-Level Model

The keystroke-level model (KLM) is the simplest model of the goals, operators, methods, and selection rules (GOMS) family. The KLM computes formative quantitative predictions of task execution time. This paper provides a systematic literature review of KLM extensions across various applications and setups. The objective of this review is to address research questions concerning the development and validation of extensions. A total of 54 KLM extensions have been exhaustively reviewed. The results show that the original keystroke and mental act operators were continuously preserved or adapted and that the drawing operator was used the least. Excluding the original operators, almost 45 operators were collated from the primary studies. Only half of the studies validated their model’s efficiency through experiments. The results also identify several research gaps, such as the shortage of KLM extensions for post-GUI/WIMP interfaces. Based on the results obtained in this work, this review finally provides guidelines for researchers and practitioners.

A Machine Learning Approach for Predicting Execution Time of Spark Jobs

Spark has gained growing attention in the past couple of years as an in-memory cloud computing platform. It supports execution of various types of workloads such as SQL queries and machine learning applications. Currently, many enterprises use Spark to exploit its fast in-memory processing of large scale data. Additionally, speeding up the execution in Spark is an important problem for many real-time applications. This can be achieved by improving the scheduling approaches employed by Spark, optimizing the execution plans generated by Spark for various applications, and selecting the best cluster configuration to run an input workload. A first step for all these optimization approaches is to predict the execution time of an input Spark application. In this paper, we present a new platform that predicts with high accuracy the execution time of SQL queries and machine learning applications executed by Spark. We evaluate our proposed platform by measuring the accuracy of predicting execution time of various types of Spark jobs including TPC-H queries and machine learning classification/clustering applications. The evaluation experiments show that we are able to predict the execution time of Spark jobs using our proposed platform with accuracy greater than 90% for SQL queries and greater than 75% for machine learning jobs.

TDQN: Trace-driven analytic queuing network modeling of computer systems

Many current performance studies of computer systems are evaluated by subjecting the system under evaluation to a workload obtained from publicly available traces. Many current performance studies of computer systems are evaluated by using publicly available traces as inputs. This approach provides credibility to performance studies but generally precludes the use of computationally efficient analytic models that rely on strict stochastic assumptions. Simulation or prototype implementations, less general and more time consuming methods, are used when the use of traces limits the applicability of analytic performance models. Analytic models (e.g., queuing theory for single queues or queuing networks) have extensively been used to estimate job execution times under steady state conditions. This paper discusses the use of closed Queuing Network (QN) models during finite time intervals to estimate the execution time of jobs submitted to a computer system. The paper presents the Trace-Driven Queuing Network (TDQN) algorithm that allows job traces to be used as input to analytic models. Validations against experimental results showed that the absolute relative error between measurements and execution time predictions obtained with the TDQN algorithm is below 10% in most cases. Additionally, the paper shows how the TDQN algorithm can be used to estimate the makespan of a stream of jobs submitted to a scheduler, which decides which computer of a cluster will process the job.

Resilience-Aware Resource Management for Exascale Computing Systems

With the increases in complexity and number of nodes in large-scale high performance computing (HPC) systems over time, the probability of applications experiencing runtime failures has increased significantly. Projections indicate that exascale-sized systems are likely to operate with mean time between failures (MTBF) of as little as a few minutes. Several strategies have been proposed in recent years for enabling systems of these extreme sizes to be resilient against failures. This work provides a comparison of four state-of-the-art HPC resilience protocols that are being considered for use in exascale systems. We explore the behavior of each resilience protocol operating under the simulated execution of a diverse set of applications and study the performance degradation that a large-scale system experiences from the overhead associated with each resilience protocol as well as the re-computation needed to recover when a failure occurs. Using the results from these analyses, we examine how resource management on exascale systems can be improved by allowing the system to select the optimal resilience protocol depending upon each application's execution characteristics, as well as providing the system resource manager the ability to make scheduling decisions that are “resilience aware” through the use of more accurate execution time predictions.

A Design-Time/Run-Time Application Mapping Methodology for Predictable Execution Time in MPSoCs

Executing multiple applications on a single MPSoC brings the major challenge of satisfying multiple quality requirements regarding real-time, energy, and so on. Hybrid application mapping denotes the combination of design-time analysis with run-time application mapping. In this article, we present such a methodology, which comprises a design space exploration coupled with a formal performance analysis. This results in several resource reservation configurations, optimized for multiple objectives, with verified real-time guarantees for each individual application. The Pareto-optimal configurations are handed over to run-time management, which searches for a suitable mapping according to this information. To provide any real-time guarantees, the performance analysis needs to be composable and the influence of the applications on each other has to be bounded. We achieve this either by spatial or a novel temporal isolation for tasks and by exploiting composable networks-on-chip (NoCs). With the proposed temporal isolation, tasks of different applications can be mapped to the same resource, while, with spatial isolation, one computing resource can be exclusively used by only one application. The experiments reveal that the success rate in finding feasible application mappings can be increased by the proposed temporal isolation by up to 30% and energy consumption can be reduced compared to spatial isolation.

ESPRET: A tool for execution time estimation of manual test cases

Manual testing is still a predominant and an important approach for validation of computer systems, particularly in certain domains such as safety-critical systems. Knowing the execution time of test cases is important to perform test scheduling, prioritization and progress monitoring. In this work, we present, apply and evaluate ESPRET (EStimation and PRediction of Execution Time) as our tool for estimating and predicting the execution time of manual test cases based on their test specifications. Our approach works by extracting timing information for various steps in manual test specification. This information is then used to estimate the maximum time for test steps that have not previously been executed, but for which textual specifications exist. As part of our approach, natural language parsing of the specifications is performed to identify word combinations to check whether existing timing information on various test steps is already available or not. Since executing test cases on the several machines may take different time, we predict the actual execution time for test cases by a set of regression models. Finally, an empirical evaluation of the approach and tool has been performed on a railway use case at Bombardier Transportation (BT) in Sweden.

Scheduling in volunteer computing networks, based on neural network prediction of the job execution time

ABSTRACTImprovement of scheduling which broadly speaking means the distribution of jobs to volunteers is very important for improving the effectiveness of volunteer computing networks operating on the basis of computing resources connected to the Internet. The scheduling strategy based on the prediction of the job execution time is chosen as the main strategy to solve this problem. Suggested approach includes a neural network mechanism for computing the predictive estimate of the job execution time and a genetic algorithm for distributing jobs to volunteers with adjustment of parameters that makes it possible to respond to changes in the computing environment. The features of the approach are illustrated by computing experiments. In addition, we consider an example of the distribution of jobs to two volunteers for a project consisting of three applications. Even approximate (interval) estimates of job execution time allowed reducing the total execution time of the project and thereby optimising the computing process.

Thoroughly Exploring GPU Buffering Options for Stencil Code by Using an Efficiency Measure and a Performance Model

Stencil computations form the basis for computer simulations across almost every field of science, such as computational fluid dynamics, data mining, and image processing. Their mostly regular data access patterns potentially enable them to take advantage of the high computation and data bandwidth of GPUs, but only if data buffering and other issues are handled properly. Finding a good code generation strategy presents a number of challenges, one of which is the best way to make use of memory. GPUs have several types of on-chip storage including registers, shared memory, and a read-only cache. The choice of type of storage and how it’s used, a buffering strategy , for each stencil array ( grid function , [GF]) not only requires a good understanding of its stencil pattern, but also the efficiency of each type of storage for the GF, to avoid squandering storage that would be more beneficial to another GF. For a stencil computation with $N$ GFs, the total number of possible assignments is $\beta ^{N}$ where $\beta$ is the number of buffering strategies. Our code-generation framework supports five buffering strategies ( $\beta =5$ ). Large, complex stencil kernels may consist of dozens of GFs, resulting in significant search overhead. In this work, we present an analytic performance model for stencil computations on GPUs and study the behavior of read-only cache and L2 cache. Next, we propose an efficiency-based assignment algorithm which operates by scoring a change in buffering strategy for a GF using a combination of (a) the predicted execution time and (b) on-chip storage usage. By using this scoring, an assignment for $N$ GFs can be determined in $(\beta -1)N(N+1)/2$ steps. Results show that the performance model has good accuracy and that the assignment strategy is highly efficient.

Performance modeling for GPUs using abstract kernel emulation

Performance modeling of GPU kernels is a significant challenge. In this paper, we develop a novel approach to performance modeling for GPUs through abstract kernel emulation along with latency/gap modeling of resources. Experimental results on all benchmarks from the Rodinia suite demonstrate good accuracy in predicting execution time on multiple GPU platforms.

An Evaluation Model and Benchmark for Parallel Computing Frameworks

MARS and Spark are two popular parallel computing frameworks and widely used for large-scale data analysis. In this paper, we first propose a performance evaluation model based on support vector machine (SVM), which is used to analyze the performance of parallel computing frameworks. Furthermore, we give representative results of a set of analysis with the proposed analytical performance model and then perform a comparative evaluation of MARS and Spark by using representative workloads and considering factors, such as performance and scalability. The experiments show that our evaluation model has higher accuracy than multifactor line regression (MLR) in predicting execution time, and it also provides a resource consumption requirement. Finally, we study benchmark experiments between MARS and Spark. MARS has better performance than Spark in both throughput and speedup in the executions of logistic regression and Bayesian classification because MARS has a large number of GPU threads that can handle higher parallelism. It also shows that Spark has lower latency than MARS in the execution of the four benchmarks.

A clustering-based hybrid approach for dual data reduction

The research on data reduction techniques has become important to enhance the efficacy and efficiency of data mining algorithms which may otherwise be compromised in the presence of a large number of irrelevant attributes and redundant instances. Data can be reduced by selecting either a subset of attributes or instances. Dual selection treats the problem of feature and instance selection together as a single optimisation problem. The problem of dual selection is relatively difficult as it involves an enormously large search space. In this paper, we propose a hybrid instance feature selection; HIFS-CHC method using heterogeneous recombination and cataclysmic mutation; CHC adaptive search genetic algorithm to solve the problem of dual selection. The proposed approach works in two stages. In the first stage, K-means clustering algorithm is used to reduce the search space. The second stage incorporates stratified prototype selection and CHC algorithm for data reduction. The clustering based hybrid scheme is experimentally tested on sixteen benchmark datasets and compared with the other similar data reduction algorithms with respect to the predictive accuracy, reduction rate and execution time. Experimental results show that the proposed method outperforms the other methods in terms of reduction rate and execution time while preserving the predictive accuracy almost at the same level.

Scaling associative classification for very large datasets

Supervised learning algorithms are nowadays successfully scaling up to datasets that are very large in volume, leveraging the potential of in-memory cluster-computing Big Data frameworks. Still, massive datasets with a number of large-domain categorical features are a difficult challenge for any classifier. Most off-the-shelf solutions cannot cope with this problem. In this work we introduce DAC, a Distributed Associative Classifier. DAC exploits ensemble learning to distribute the training of an associative classifier among parallel workers and improve the final quality of the model. Furthermore, it adopts several novel techniques to reach high scalability without sacrificing quality, among which a preventive pruning of classification rules in the extraction phase based on Gini impurity. We ran experiments on Apache Spark, on a real large-scale dataset with more than 4 billion records and 800 million distinct categories. The results showed that DAC improves on a state-of-the-art solution in both prediction quality and execution time. Since the generated model is human-readable, it can not only classify new records, but also allow understanding both the logic behind the prediction and the properties of the model, becoming a useful aid for decision makers.