Job Runtime Research Articles

Combining Machine Learning techniques and Genetic Algorithm for predicting run times of High Performance Computing jobs

This study proposes a novel approach combining Machine Learning (ML) techniques and Genetic Algorithms (GA) for predicting High-Performance Computing (HPC) job run times. The objective is to create a prediction method universally applicable to any HPC system, irrespective of workload characteristics, application specific parameters, user behavior, or hardware architecture. Since user-supplied run time estimates are often inaccurate, we aim to categorize job runtimes into several classes, allowing users to select appropriate classes for their jobs. A Genetic Algorithm is developed to optimally define these runtime classes, determining both the number of classes and the time intervals represented by them. Four Machine Learning algorithms (K Nearest Neighbours, Support Vector Regression, Extreme Gradient Boosting and Deep Neural Networks) are implemented for run time prediction. A unique set of features extracted from historical job data serves as input to the Machine Learning models. The generalized nature of our method is demonstrated by validating its performance on data from six clusters with distinct configurations, applications and runtime distributions. Our results illustrate the superior performance of Machine Learning models incorporating GA-defined runtime classes for all datasets. Across all six datasets, our method achieves R2 scores exceeding 0.8, and accuracy greater than 0.7.

Enhancing Security in Cloud-Based VM Migration: A Trust-Centric Hybrid Optimization Approach

Virtual Machine (VM) migration in cloud computing is essential to accomplishing various resource management goals, such as load balancing, power management, and resource sharing. Ensuring Quality of Service (QoS) is crucial while migrating Virtual Machines (VMs), which means that VMs must run continuously during the procedure. The dynamic nature of resource requirements in cloud computing, driven by "service-on-demand," poses security risks, even while live VM migration mechanisms help to maintain VM availability. This paper addresses changing resource requirements and improves overall system security by introducing a novel method for safe live virtual machine migration. In this paper, a robust optimization approach has been proposed for live virtual machine migration named the Gorilla-based Shuffled Shepherd Optimization Approach (GBSSOA). The proposed approach maximizes the number of fitness targets, such as migration time, QoS parameters, and job runtime. The method takes into account trust values in addition to the previously described performance measures, with a focus on security. Key performance indicators QoS as workload, migration time, trust, and GBSSOA-based secure live virtual machine migration are taken into account during the evaluation. The obtained values of 0.141, 0.654, 342.254 ms, and 0.569, respectively, show noteworthy accomplishments in the results. These results highlight how well the strategy developed may simultaneously improve performance metrics and strengthen the security of live virtual machine migration in dynamic cloud computing environments.

Enhancing Security in Cloud-Based VM Migration: A Trust-Centric Hybrid Optimization Approach

Virtual Machine (VM) migration in cloud computing is essential to accomplishing various resource management goals, such as load balancing, power management, and resource sharing. Ensuring Quality of Service (QoS) is crucial while migrating VMs, which means that VMs must run continuously during the procedure. The dynamic nature of resource requirements in cloud computing, driven by “service-on-demand,” poses security risks, even while live VM migration mechanisms help to maintain VM availability. This paper addresses changing resource requirements and improves overall system security by introducing a novel method for safe live VM migration. In this paper, a robust optimization approach has been proposed for live VM migration named the Gorilla-based Shuffled Shepherd Optimization Approach (GBSSOA). The proposed approach maximizes the number of fitness targets, such as migration time, QoS parameters, and job runtime. The method takes into account trust values in addition to the previously described performance measures, with a focus on security. Key performance indicators QoS as workload, migration time, trust, and GBSSOA-based secure live VM migration are taken into account during the evaluation. The obtained values of 0.141, 0.654, 342.254ms, and 0.569, respectively, show noteworthy accomplishments in the results. These results highlight how well the strategy developed may simultaneously improve performance metrics and strengthen the security of live VM migration in dynamic cloud computing environments.

SLearn: A Case for Task Sampling Based Learning for Cluster Job Scheduling

The ability to accurately estimate job runtime properties allows a scheduler to effectively schedule jobs. State-of-the-art online cluster job schedulers use history-based learning, which uses past job execution information to estimate the runtime properties of newly arrived jobs. However, with fast-paced development in cluster technology (in both hardware and software) and changing user inputs, job runtime properties can change over time, which lead to inaccurate predictions. In this paper, we explore the potential and limitation of real-time learning of job runtime properties, by proactively sampling and scheduling a small fraction of the tasks of each job. Such a task-sampling-based approach exploits the similarity among runtime properties of the tasks of the same job and is inherently immune to changing job behavior. Our analytical and experimental analysis of 3 production traces with different skew and job distribution shows that learning in space can be substantially more accurate. Our simulation and testbed evaluation on Azure of the two learning approaches anchored in a generic job scheduler using 3 production cluster job traces shows that despite its online overhead, learning in space reduces the average Job Completion Time (JCT) by 1.28×, 1.56×, and 1.32× compared to the prior-art history-based predictor.

Job runtime prediction of HPC cluster based on PC-Transformer

Job runtime prediction of HPC cluster based on PC-Transformer

Runtime Variation in Big Data Analytics

The dynamic nature of resource allocation and runtime conditions on Cloud can result in high variability in a job's runtime across multiple iterations, leading to a poor experience. Identifying the sources of such variation and being able to predict and adjust for them is crucial to cloud service providers to design reliable data processing pipelines, provision and allocate resources, adjust pricing services, meet SLOs and debug performance hazards. In this paper, we analyze the runtime variation of millions of production Scope jobs on Cosmos, an exabyte-scale internal analytics platform at Microsoft. We propose an innovative 2-step approach to predict job runtime distribution by characterizing typical distribution shapes combined with a classification model with an average accuracy of >96%, using an innovative interpretable machine-learning algorithm out-performing traditional regression models and better capturing long tails. We examine factors such as job plan characteristics and inputs, resource allocation, physical cluster heterogeneity and utilization, and scheduling policies. To the best of our knowledge, this is the first study on predicting categories of runtime distributions for enterprise analytics workloads at scale. Furthermore, we examine how our methods can be used to analyze what-if scenarios, focusing on the impact of resource allocation, scheduling, and physical cluster provisioning decisions on a job's runtime consistency and predictability.

Exploring job running path to predict runtime on multiple production supercomputers

There are massive jobs submitted in the supercomputer, and the job management system is typically deployed to schedule these jobs and allocate compute resources. FCFS (First Come First Serve) is a popular scheduling policy and the job's priority is determined based on the arrival time. However, under the FCFS strategy, if existing idle resources cannot meet the requirement of the head job in the waiting queue, they cannot be allocated to other jobs, which suffers from resource waste. To optimize the resource utilization, the backfilling method is proposed, which allocates the reserved idle compute nodes to a small-size and short-running non-head job, on the premise of not delaying the original head job. Obtaining the job's runtime in advance is necessary for backfilling and the traditional method relies on the user's estimation. Unfortunately, the estimated runtime provided by users is generally overestimated. Many studies extract features from historical job logs and adopt machine learning to predict the runtime. However, traditional features are insufficient to characterize the job. In this paper, we collect job logs from two supercomputers and present a novel runtime prediction framework called PREP. PREP explores the job's running path as a new feature, which implies plentiful information about the job's properties, such as the user, the project, the scale of data sets, and the parameters used. As there is a strong correlation between the job's runtime and its running path, we group jobs with similar paths into a cluster and train a runtime prediction model for each cluster respectively. Extensive evaluations demonstrate that introducing the new feature can achieve higher prediction accuracy (88.5% and 82.3% in two production supercomputers respectively), and our framework has a more desirable prediction performance than other popular strategies like Last-2 and IRPA. In addition, the predicted runtime is inserted into the real job trace of a slurm simulator to verify the advantages of PREP.

Innovation and Entrepreneurship of College Students Based on Random Matrix Big Data Analysis Model Educational Ecological Model Optimization

In this paper, the random matrix big data analysis model is thoroughly studied and constructed, the ecological model of college students’ innovation and entrepreneurship education is analyzed, and the optimization model of college students’ Innovation and entrepreneurship education environmental model based on the random matrix big data analysis model is designed. This paper briefly explains the random matrix and its M-P rate theory deduces the idea of feature extraction by the difference of eigenvalue limit spectrum distribution between different nonrandom matrices and random matrices, gives the data matrix representation method of FEMPL and the specific feature composition basis, and describes the steps of FEMPL feature extraction. A performance model for predicting the running time of Hadoop jobs is constructed using a random matrix. In this paper, innovation and entrepreneurship education has been carried out gradually, and the innovation and entrepreneurship education curriculum, platform, and mechanism have been progressively established. However, there is still a gap between the proper level of innovation and entrepreneurship education development. This study takes education ecology as the research perspective, analyzes the ecosystem of typical schools of innovation and entrepreneurship education, summarizes the dimensions and parameters of the invention and entrepreneurship education ecosystem, constructs an ecological model of innovation and entrepreneurship education for college students, and analyzes the problems and causes of the current innovation and entrepreneurship education ecology for college students based on the model, to propose specific strategies to promote the ecological development of innovation and entrepreneurship education for college students.

Collaborative Cluster Configuration for Distributed Data-Parallel Processing: A Research Overview

Many organizations routinely analyze large datasets using systems for distributed data-parallel processing and clusters of commodity resources. Yet, users need to configure adequate resources for their data processing jobs. This requires significant insights into expected job runtimes and scaling behavior, resource characteristics, input data distributions, and other factors. Unable to estimate performance accurately, users frequently overprovision resources for their jobs, leading to low resource utilization and high costs.In this paper, we present major building blocks towards a collaborative approach for optimization of data processing cluster configurations based on runtime data and performance models. We believe that runtime data can be shared and used for performance models across different execution contexts, significantly reducing the reliance on the recurrence of individual processing jobs or, else, dedicated job profiling. For this, we describe how the similarity of processing jobs and cluster infrastructures can be employed to combine suitable data points from local and global job executions into accurate performance models. Furthermore, we outline approaches to performance prediction via more context-aware and reusable models. Finally, we lay out how metrics from previous executions can be combined with runtime monitoring to effectively re-configure models and clusters dynamically.

AI4IO: A suite of AI-based tools for IO-aware scheduling

Traditional workload managers do not have the capacity to consider how IO contention can increase job runtime and even cause entire resource allocations to be wasted. Whether from bursts of IO demand or parallel file systems (PFS) performance degradation, IO contention must be identified and addressed to ensure maximum performance. In this paper, we present AI4IO (AI for IO), a suite of tools using AI methods to prevent and mitigate performance losses due to IO contention. AI4IO enables existing workload managers to become IO-aware. Currently, AI4IO consists of two tools: PRIONN and CanarIO. PRIONN predicts IO contention and empowers schedulers to prevent it. CanarIO mitigates the impact of IO contention when it does occur. We measure the effectiveness of AI4IO when integrated into Flux, a next-generation scheduler, for both small- and large-scale IO-intensive job workloads. Our results show that integrating AI4IO into Flux improves the workload makespan up to 6.4%, which can account for more than 18,000 node-h of saved resources per week on a production cluster in our large-scale workload.

Improving the performance of batch schedulers using online job runtime classification

Job scheduling in high-performance computing platforms is a hard problem that involves uncertainties on both the job arrival process and their execution times. Users typically provide only loose upper bounds for job execution times, which are not so useful for scheduling heuristics based on processing times. Previous studies focused on applying regression techniques to obtain better execution time estimates, which worked reasonably well and improved scheduling metrics. However, these approaches require a long period of training data.In this work, we propose a simpler approach by classifying jobs as small or large and prioritizing the execution of small jobs over large ones. Indeed, small jobs are the most impacted by queuing delays, but they typically represent a light load and incur a small burden on the other jobs. The classifier operates online and learns by using data collected over the previous weeks, facilitating its deployment and enabling a fast adaptation to changes in the workload characteristics.We evaluate our approach using four scheduling policies on seven HPC platform workload traces. We show that: first, incorporating such classification reduces the average bounded slowdown of jobs in all scenarios, second, in most considered scenarios, the improvements are comparable to the ideal hypothetical situation where the scheduler would know in advance the exact running time of jobs.

Learning-based Green Workload Placement for Energy Internet in Smart Cities

The Energy Internet is a fundamental infrastructure for deploying green city applications, where energy saving and job acceleration are two critical issues to address. In contrast to existing approaches that focus on static metrics with the assumption of complete prior knowledge of resource information, both application-level properties and energy-level requirements are realized in this paper by jointly considering energy saving and job acceleration during job runtime. Considering the online environment of smart city applications, the main objective is transferred as an optimization problem with a model partition and function assignment. To minimize the energy cost and job completion time together, a green workload placement approach is proposed by using the multi-action deep reinforcement learning method. Evaluations with real-world applications demonstrate the superiority of this method over state-of-the-art methods.

Influence of Job Runtime Prediction on Scheduling Quality

Influence of Job Runtime Prediction on Scheduling Quality

Predicting running time of aerodynamic jobs in HPC system by combining supervised and unsupervised learning method

Improving resource utilization is an important goal of high-performance computing systems of supercomputing centers. To meet this goal, the job scheduler of high-performance computing systems often uses backfilling scheduling to fill short-time jobs into job gaps at the front of the queue. Backfilling scheduling needs to obtain the running time of the job. In the past, the job running time is usually given by users and often far exceeded the actual running time of the job, which leads to inaccurate backfilling and a waste of computing resources. In particular, when the predicted job running time is lower than the actual time, the damage caused to the utilization of the system’s computing resources becomes more serious. Therefore, the prediction accuracy of the job running time is crucial to the utilization of system resources. The use of machine learning methods can make more accurate predictions of the job running time. Aiming at the parallel application of aerodynamics, we propose a job running time prediction framework SU combining supervised and unsupervised learning and verify it on the real historical data of the high-performance computing systems of China Aerodynamics Research and Development Center (CARDC). The experimental results show that SU has a high prediction accuracy (80.46%) and a low underestimation rate (24.85%).

Magnet

Over the past decade, Apache Spark has become a popular compute engine for large scale data processing. Similar to other compute engines based on the MapReduce compute paradigm, the shuffle operation, namely the all-to-all transfer of the intermediate data, plays an important role in Spark. At LinkedIn, with the rapid growth of the data size and scale of the Spark deployment, the shuffle operation is becoming a bottleneck of further scaling the infrastructure. This has led to overall job slowness and even failures for long running jobs. This not only impacts developer productivity for addressing such slowness and failures, but also results in high operational cost of infrastructure. In this work, we describe the main bottlenecks impacting shuffle scalability. We propose Magnet, a novel shuffle mechanism that can scale to handle petabytes of daily shuffled data and clusters with thousands of nodes. Magnet is designed to work with both on-prem and cloud-based cluster deployments. It addresses a key shuffle scalability bottleneck by merging fragmented intermediate shuffle data into large blocks. Magnet provides further improvements by co-locating merged blocks with the reduce tasks. Our benchmarks show that Magnet significantly improves shuffle performance independent of the underlying hardware. Magnet reduces the end-to-end runtime of Linkedln's production Spark jobs by nearly 30%. Furthermore, Magnet improves user productivity by removing the shuffle related tuning burden from users.

A gray-box modeling methodology for runtime prediction of Apache Spark jobs

Apache Spark jobs are often characterized by processing huge data sets and, therefore, require runtimes in the range of minutes to hours. Thus, being able to predict the runtime of such jobs would be useful not only to know when the job will finish, but also for scheduling purposes, to estimate monetary costs for cloud deployment, or to determine an appropriate cluster configuration, such as the number of nodes. However, predicting Spark job runtimes is much more challenging than for standard database queries: cluster configuration and parameters have a significant performance impact and jobs usually contain a lot of user-defined code making it difficult to estimate cardinalities and execution costs. In this paper, we present a gray-box modeling methodology for runtime prediction of Apache Spark jobs. Our approach comprises two steps: first, a white-box model for predicting the cardinalities of the input RDDs of each operator is built based on prior knowledge about the behavior and application parameters such as applied filters data, number of iterations, etc. In the second step, a black-box model for each task constructed by monitoring runtime metrics while varying allocated resources and input RDD cardinalities is used. We further show how to use this gray-box approach not only for predicting the runtime of a given job, but also as part of a decision model for reusing intermediate cached results of Spark jobs. Our methodology is validated with experimental evaluation showing a highly accurate prediction of the actual job runtime and a performance improvement if intermediate results can be reused.

A hybrid scheduling platform: a runtime prediction reliability aware scheduling platform to improve HPC scheduling performance

The performance of scheduling algorithms for HPC jobs highly depends on the accuracy of job runtime values. Prior research has established that neither user-provided runtimes nor system-generated runtime predictions are accurate. We propose a new scheduling platform that performs well in spite of runtime uncertainties. The key observation that we use for building our platform is the fact that two important classes of scheduling strategies (backfilling and plan based) differ in terms of sensitivity to runtime accuracy. We first confirm this observation by performing trace-based simulations to characterize the sensitivity of different scheduling strategies to job runtime accuracy. We then apply gradient boosting tree regression as a meta-learning approach to estimate the reliability of the system-generated job runtimes. The estimated prediction reliability of job runtimes is then used to choose a specific class of scheduling algorithm. Our hybrid scheduling platform uses a plan-based scheduling strategy for jobs with high expected runtime accuracy and backfills the remaining jobs on top of the planned jobs. While resource sharing is used to minimize fragmentation of resources, a specific ratio of CPU cores is reserved for backfilling of less predictable jobs to avoid starvation of these jobs. This ratio is adapted dynamically based on the resource requirement ratio of predictable jobs among recently submitted jobs. We perform extensive trace-driven simulations on real-world production traces to show that our hybrid scheduling platform outperforms both pure backfilling and pure plan-based scheduling algorithms.

An efficient attribute reduction algorithm using MapReduce

Classical attribute reduction algorithms based on attribute significance initiate too many jobs ( O(| C|2)) when they run in MapReduce. To improve the efficiencies of these algorithms, we proposed a novel reduction algorithm. Instead of focusing on attribute significance, the notion of a core attribute was applied to construct a new heuristic reduction algorithm, and only | C| jobs were considered to obtain a reduct. The algorithm only included two basic operations: compare and sort. The latter was optimised using the shuffle mechanism in MapReduce, which provided an efficient sorting ability for big data. In particular, we connected jobs in an iterative form to transfer the processing result of the former job to the latter job. Finally, experimental results demonstrated that the proposed attribute reduction algorithm was efficient and significantly improved upon the classical algorithms in runtime and number of jobs.

Towards Optimality in Parallel Job Scheduling

To keep pace with Moore's law, chip designers have focused on increasing the number of cores per chip. To effectively leverage these multi-core chips, one must decide how many cores to assign to each job. Given that jobs receive sublinear speedups from additional cores, there is a tradeoff: allocating more cores to an individual job reduces the job's runtime, but decreases the efficiency of the overall system. We ask how the system should assign cores to jobs so as to minimize the mean response time over a stream of incoming jobs. To answer this question, we develop an analytical model of jobs running on a multi-core machine. We prove that EQUI, a policy which continuously divides cores evenly across jobs, is optimal when all jobs follow a single speedup curve and have exponentially distributed sizes. We also consider a class of "fixed-width" policies, which choose a single level of parallelization, k, to use for all jobs. We prove that, surprisingly, fixed-width policies which use the optimal fixed level of parallelization, k*, become near-optimal as the number of cores becomes large. In the case where jobs may follow different speedup curves, finding a good scheduling policy is even more challenging. In particular, EQUI is no longer optimal, but a very simple policy, GREEDY*, performs well empirically.

Towards Bandwidth Guarantee for Virtual Clusters Under Demand Uncertainty in Multi-Tenant Clouds

In the cloud, multiple tenants share the resource of datacenters and their applications compete with each other for scarce network bandwidth. Current studies have shown that the lack of bandwidth guarantee causes unpredictable network performance, leading to poor application performance. To address this issue, several virtual network abstractions have been proposed which allow the tenants to reserve virtual clusters with specified bandwidth between the Virtual Machines (VMs) in the datacenters. However, all these existing proposals require the tenants to deterministically characterize the bandwidth demands in the abstractions, which can be difficult and result in inefficient bandwidth reservation due to the demand uncertainty. In this paper, we explore a virtual cluster abstraction with stochastic bandwidth characterization to address the bandwidth demand uncertainty. We propose Stochastic Virtual Cluster (SVC), which models the bandwidth demand between VMs in a probabilistic way. Based on SVC, we develop a stochastic framework for virtual cluster allocation, in which the admitted virtual cluster's bandwidth demands are satisfied with a high probability. Efficient VM allocation algorithms are proposed to implement the framework while reducing the possibility of link congestion through minimizing the maximum bandwidth occupancy of a virtual cluster on physical links. Using simulations, we show that SVC achieves the trade-off between the job concurrency and the average job running time, and demonstrate its effectiveness for accommodating cloud application workloads with highly volatile bandwidth demands and its improvement to work-conserving bandwidth enforcement.