Job Completion Research Articles

Forseti: Dynamic chunk-level reshaping for data processing on heterogeneous clusters

Data-intensive computing frameworks typically split job workload into fixed-size chunks, allowing them to be processed as parallel tasks on distributed machines. Ideally, when the machines are homogeneous and have identical speed, chunks of equal size would finish processing at the same time. However, such determinism in processing time cannot be guaranteed in practice. Diverging processing times can result from various sources such as system dynamics, machine heterogeneity, and variable network conditions. Such variation, together with dynamics and uncertainty during task processing, can lead to significant performance degradation at job level, due to long tails in job completion time resulted from residual chunk workload and stragglers.In this paper, we propose Forseti, a novel processing scheme that is able to reshape data chunk size on the fly with respect to heterogeneous machines and a dynamic execution environment. Forseti mitigates residual workload and stragglers to achieve significant improvement in performance. We note that Forseti is a fully online scheme and does not require any a priori knowledge of the machine configuration nor job statistics. Instead, it infers such information and adjusts data chunk sizes at runtime, making the solution robust even in environments with high volatility. In its implementation, Forseti also exploits a virtual machine reuse feature to avoid task start-up and initialization cost associated with launching new tasks. We prototype Forseti on a real-world cluster and evaluate its performance using several realistic benchmarks. The results show that Forseti outperforms a number of baselines, including default Hadoop by up to 68% and SkewTune by up to 50% in terms of average job completion time.

Efficient Scheduling of Jobs and Allocation of Resources in Cloud Computing

Due to the drastic utilization of clouds, a Proper and proficient allocation of resources in dynamically working environment of cloud systems turns into the challenging task. Different promising mechanisms have been created to work on the effectiveness of process of resource allocation. Yet at the same time there is some incompetency as far as resource allocation and job scheduling, when the systems become highly loaded. Hence, an effective algorithm for scheduling of jobs is needed to work on the proficiency of resource allocation activities. In this paper a advanced technique for scheduling of jobs is proposed for effective and unique process of allocation of resources in cloud computing. By making use of prediction-based techniques and mechanism of updating resource tables in dynamic manner, we achieve, better allocation of resources in the form of response time and completion of jobs. The experimental results demonstrate the effective outcomes compared to existing techniques, by achieving exactness in values for resource table updation.

Differences in the Physical Workload of Technicians and Managers in the Maintenance and Repair Division

Introduction: PT. PAL Indonesia (Persero) is a state-owned company in the shipping industry. For running the production cycle, PT. PAL Indonesia (Persero) has several divisions. One of them is the Maintenance and Repair Division. This division involves workers with positions as technicians and managers who have different tasks and functions. These differences will result in unequal physical workloads. The purpose of this study is to analyze the differences in the physical workload of technicians and managers of the Maintenance and Repair Division of PT. PAL Indonesia (Persero). Methods: This study used an analytic observational method with a cross sectional design. The independent variable used in this research was the position of workers consisting of managers and technician staff. Meanwhile, the dependent variable was a physical workload. The research respondents were 24 technicians and 6 managers. Data were obtained through questionnaires and calorimeters to measure calorie needs. Results: The technicians were aged 20 - 33 years old (62.5%), and the managers were aged 48 - 61 years (83.3%). Technicians' nutritional status was categorized as normal, as many as 21 people and the nutritional status of managers was overweight, as many as 4 people. The physical workload of technicians was in the medium category, as many as 22 people and 5 managers had a moderate physical workload. Conclusion: There was no difference in the physical workload of technicians and managers of the Maintenance and Repair Division. This is because PT. PAL Indonesia (Persero) has implemented cooperation in job completion.

Preemptive Scheduling for Distributed Machine Learning Jobs in Edge-Cloud Networks

Recent advances in 5G and edge computing enable rapid development and deployment of edge-cloud systems, which are ideal for delay-sensitive machine learning (ML) applications such as autonomous driving and smart city. Distributed ML jobs often need to train a large model with enormous datasets, which can only be handled by deploying a distributed set of workers in an edge-cloud system. One common approach is to employ a parameter server (PS) architecture, in which training is carried out at multiple workers, while PSs are used for aggregation and model updates. In this architecture, one of the fundamental challenges is how to dispatch ML jobs to workers and PSs such that the average job completion time (JCT) can be minimized. In this work, we propose a novel online preemptive scheduling framework to decide the location and the execution time window of concurrent workers and PSs upon each job arrival. Specifically, our proposed scheduling framework consists of: i) a job dispatching and scheduling algorithm that assigns each ML job to workers and decides the schedule to train each data chunk; ii) a PS assignment algorithm that determines the placement of PS. We prove theoretically that our proposed algorithm is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D_{max}(1+1/\epsilon)$ </tex-math></inline-formula> -competitive with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(1 + \epsilon)$ </tex-math></inline-formula> -speed augmentation, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D_{max}$ </tex-math></inline-formula> is the maximal number of data chunks in any job. Extensive testbed experiments and trace-driven simulations show that our algorithm can reduce the average JCT by up to 30% compared with state-of-the-art baselines.

Co-Scheduler: A Coflow-Aware Data-Parallel Job Scheduler in Hybrid Electrical/Optical Datacenter Networks

To support higher demand for datacenter networks, networking researchers have proposed hybrid electrical/optical datacenter networks (Hybrid-DCN) that leverages optical circuit switching (OCS) along with traditional electrical packet switching (EPS). However, due to the high <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">reconfiguration</i> delay of OCS, OCS is used only for bulk data transfers between racks to amortize the reconfiguration delay. Existing job schedulers for data-parallel frameworks are not designed for Hybrid-DCN, since they neither place tasks to aggregate data traffic to take advantage of OCS, nor schedule tasks to minimize the Coflow completion time (CCT). In this paper, we describe the mismatch between existing job schedulers and the advanced Hybrid-DCN, introduce the requirements for the new scheduler, and present the implementation of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Co-scheduler</i> , a job scheduler for data-parallel frameworks that aims to improve job performance by placing the tasks of jobs to aggregate enough data traffic to better leverage OCS to minimize the CCT in Hybrid-DCN. Specifically, for every job, Co-scheduler computes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">guidelines</i> on how many racks to place the job’s input data and the job’s tasks. The guidelines are dynamically generated based on the real-time job characteristics or predictable job characteristics from prior runs, with the aim of leveraging OCS whenever possible and efficient and minimizing CCT of jobs. Co-scheduler then schedules the tasks of jobs based on the guidelines. We evaluate the effectiveness of Co-scheduler using trace-driven simulation. The evaluation demonstrates that Co-scheduler can improve makespan, average job completion time, and average CCT of a workload by up to 56%, 61%, and 79%, respectively, compared to the state-of-the-art schedulers.

Research on multi-core scheduling method for Sporadic task by line tree mode

At present, most of the scheduling models for real-time periodic tasks are established based on independent fixed periodic tasks and few by considering the task model with period allowed to change and the processor model in the scheduling process. In this paper, a task model and processor model based on line tree(LT) for sporadic real-time periodic tasks are designed, and a transformation algorithm from task line tree(TLT) model to processor line tree(PLT) model is proposed. The algorithm takes the least common multiple of all real-time periodic tasks as the benchmark of layer number, and based on the complete job replacement rule, the minimum common multiple of all real-time periodic tasks is used as the reference of layer number. In order to achieve the optimal multiprocessor scheduling result, the method of replacing the empty node of non-migrating job with the node of migrating job and the condition of allowing job to publish tasks ahead of time with variable cycle are used. Experimental results show that comparing with PEDF, GEDF and RMFF, the present method not only has higher core utilization and lower time loss rate, but also reduces the number of context switching and migration.

Standalone Chatbot Application in Python

Abstract: A chatbot is an artificial intelligence (AI) software that can simulate a conversation (or a chat) with a user in natural language through messaging applications, websites, mobile apps or through the telephone. This software is used to do duties such as replying swiftly to users, informing them, assisting with product purchases, and delivering better customer support. Business groups are increasingly using chatbots because they may minimize customer support costs and handle several consumers at once. However, in order to complete various jobs, chatbots must be as efficient as possible. In this project, we provide the architecture of a chatbot, which provides a human like and accurate response for any questions raised by users using Natural Language ToolKit(NLTK) and PyTorch with python language.

The Impact of Interface Design Element Features on Task Performance in Older Adults: Evidence from Eye-Tracking and EEG Signals.

It is crucial that the interface design of mobile apps be age-appropriate at this stage of global aging, as the new epidemic has resulted in a higher sense of isolation among older persons. In this study, four typical senior social service mobile applications were chosen to give older persons the ability to complete user login duties. The participants were 16 older adults (7 men and 9 women) aged 55 to 76. Both objective and subjective data, including task completion time, gaze length, pupil diameter changes, EEG wave amplitude changes, and subjective sensations of older persons, were gathered using a combination of eye-movement and EEG signal approaches. The program was created to investigate the effects of interface design aspects on older people’s task performance, including interface layout, interface color, information density, icon size and position, etc. The study’s findings revealed that when the user task completion time and average fixation duration were shorter, the line of sight was more equally distributed, the visual focus was closer to the login button, and the average EEG amplitude of the user changed more, the older adults performed better. The palace layout had a more positive effect on job completion among older individuals when it came to interface layout. In terms of interface color, colored (contrasting) colors should serve to highlight the interface’s essential information points while they can be removed. In terms of interface information density, a low-density level interface design can simplify and lower the cognitive load of task execution for older people. The first level of icons in the interface and their position in the visual center of the interface is the best interface design for older persons in terms of icon size and position. The results of this study have theoretical ramifications for a thorough understanding of the factors influencing older people’s task performance, practical ramifications for the design of older people-centered interfaces, and they contribute to our understanding of the characteristics of older people’s interface interaction behavior.

Qore-DL: A QoS-aware joint optimization framework for distributed deep learning training

Resource management of deep learning training (DLT) jobs is critical for cluster resource efficiency and client QoS assurance. Most existing scheduling frameworks require clients to specify job resource configuration, which can lead to over-provision or under-provision issues. Additionally, the performance of some static scheduling frameworks degrades in highly dynamic clusters. In this paper, we propose a QoS-aware joint resource optimization framework called Qore-DL for distributed DLT jobs. We divide the lifecycle of a DLT job into submission, queuing and running stages. Qore-DL automatically configures reasonable resources for submitted jobs and greedily assigns scheduled jobs to hosts. For running jobs, Qore-DL employs a heuristic scheme to adjust their resources. Qore-DL jointly considers the optimization of QoS satisfaction and resource efficiency at the three stages of DLT jobs. We implemented the prototype of Qore-DL in TensorFlow based on Kubernetes and conducted extensive experiments in CPU and GPU clusters to evaluate its performance. The experiment results show that, compared with its counterparts, Qore-DL can improve the job completion rate by up to 42.4% and the cluster resource efficiency by up to 21.8%.

SMART: Speedup Job Completion Time by Scheduling Reduce Tasks

SMART: Speedup Job Completion Time by Scheduling Reduce Tasks

Scheduling on uniform machines with a conflict graph: complexity and resolution

AbstractThis paper deals with the problem of scheduling a set of unit‐time jobs on a set of uniform machines. The jobs are subject to conflict constraints modeled by a graph G called the conflict graph, in which adjacent jobs cannot be processed on a same machine. The objective considered herein is the minimization of maximum job completion time in the schedule, which is famous to be NP‐hard in the strong sense. The first part of this paper is an extensive study of the computational complexity of the problem restricted to several graph classes, namely: split graphs, interval graphs, forests, trees, paths and cycles. Afterward, we focus on the resolution of the problem with arbitrary conflict graphs. For this latter, a combination of a mixed integer linear programming (MILP) formulation, lower and upper bounds is proposed. A wild range of computational experiments proved the efficiency of this technique to tremendously reduce runtime and produce more optimal solutions (around 80% in average). Furthermore, a deep analysis of the resolution process based on both the density of the conflict graph as well as machine speeds (including identical machines) is thoroughly reported.

Load balancing technique based on hybrid resource utilization in cloud computing

Cloud computing uses the internet to supply dynamic services including memory, data, bandwidth and applications. Work schedules have an influence on cloud service reliability and performance. A proper provisioning method is required for a systematic resource allocation, which comprises of large virtual resources. Depending on the present state of the system, load balancing solutions can be distinguished as dynamic or static. Dynamic or static load balancing solutions can be employed to increase server response time or to raise load balancing factors for quicker and more efficient resource utilization. To decrease the load across resources and maximize CPU usage, a hybrid load balancing technique is developed. In the cloud, we have a finite quantity of resources that must be efficiently managed in order to fulfill tasks. Requests are transmitted to a cloud server, which assigns work via quadratic probing. During load balancing, the load is shifted from heavy-weighted servers to lighter-weighted servers, enhancing CPU usage. The suggested methodology's performance was assessed using average mean response time, make-span, average make-span, and average resource utilization. The Load Balancing Algorithm (LBA) is created with the primary purpose of reducing job completion time and increasing the average resource utilization ratio.

Joint optimization of production and maintenance for a serial–parallel hybrid two-stage production system

Effective machine maintenance and job scheduling strategies are two critical interactive tasks to upgrade the production efficiency in manufacturing industries. In this study, we mainly consider the interaction between production scheduling and maintenance, while optimizing the job scheduling problem in a serial–parallel hybrid two-stage production system. We formulate the problem by introducing the stochastic and workload-ratio degradation, and the actual job completion time and maintenance costs can be modeled according to the degradation status of each machine. To solve this problem, a two-step opportunistic maintenance (OM) strategy is developed, and finally an adaptive random-key genetic algorithm (GA) is designed by incorporating the OM coefficients. Numerical studies have been conducted to validate the proposed approach, and the results indicate the effectiveness compared to three existing methods under varying machine degradation scenarios.

Balanced Nonadaptive Redundancy Scheduling

Distributed computing systems implement redundancy to reduce the job completion time and variability. Despite a large body of work about computing redundancy, the analytical performance evaluation of redundancy techniques in queuing systems is still an open problem. In this work, we take one step forward to analyze the performance of scheduling policies in systems with redundancy. In particular, we study the pattern of shared servers among replicas of different jobs. To this end, we employ combinatorics and graph theory and define and derive performance indicators using the statistics of the overlaps. We consider two classical nonadaptive scheduling policies: random and round-robin. We then propose a scheduling policy based on combinatorial block designs. Compared with conventional scheduling, the proposed scheduling improves the performance indicators. We study the expansion property of the graphs associated with round-robin and block design-based policies. It turns out the superior performance of the block design-based policy results from better expansion properties of its associated graph. As indicated by the performance indicators, the simulation results show that the block design-based policy outperforms random and round-robin scheduling in different scenarios. Specifically, it reduces the average waiting time in the queue to up to 25% compared to the random policy and up to 100% compared to the round-robin policy.

An Optimization Method for Enterprise Resource Integration Based on Improved Particle Swarm Optimization.

An enterprise's development and growth are inextricably linked to rational and efficient resource integration and optimization. This study focuses on the reorganization and integration of industrial elements inside the firm from the standpoint of resource integration. The ideal resource integration strategy is investigated by integrating the industrial parts of a certain enterprise in order to increase the efficiency of project completion and lower enterprise expenses. The enterprise's internal material and human resources are limited, but it is frequently necessary to execute numerous activities at the same time, and each activity must meet multiple goals. This research investigates how to properly integrate and schedule resources while attaining different goals. This research proposes using an enhanced particle swarm optimization technique (IPSO) to combine firms' internal resources. In order to address the issue of uneven particle dispersion caused by random population initialization, IPSO incorporates chaos theory into particle population initialization. The logistic mapping sequence generates a huge number of particles, and the particles with the highest quality are chosen for initialization. This can increase particle quality, allowing particles to be spread equally during setup. In the late stage, the classic particle swarm optimization algorithm (PSO) has a slow convergence rate, causing the algorithm to readily slip into a local optimal solution. This research proposes a dynamic inertia weight update approach based on fitness value. In the later stages of the algorithm, this strategy can improve the convergence speed and quality of the global optimal solution, allowing the particles to do a global search and eventually identify the population's ideal solution. Furthermore, IPSO creates a fitness function depending on task completion time. IPSO is used to test the performance of an enterprise's resource integration case. Experiments show that the method utilized can swiftly locate the ideal solution, complete the integration, and optimization of enterprise resources in the shortest job completion time, and for the least amount of money.

A Survey on Malleability Solutions for High-Performance Distributed Computing

Maintaining a high rate of productivity, in terms of completed jobs per unit of time, in High-Performance Computing (HPC) facilities is a cornerstone in the next generation of exascale supercomputers. Process malleability is presented as a straightforward mechanism to address that issue. Nowadays, the vast majority of HPC facilities are intended for distributed-memory applications based on the Message Passing (MP) paradigm. For this reason, many efforts are based on the Message Passing Interface (MPI), the de facto standard programming model. Malleability aims to rescale executions on-the-fly, in other words, reconfigure the number and layout of processes in running applications. Process malleability involves resources reallocation within the HPC system, handling processes of the application, and redistributing data among those processes to resume the execution. This manuscript compiles how different frameworks address process malleability, their main features, their integration in resource management systems, and how they may be used in user codes. This paper is a detailed state-of-the-art devised as an entry point for researchers who are interested in process malleability.

Energy efficient workflow scheduling for fog-enhanced IOT based healthcare application

The Internet of Things has generated a tremendous amount of data. These files must be handled quickly and are typically kept in a cloud data center. Cloud computing can provide abundant resources to process such information IoT devices relating to data security, energy consumption, and data processing are driving the growth and interplay of fog computing and cloud computing. The problem of resource scheduling in a system designed to reduce task execution time and overall energy consumption. To partition the graph and determine task processing permutations and processor assignment, the tri EDA and division operator are employed. The comparison findings of a single application simulation reveal that the Pareto set produced by the proposed methodology can dominate a considerable number of those solutions produced by the heuristic technique and straightforward EDA. When it comes to multi-objective simulation, IoT devices can last much longer and perform similarly to other algorithms in terms of nodes' energy consumption and job completion time.

Work Stressors and Occupational Health of Young Employees: The Moderating Role of Work Adaptability.

Work adaptability refers to the work experience, habits, and skills that enable an individual to adapt to current or changing work tasks and situations. It is a coping resource that individuals use to mitigate various types of stress. Adopting the interaction model of work stress, this study investigated 168 young employees in 20 organizations in Zhejiang Province through interview research and a questionnaire survey. The results show that work adaptability has a significant main effect on occupational health. The work adaptability of employees plays a moderating role in the relationship between occupational health and lack of work meaning stress, role conflict stress, interpersonal relationship stress, negative organizational atmosphere stress, and total score of work stressors. Young employees with high work adaptability have worse occupational health under high-level stress situations due to a lack of work meaning. For promoting occupational health in young employees, organizations should have this group of workers complete meaningful jobs or inform them of the importance of their jobs, reduce role conflict, and create a supportive organizational atmosphere. For management, it is imperative to eliminate high-level stress that stems from a lack of work meaning in order to retain young employees with high work adaptability. These findings shed light on how work adaptability helps young employees deal with stress and improve their occupational health. In organizational and self-stress management, it is beneficial to improve employees’ work adaptability continuously as a means of effectively resisting stress and maintaining occupational health.

PENINGKATAN HASIL BELAJAR SISWA DENGAN METODE PEMBERIAN LATIHAN PRAKTEK MANDIRI TERBIMBING PADA MATA PELAJARAN INSTALASI PENERANGAN LISTRIK DI KELAS XI TITL 1 SMK NEGERI 1 MANDAU SEMESTER 1 TAHUN PELAJARAN 2019 / 2020

This research begins with students who cannot do practical job sheets and only copy and copy their friends' work, the authors see that students need guidance / direction so that they are willing to work and be active in PHB assembly practice activities and are able to complete practical job sheets. well and perfectly. The research that the writer did was Classroom Action Research (CAR). The subjects in this action research were students of Class XI Electrical Power Installation Engineering (TITL 1) at SMK Negeri 1 Mandau with 11 students. The data collection technique used in this research is a series of photo folio assessments to evaluate the quality and quantity of the self-guided training method carried out by researchers collaborating with an observer. Data analysis was carried out qualitatively. Based on the results of observations and student learning outcomes in each cycle, it is known that: (1) Providing Guided Independent Practice Exercises is proven to improve student learning outcomes in the subject of Electrical Lighting Installation in Class XI TITL SMK Negeri 1 Mandau, amounting to 96.29% (2) Carrying out practical competency exams (UKP) with not too many test materials can spur students to get good grades of 96%. It can be concluded that the implementation of research using learning with this training method can improve learning outcomes.Keywords: Method of Giving Guided Independent Practice Exercises, Electrical Lighting Installation Subjects.

PAS: Performance-Aware Job Scheduling for Big Data Processing Systems

Big data analytics has become increasingly vital in many modern enterprise applications such as user profiling and business process optimization. Today’s big data processing systems, such as Hadoop MapReduce, Spark, and Hive, treat big data applications as a batch of jobs for scheduling. Existing schedulers in production systems often maintain fair allocation without considering application performance and resource utilization simultaneously. It is challenging to perform job scheduling in big data systems to achieve both low turnaround time and high resource utilization due to the high complexity in data processing logics and the dynamic variation in workloads. In this article, we propose a performance-aware scheduler, referred to as PAS, which dynamically schedules big data jobs in Hadoop YARN and Spark and autonomously adjusts scheduling policies to improve application performance and resource utilization. Specifically, PAS schedules multiple concurrent jobs using different policies based on the predicted job completion time and employs a greedy approach and a one-step lookahead strategy to opportunistically maximize the average job performance while still maintaining a satisfactory level of resource utilization. We implement PAS in Hadoop YARN and evaluate its performance with HiBench, a well-known big data processing benchmark. Experimental results show that PAS reduces the average turnaround time by 25% and the makespan by 15% in comparison with four state-of-the-art schedulers.