Parallel And Distributed Computing Research Articles

MCDS: AI Augmented Workflow Scheduling in Mobile Edge Cloud Computing Systems

Workflow scheduling is a long-studied problem in parallel and distributed computing (PDC), aiming to efficiently utilize compute resources to meet user's service requirements. Recently proposed scheduling methods leverage the low response times of edge computing platforms to optimize application Quality of Service (QoS). However, scheduling workflow applications in mobile edge-cloud systems is challenging due to computational heterogeneity, changing latencies of mobile devices and the volatile nature of workload resource requirements. To overcome these difficulties, it is essential, but at the same time challenging, to develop a long-sighted optimization scheme that efficiently models the QoS objectives. In this work, we propose MCDS: Monte Carlo Learning using Deep Surrogate Models to efficiently schedule workflow applications in mobile edge-cloud computing systems. MCDS is an Artificial Intelligence (AI) based scheduling approach that uses a tree-based search strategy and a deep neural network-based surrogate model to estimate the long-term QoS impact of immediate actions for robust optimization of scheduling decisions. Experiments on physical and simulated edge-cloud testbeds show that MCDS can improve over the state-of-the-art methods in terms of energy consumption, response time, SLA violations and cost by at least 6.13, 4.56, 45.09 and 30.71 percent respectively.

Evolving PDC curriculum and tools: A study in responding to technological change

Much has changed about parallel and distributed computing (PDC) since the author began teaching the topic in the late 1990s. This paper reviews some of the key changes to the field and describes their impacts on his work as a PDC educator. Such changes include: the availability of free implementations of the message passing interface (MPI) for distributed-memory multiprocessors; the development of the Beowulf cluster; the advent of multicore architectures; the development of free multithreading languages and libraries such as OpenMP; the availability of (relatively) inexpensive manycore accelerator devices (e.g., GPUs); the availability of free software platforms like CUDA, OpenACC, OpenCL, and OpenMP for using accelerators; the development of inexpensive single board computers (SBCs) like the Raspberry Pi, and other changes. The paper details the evolution of PDC education at the author's institution in response to these changes, including curriculum changes, seven different Beowulf cluster designs, and the development of pedagogical tools and techniques specifically for PDC education. The paper also surveys many of the hardware and software infrastructure options available to PDC educators, provides a strategy for choosing among them, and provides practical advice for PDC pedagogy. Through these discussions, the reader may see how much PDC education has changed over the past two decades, identify some areas of PDC that have remained stable during this same time period, and so gain new insight into how to efficiently invest one's time as a PDC educator.

Visual analogy videos for understanding fundamental parallel scheduling policies

Parallel and distributed computing (PDC) education is increasingly gaining greater recognition as a core topic in undergraduate computing degrees. While the application of PDC concepts to software development involves the use of highly-technical tools and libraries typically reserved for advanced courses, PDC educators are seeking pedagogical approaches that can be used to introduce PDC concepts in earlier, introductory courses. This study presents such an approach, and aims to introduce undergraduate students to fundamental PDC concepts without the expectation that they can apply those concepts. The proposed approach is inspired by the success seen in the wider computing education literature, where analogies and visualization have helped students understand other abstract computing topics. The proposed learning resources come in the form of a series of short videos, carefully aligned to a learning activity that guides towards achieving the intended learning outcomes. In addition to being a simple activity to complete with students, evaluations illustrate its value even with minimal guidance from the instructor. The proposed approach is studied as both a synchronous in-class activity guided by the instructor, as well as an asynchronous online self-directed activity. These two studies produced different outcomes with respect to student learning, revealing an important implication for designers of instructional material to consider.

A novel framework for policy based on-chain governance of blockchain networks

Although blockchains are systems that are intended to survive on their own, they still may need human intervention like other Parallel and Distributed Computing (PDC) architectures. The reasons for this intervention are twofold: management of blockchain nodes and governance of blockchain. The second focuses on sustaining blockchain unlike the first one concerning with operation of individual nodes. Proposed blockchain governance models so far include cumbersome processes in which deciding and deployment of changes rely on the involvement of multiple actors. They have been criticized from various aspects such as reducing the decentralized nature of blockchain. In this paper, we propose a novel on-chain governance model exploiting both policy-based management and decentralized identity technologies. Since it allows controlling the behavior of the system via higher level expressions that are easy to understand, the policy based management concept will fit in well with blockchain ecosystems having stakeholders from different knowledge levels. This model is not only more effective regarding existing off-chain governance models but also is simpler and more user friendly than other on-chain governance models. Besides, the emerging decentralized identity concept has the potential for solving many problems related to decision-making process of blockchain governance. By employing this concept, our approach fortifies decentralization, security and fairness while providing well-directed governance. The smart contract based modular design of our framework opens the door for adoption of its components by other governance models as well.

Shortest-path routing for optimal all-to-all personalized-exchange embedding on hierarchical hypercube networks

Hypercube (HC) networks (N=2n) provide efficient communication for parallel-and-distributed computing (PDC) but the HC-based multi-processor (MP) system is costly and not scalable, while hierarchical hypercube (HHC) networks (N=2n, n=2m+m) are less expensive and more scalable. However, the traditional HHC-routing easily conflicts, especially when executing multiple tasks (k > 2m nodes-per-task). In the past, the node-disjoint-path routing could be used to avoid the conflict but that reliable (S, D) routing limited S(source)=0. Later, the parallel N2N (node-to-node) disjoint-path was proposed for reliable routing but limited (m+1)/2 pairs-of-nodes. Therefore, this study proposes the generalized N2N shortest-path (SP) routing (with arbitrary S) and the parallel N2N SP-routing, based on our hypothesis “the shortest-path routing and the proper HHC-partitioning can avoid the HHC-conflict directly”. Next, our innovation and contribution are 1. the GCD (grouping of cross dual-cube) partitioning to solve the HHC-conflict for k≤2m+1 nodes-per-task, and 2. the SP-ATAPE (all-to-all personalized exchange) embedding on the HHC-MPs. The correctness of the SP-routing was proven and the ATAPE communication was experimented to validate our conflict solution. The ATAPE results confirmed that in any group the GCD mapping could make all tasks (k= 2m+1 nodes-per-task), synchronized with the same control, working without the conflict.

MCDS: AI Augmented Workflow Scheduling in Mobile Edge Cloud Computing Systems

Workflow scheduling is a long-studied problem in parallel and distributed computing (PDC), aiming at efficiently utilizing compute resources to meet user's service requirements. Recently proposed scheduling methods leverage the low response times of edge computing platforms to optimize application Quality of Service (QoS). However, scheduling workflow applications in mobile edge-cloud systems is challenging due to computational heterogeneity, changing latencies of mobile devices and the volatile nature of workload resource requirements. To overcome these difficulties it is important, but at the same time challenging, to develop a long-sighted optimization scheme with efficient modelling of the QoS objectives. In this work, we propose MCDS: Monte Carlo Learning using Deep Surrogate Models to efficiently schedule workflow applications in mobile edge-cloud computing systems. MCDS is an Artificial Intelligence (AI) based scheduling approach that uses a tree-based search strategy and a deep neural network based surrogate model to estimate the long-term QoS impact of immediate actions for robust optimization of scheduling decisions. Experiments on physical and simulated edge-cloud testbeds show that MCDS can improve over the state-of-the-art methods in terms of energy consumption, response time, SLA violations and cost by at least 6.13, 4.56, 45.09 and 30.71 percent respectively.

Preparing the software engineer for a modern multi-core world

Parallel and Distributed Computing (PDC) was traditionally viewed as an advanced subject reserved for elective graduate courses. The last decade has seen two areas with rapid growth, whose synergy is demanding new skills for software engineers in a modern multi-core world. The first has been society’s increasing demand for software engineering solutions, evident in the integral role that software plays in daily life. Unlike traditional PDC applications in the scientific and engineering domains, modern software applications are interacting directly with millions of users on mainstream laptops, smartphones and tablets. The second trend is that of multi-core processors powering such devices, which is further fueling the potential of software applications. This paper proposes a Modern Parallel Programming Framework that recognizes that successful software engineering in this domain involves a combination of hard skills and soft skills. A course dedicated to this goal is presented and evaluated, incorporating a research-infused, problem-based and active learning approach.

Building web-based services for practical exercises in parallel and distributed computing

The paper presents an approach to the design and implementation of web-based environments for practical exercises in parallel and distributed computing (PDC). The presented approach introduces minimal development and operational costs by relying on Everest, a general-purpose platform for building computational web services. The flexibility of proposed service-oriented architecture enables the development of different types of services targeting various use cases and PDC topics. The generic execution services support the execution of different types of parallel and distributed programs on corresponding computing systems, while the assignment evaluation services implement the execution and evaluation of solutions to programming assignments. As was demonstrated by teaching two introductory PDC courses, the presented approach helps to enhance students’ practical experience while avoiding low-level interfaces, reducing the grading time and providing a level of automation necessary for scaling the course to a large number of students. In contrast to other efforts, the exploited Platform as a Service model provides the ability to quickly reuse this approach by other PDC educators without installation of the Everest platform.

Unifying computing resources and access interface to support parallel and distributed computing education

This article presents how various on-site and remote computing resources are combined into a framework to support teaching parallel and distributed computing (PDC) at the undergraduate level. The combination of these resources enables the delivery of PDC programming, system, and architectural concepts via a browser-based common interface (JupyterHub) and a single programming environment (Python and its supported libraries). This also allows lecturers and students to focus more on the principles of PDC and less on the technicalities of native languages for different platforms. We describe how this framework can support a comprehensive set of PDC course modules, including lectures, assignments, and projects, for a full semester junior-level class. Adoption of this framework in various teaching environments at Clemson University has received positive feedback from both instructors and participants.

Scalable Distributed Semantic Network for knowledge management in cyber physical system

The remarkable growth of emerging technologies and computing paradigms in cyberspace and the cyber physical systems generate a huge mass of data sources. These different autonomous and heterogeneous data sources can contain complementary or semantically equivalent information stored under different formats that vary from structured, semi structured, to unstructured. These heterogeneities influence on data semantics and meaning. Therefore, knowledge management became more and more difficult and sometimes fruitless. In this paper, we propose a new scalable model, named Distributed Semantic Network (DSN), for heterogeneous data representation and can extract more semantic information from different data sources. We use the prior knowledge of WordNet and Wikipedia to scale out DSN horizontally and vertically. Furthermore, we proposed a MapReduce based framework to construct the knowledge base more effectively in Parallel and Distributed Computing (PDC). The experimental results show that DSN can better model the semantic information in the text. It can extract a larger amount of information from the text with a higher precision, achieving 34% increase in quantity and 15% promotion on precision than the best-performing alternative method on same datasets. On the three datasets, our proposed PDC framework shorten the process time by 5.8–11.5 times.

Research-oriented teaching of PDC topics in integration with other undergraduate courses at multiple levels: A multi-year report

Parallel and distributed computing (PDC) is finding its usage from system, algorithms, and architecture perspectives in research and industries of many domains. Due to its ever-increasing applications and benefits, the need of skilled manpower in the area of PDC is also increasing. It is felt that if the basic knowledge taught through the core course of PDC is supplemented with the discussion of PDC topics in integration with other computer science courses, then it will not only provide the students with more opportunities to ‘think in parallel’, but will also motivate them to harness the best of PDC. In this paper, we present our experiences of performing research-oriented teaching of PDC topics in integration with other undergraduate courses since 2014 to 2016 spread over multiple semesters. The courses mainly include Software Engineering, Computer Networks, Computer Architecture, Network Programming, and Network based Laboratory taught to undergraduate level students of Computer Science and Engineering. Our integration plan is encouraged from the objectives of NSF/TCPP–IEEE core curriculum initiative on PDC. Most of these courses are compulsory courses for undergraduate students of our department. In addition to the goals of the respective courses, we tried to fulfill many pedagogical goals corresponding to the PDC course. The teaching-contents of these courses have been adapted to also cover the aspects of PDC. A well-defined methodology for selection of PDC topics for integration, selection of topics for laboratory projects and home assignments, conduct of examinations (mid-term/end-term), pre-post feedback and evaluation, and other related activities has been planned. The methodology is intended to fulfill the priorly set goals and to achieve the intended learning outcomes (ILOs). This paper presents the methodology used, detailed topics and integration plan of PDC topics along with corresponding bloom levels, ILOs, evaluation strategies, and performance evaluation based on the students’ feedback and statistical analysis. Success of integration has been validated by performing statistical analysis of students’ pre-post feedback and performance in examinations.

OnRamp: A web-portal for teaching parallel and distributed computing

Computer Science students must understand parallel and distributed computing (PDC) concepts to be effective computer scientists in the workforce, as reflected in the 2013 ACM Curriculum guidelines. Significant work has been done by CS educators to develop curriculum materials and increase access to parallel compute environments (PCEs) by leveraging and clustering a plethora of small multicore systems. Even with these resources there is a barrier to entry for students to use PCEs, namely the unfamiliar and complex system software ecosystem of modern PCEs. The OnRamp project lowers that barrier to entry for exploring PDC concepts on a variety of PCEs by abstracting away the details of interacting with the system and focuses the students’ attention on the PDC concepts. This top-down approach is in contrast to existing approaches involving all aspects of PDC necessarily being taught before the key concepts can be explored. In this paper we discuss the motivation, design and implementation of OnRamp, a general purpose web portal for supporting the exploration of PDC concepts that harnesses the existing educational resources created by the CS education community. It coaches students through interactive modules that teach them about PDC concepts and PCEs while allowing them to launch parallel applications from day one.

Pervasive parallel and distributed computing in a liberal arts college curriculum

We present a model for incorporating parallel and distributed computing (PDC) throughout an undergraduate CS curriculum. Our curriculum is designed to introduce students early to parallel and distributed computing topics and to expose students to these topics repeatedly in the context of a wide variety of CS courses. The key to our approach is the development of a required intermediate-level course that serves as an introduction to computer systems and parallel computing. It serves as a requirement for every CS major and minor and is a prerequisite to upper-level courses that expand on parallel and distributed computing topics in different contexts. With the addition of this new course, we are able to easily make room in upper-level courses to add and expand parallel and distributed computing topics. The goal of our curricular design is to ensure that every graduating CS major has exposure to parallel and distributed computing, with both a breadth and depth of coverage. Our curriculum is particularly designed for the constraints of a small liberal arts college, however, much of its ideas and its design are applicable to any undergraduate CS curriculum.