Distributed Systems Research Research Articles

Reducing data volume in big data: Parallel processing based data filtering techniques

Reducing data volume in big data: Parallel processing based data filtering techniques Professor Shikharesh Majumdar from Carleton University examines reducing data volume in big data, focusing on parallel processing based data filtering techniques. Volume, velocity, and variety are three well-known characteristics of big data. The large data volumes often introduce formidable challenges to processing such data in a timely and economical manner. Research on data filtering is underway under the leadership of Shikharesh Majumdar at the Real Time and Distributed Systems Research Centre at Carleton University. Users are often interested only in a subset of the raw data.

A PID Control Architecture Based on IEC 61499

This work presents an approach for PID control and optimization in a distributed system, using the IEC 61499 control standard. This standard enables communication among different PLCs, which are used to develop a three-layered event-driven control of a SISO loop. The lowest layer is in charge of cyclical data acquisition. The second layer carries out an event-based PID control. The highest layer runs a control optimization algorithm, specifically a simple tuning approach based on Ziegler-Nichols that is used to determine the PID parameters for different operating points. The proposed approach is assessed in a tank level SISO control problem, whose behavior can be modeled as a first-order plus dead time system in each operating point. For that purpose, it has been implemented using two PLCs and a software PLC running on an industrial computer. The experimental results on the SISO level control loop show the feasibility of the proposed approach for event-driven control in a distributed system and open interesting research questions in the intersection of controller design and distributed automation.

Advancing Reproducibility in Parallel and Distributed Systems Research

This installment of Computer’s series highlighting the work published in IEEE Computer Society journals comes from IEEE Transactions on Parallel and Distributed Systems.

Guest Editorial: Special Section on SC19 Student Cluster Competition

Reproducibility is foundational to solid scientific and technical research, and the ability to repeat the research that produced is a key approach for confirming the validity of a new scientific discovery. The IEEE Transactions on Parallel and Distributed Systems (TPDS) is committed to enabling reproducible research through transparency and the availability and potential reuse of code. As part of its Reproducibility Initiative, TPDS is exploring post-publication peer review of code associated with articles published in TPDS. Authors who have published in TPDS can make their published article more reproducible and earn a reproducibility badge by submitting their associated code for post-publication peer review. However, the nature of the parallel and distributed systems research covered by TPDS makes it challenging to evaluate code and data challenging for reproducibility. This is because such an evaluation may require access to specific hardware, system architectures and scales, OS configurations, and so on, which may not be feasible or practical. Consequently, TPDS is exploring an alternate approach where members of the community can submit short, supplemental ‘critique’ papers that present their experiences in reproducing published results using the artifacts and/or evaluations or experiences with published artifacts. These supplemental paper submissions are reviewed and, if accepted, are linked to the original publication and are citable, serving to validate the reproducibility of the original publication. This special section, consisting of a primary paper and 6 critique papers that reproduce the results of the primary paper, is the initial pilot of this approach. The special section builds on the efforts of the SC19 Student Cluster Competition, which was part of the SC19 conference (https://sc19.supercomputing.org/) and represents a step forward in enabling reproducible research publications in parallel and distributed systems.

File System Support for Privacy-Preserving Analysis and Forensics in Low-Bandwidth Edge Environments

In this paper, we present initial results from our distributed edge systems research in the domain of sustainable harvesting of common good resources in the Arctic Ocean. Specifically, we are developing a digital platform for real-time privacy-preserving sustainability management in the domain of commercial fishery surveillance operations. This is in response to potentially privacy-infringing mandates from some governments to combat overfishing and other sustainability challenges. Our approach is to deploy sensory devices and distributed artificial intelligence algorithms on mobile, offshore fishing vessels and at mainland central control centers. To facilitate this, we need a novel data plane supporting efficient, available, secure, tamper-proof, and compliant data management in this weakly connected offshore environment. We have built our first prototype of Dorvu, a novel distributed file system in this context. Our devised architecture, the design trade-offs among conflicting properties, and our initial experiences are further detailed in this paper.

Enabling Reproducible Research in Parallel and Distributed Systems

This installment of Computer’s series highlighting the work published in IEEE Computer Society Journals comes from IEEE Transactions on Parallel and Distributed Systems.

The evolution of distributed computing systems: from fundamental to new frontiers

Distributed systems have been an active field of research for over 60 years, and has played a crucial role in computer science, enabling the invention of the Internet that underpins all facets of modern life. Through technological advancements and their changing role in society, distributed systems have undergone a perpetual evolution, with each change resulting in the formation of a new paradigm. Each new distributed system paradigm—of which modern prominence include cloud computing, Fog computing, and the Internet of Things (IoT)—allows for new forms of commercial and artistic value, yet also ushers in new research challenges that must be addressed in order to realize and enhance their operation. However, it is necessary to precisely identify what factors drive the formation and growth of a paradigm, and how unique are the research challenges within modern distributed systems in comparison to prior generations of systems. The objective of this work is to study and evaluate the key factors that have influenced and driven the evolution of distributed system paradigms, from early mainframes, inception of the global inter-network, and to present contemporary systems such as edge computing, Fog computing and IoT. Our analysis highlights assumptions that have driven distributed systems appear to be changing, including (1) an accelerated fragmentation of paradigms driven by commercial interests and physical limitations imposed by the end of Moore’s law, (2) a transition away from generalized architectures and frameworks towards increasing specialization, and (3) each paradigm architecture results in some form of pivoting between centralization and decentralization coordination. Finally, we discuss present day and future challenges of distributed research pertaining to studying complex phenomena at scale and the role of distributed systems research in the context of climate change.

Hierarchical Byzantine fault-tolerance protocol for permissioned blockchain systems

Emerging blockchain technology has introduced a new challenge to the distributed system research: Can Byzantine fault-tolerance protocols scale up to, for example, hundreds of nodes? In this work, we introduce HiBFT, a hierarchical Byzantine fault-tolerance protocol to address the problem. The core idea is to divide replicas into groups and exchange consensus messages among groups, thus avoiding the necessity of message broadcasting. The motivation for such approach bases on the hierarchical property of network architecture in permissioned block chains, our target deployments. HiBFT works very much in the same way as the classical Practical Byzantine Fault-Tolerance protocol. However, it replaces the concept of physical replica with a logical one that represents a replica group. As such, protocol message complexity can be reduced from $$O(N^2)$$ to $$O(s^2)$$ where N and s are the total number of replicas and the number of groups. Additionally, using threshold signature scheme for representing a logical group results in two important improvements: The cost of signature verification is significantly reduced at each replica; blocks can be secured more effectively in terms of signature size. Our protocol guarantees safety and liveness under partially synchronous assumption with a correctness proof. Our experiment results show that the protocol can scale up to hundred of nodes.

Tsumiki: A Meta-Platform for Building Your Own Testbed

Network testbeds are essential research tools that have been responsible for valuable network measurements and major advances in distributed systems research. However, no single testbed can satisfy the requirements of every research project, prompting continual efforts to develop new testbeds. The common practice is to re-implement functionality anew for each testbed. This work introduces a set of ready-to-use software components and interfaces called Tsumiki to help researchers to rapidly prototype custom networked testbeds without substantial effort. We derive Tsumiki's design using a set of component and interface design principles, and demonstrate that Tsumiki can be used to implement new, diverse, and useful testbeds. We detail a few such testbeds: a testbed composed of Android devices, a testbed that uses Docker for sandboxing, and a testbed that shares computation and storage resources among Facebook friends. A user study demonstrated that students with no prior experience with networked testbeds were able to use Tsumiki to create a testbed with new functionality and run an experiment on this testbed in under an hour. Furthermore, Tsumiki has been used in production in multiple testbeds, resulting in installations on tens of thousands of devices and use by thousands of researchers.

Analyzing controller conflicts in multimodal smart grids - framework design

In future energy systems dominated by distributed energy resources, additional flexibility potential will be unlocked by interlinking the infrastructures of different energy systems. Direct and indirect coupling of different energy systems like power grids, gas and heat infrastructures leads to multimodal energy systems. Combined with an ongoing pervasion of the resulting interconnected system with information and communication technology (ICT), multimodal smart grids evolve. The pervasion of the system with autonomous controllers leads on to the development of a distributed adaptive system. From an engineering perspective, it shows the desired characteristics regarding scalability and self-stabilization. Unlike these desired aspects, other effects can emerge from the distributed and adaptive nature of this system: Autonomous controllers might counteract, thus leading to instable system states. From the perspective of distributed systems research, these controller conflicts can thus be understood as unintended effects of emergence. In this short paper, we present design decision for a framework that will be developed for the evaluation of controller conflicts in multimodal smart grids. The concept of technical emergence will be applied to deterministically simulate controller conflicts generated by autonomous controllers of interlinked infrastructures.

A comparison of resource allocation process in grid and cloud technologies

Grid Computing and Cloud Computing are two different technologies that have emerged to validate the long-held dream of computing as utilities which led to an important revolution in IT industry. These technologies came with several challenges in terms of middleware, programming model, resources management and business models. These challenges are seriously considered by Distributed System research. Resources allocation is a key challenge in both technologies as it causes the possible resource wastage and service degradation. This paper is addressing a comprehensive study of the resources allocation processes in both technologies. It provides the researchers with an in-depth understanding of all resources allocation related aspects and associative challenges, including: load balancing, performance, energy consumption, scheduling algorithms, resources consolidation and migration. The comparison also contributes an informal definition of the Cloud resource allocation process. Resources in the Cloud are being shared by all users in a time and space sharing manner, in contrast to dedicated resources that governed by a queuing system in Grid resource management. Cloud Resource allocation suffers from extra challenges abbreviated by achieving good load balancing and making right consolidation decision.

Abstracting the Geniuses Away from Failure Testing

This article presents a call to arms for the distributed systems research community to improve the state of the art in fault tolerance testing. Ordinary users need tools that automate the selection of custom-tailored faults to inject. We conjecture that the process by which superusers select experiments can be effectively modeled in software. The article describes a prototype validating this conjecture, presents early results from the lab and the field, and identifies new research directions that can make this vision a reality.

A Novel Approach Towards Sustainable Testbeds: ToMaTo on CloudLab

AbstractAlthough testbeds and experimental facilities are accepted as tools for research in the scope of distributed systems research, such as Future Internet Research, they lack sustainability; once established as a project, they must be maintained and renewed every few years. However, unfortunately, due to an absence of long-term funding for operating/maintenance and reinvestment, a lot of these are terminated just after the project is complete.In this paper, we describe a software-defined testbed on demand; this is based on the Topology Management Tool (ToMaTo), a software environment which was developed within the G-Lab project. ToMaTo, initially designed to run on the G-Lab infrastructure, is now independent of the underlying infrastructure and can be deployed on any Cloud infrastructure on demand. We describe here the deployment of ToMaTo to the CloudLab infrastructure offered by the NSF-funded CloudLab project.This approach offers a new method for sustainable testbeds where neither upfront investment nor the recurring and operating/maintenance costs for the infrastructure will burden the budgets of the projects.

On analyzing poblems of distributed systems and current internet in front of the future internet architectures

Nowadays, there are hundreds of underway worldwide projects to redesign both com- munication protocols and architecture of the Internet. These initiatives are collectively called “future Internet” research. Most of these initiatives rely on existing distributed systems, which often limit or even prevent the development of “clean slate” solutions. The main reason is that the great majority of distributed systems are tightly-linked with the TCP/IP protocol stack. In this article, we provide a first glance discussion on the relationships between future Internet and distributed systems research, focusing on dependencies and similar requirements among these areas. From this analysis, it beco- mes evident that many of the future Internet requirements (and open challenges) are repeated in the distributed systems landscape. Although there are many studies on both research fronts individually, the study of the key challenges of future Internet when addressing distributed systems requirements is a topic yet not explored in our contemporary research. This paper aims at determining the gaps and requirements future Internet must fulfill in order to support future distributed systems. To support this objective, a set of design metrics are identified and a convergent design space is proposed.

HAMR: A dataflow-based real-time in-memory cluster computing engine

As the attention given to big data grows, cluster computing systems for distributed processing of large data sets become the mainstream and critical requirement in high performance distributed system research. One of the most successful systems is Hadoop, which uses MapReduce as a programming/execution model and takes disks as intermedia to process huge volumes of data. Spark, as an in-memory computing engine, can solve the iterative and interactive problems more efficiently. However, currently it is a consensus that they are not the final solutions to big data due to a MapReduce-like programming model, synchronous execution model and the constraint that only supports batch processing, and so on. A new solution, especially, a fundamental evolution is needed to bring big data solutions into a new era. In this paper, we introduce a new cluster computing system called HAMR which supports both batch and streaming processing. To achieve better performance, HAMR integrates high performance computing approaches, i.e. dataflow fundamental into a big data solution. With more specifications, HAMR is fully designed based on in-memory computing to reduce the unnecessary disk access overhead; task scheduling and memory management are in fine-grain manner to explore more parallelism; asynchronous execution improves efficiency of computation resource usage, and also makes workload balance across the whole cluster better. The experimental results show that HAMR can outperform Hadoop MapReduce and Spark by up to 19x and 7x respectively, in the same cluster environment. Furthermore, HAMR can handle scaling data size well beyond the capabilities of Spark.

A survey of general-purpose experiment management tools for distributed systems

In the field of large-scale distributed systems, experimentation is particularly difficult. The studied systems are complex, often nondeterministic and unreliable, software is plagued with bugs, whereas the experiment workflows are unclear and hard to reproduce. These obstacles led many independent researchers to design tools to control their experiments, boost productivity and improve quality of scientific results.Despite much research in the domain of distributed systems experiment management, the current fragmentation of efforts asks for a general analysis. We therefore propose to build a framework to uncover missing functionality of these tools, enable meaningful comparisons between them and find recommendations for future improvements and research.The contribution in this paper is twofold. First, we provide an extensive list of features offered by general-purpose experiment management tools dedicated to distributed systems research on real platforms. We then use it to assess existing solutions and compare them, outlining possible future paths for improvements.

Rediscovering Distributed Systems

To satisfy requirements such as scalability and availability, many of today's applications are distributed systems. But because designing and implementing such systems is inherently difficult, developers are increasingly turning to decades of distributed systems research for guidance. This article briefly explores important areas of distributed systems research along with some classic papers distributed systems developers will find useful.

Complexity analysis of a matchmaker based on hidden Markov model for decentralised grid scheduling

The design of scheduling algorithms for a heterogeneous computing system interconnected with an arbitrary communication network is one of the main concerns in distributed systems research, due to the heterogeneous nature of resource, computing capacity, memory, devices, internet and institutional policies. Different models have been proposed for the scheduler, which contemplate point-to-point communication to allow the direct exchange of information and provide more flexibility to the planning within the open systems. This balances the load of the system, improves interoperability between the components, the management of resources, performance, reliability and a high degree of scalability. However, related works assumed the lack of control of the resource by scheduler. We propose the analysis of theoretical and experimental complexity of an algorithm based on hidden Markov models because it facilitates the prediction of availability of the grid resources and can evolve over time.

ToMaTo a Virtual Research Environment for Large Scale Distributed Systems Research

Networks and distributed systems are an important field of research. To enable experimental research in this field we propose a new tool ToMaTo (Topology Management Tool) which was developed to support research projects within the BMBF funded project G-Lab. It should support researchers from various branches of science who investigate distributed systems by providing a virtual environment for their research. Using various virtualization technologies, ToMaTo is able to provide realistic components that can run realworld software as well as lightweight components that can be used to analyze algorithms at large scale. This paper describes how an additional virtualization technology from the Seattle testbed has been added to ToMaTo to allow even larger experiments with distributed algorithms. Moreover the paper describes some concrete experiments that are carried out with ToMaTo.

Designing a Software Transactional Memory for Peer-To-Peer Systems

AbstractTransactional memory is a rather novel approach to concurrency control in parallel computing, that has just recently found its way into distributed systems research. However, the research concentrates mainly on single processor solutions or cluster environment. In this paper we argue, that peer-to-peer systems would require a different design of transactional memory because of the increased failure-rate of nodes, slower network and possibility of network splits. We also present a few of our design ideas, namely increased performance and fault tolerance through the use of higher-level conflict detection and resolution via abstract data types and eventually consistency, that as we think could be important to a successful implementation of a scalable and resilient transactional memory.