Orchestration Framework Research Articles

Problems of multi-cloud solutions deploying and managing

The expediency of deploying software products in a multi-cloud is analyzed, the advantages and disadvantages of this approach, the problems associated with the use of services by several cloud providers are identified. It was investigated which cloud solutions are the most popular today and the cases in which it is better to use them, rather than competitors are considered. Multi-cloud management platforms are considered, which are focused on the use of organizations with a large number of geographically dispersed departments and offices. It has been determined what cloud resource orchestration frameworks support working with multiple clouds, and which technologies they use to make the best choice of a cloud solution.

Approaches for containerized scientific workflows in cloud environments with applications in life science

Containers are gaining popularity in life science research as they provide a solution for encompassing dependencies of provisioned tools, simplify software installations for end users and offer a form of isolation between processes. Scientific workflows are ideal for chaining containers into data analysis pipelines to aid in creating reproducible analyses. In this article, we review a number of approaches to using containers as implemented in the workflow tools Nextflow, Galaxy, Pachyderm, Argo, Kubeflow, Luigi and SciPipe, when deployed in cloud environments. A particular focus is placed on the workflow tool’s interaction with the Kubernetes container orchestration framework.

Contrast - a lightweight Python framework for beamline orchestration and data acquisition.

The emergence of fourth-generation synchrotrons is prompting the development of new systems for experimental control and data acquisition. However, asgeneral control systems are designed to cover a wide set of instruments andtechniques, they tend to become large and complicated, at the cost of experimental flexibility. Here we present Contrast, a simple Python framework for interacting with beamline components, orchestrating experiments and managing data acquisition. The system is presented and demonstrated via its application at the NanoMAX beamline of the MAX IV Laboratory.

Orchestrating ecosystems: a multi-layered framework

ABSTRACT Ecosystems are distinguished from other structural arrangements for value co-production by the nature of their governance and coordination challenges. As ecosystems are marked by their relative non-reliance upon formal, 1–1 supplier contracts to govern and coordinate productive activities, they need to find non-hierarchical ways of orchestrating ecosystem constituents such that a coherent system-level value offering is enabled and targeted at a defined user audience. Importantly, ecosystems cannot rely on ‘command-and-control’ governance to coordinate inputs from different participants, as would be the case of conventional supply chains. Instead, ecosystem leaders need to persuade others to make voluntary inputs that are consistent with the ecosystem’s overarching value offering. I call this task ‘ecosystem orchestration’. In this essay I suggest an ecosystem orchestration framework that distinguishes between technological, economic, institutional, and behavioural layers of ecosystem orchestration. I begin by highlighting distinctive governance challenges of ecosystems. I then provide a brief overview of the ecosystem orchestration literature. Then I introduce the different domains in which orchestration can be exercised. I conclude with a framework for orchestrating innovation ecosystems from birth to maturity. In addition to distinguishing between and describing technological, economic, institutional, and behavioural layers of ecosystem orchestration, the model also distinguishes between three stages of ecosystem momentum creation: initiation, scaling, and control. This framework has been designed to help practitioners to design strategies for ecosystem momentum creation.

Interactive Process Automation based on lightweight object detection in manufacturing processes

Interactive Process Automation refers to the idea of supporting the interaction of humans in processes through physical objects. This is particularly promising for human/cobot collaboration tasks where the communication is fuzzy. A typical example is a picking and placing scenario. Here, a “picking area” can serve as a user interface, i.e., objects are freely placed in a defined area, and then identified and transferred to specific positions, where deterministic processes can use them. If, for example, object A is placed at position posA by the human, automatically, the robot is instructed to pick A and place it at position posB on a tray. Realizing Interactive Process Automation for picking and placing tasks in manufacturing processes requires (i) a lightweight and flexible object detection approach and (ii) a human–machine interface design for Interactive Process Automation. This work proposes (i) an object detection approach that works solely based on synthetic training data. The object detection is embedded into (ii) generic process models that are implemented based on an existing manufacturing orchestration framework and a camera-equipped cobot. The approach is prototypically implemented and evaluated based on several experiments including a pick and place cobot station.

Object Oriented Policy Conflict Checking Framework in Cloud Networks (OOPC)

Software-Defined Networking (SDN) provides a programmable framework for multi-tenant cloud network management and orchestration. The end-to-end packet processing induced by virtual network functions (VNFs) like stateless firewall, load balancer, intrusion detection, and prevention system (IDPS) in a network involves the processing of network traffic through security policies matching the traffic pattern defined in security rules of individual VNF. The conflicting rules in terms of traffic match and conflicting actions can lead to a) violation of security requirements (authentication and authorization bypass) b) mission requirements - the presence of redundant rules (increased latency, reduced throughput). We present a new object-oriented policy conflict detection and resolution framework (OOPC), which analyzes the rule dependency relationships between the rules of heterogeneous virtual network functions (VNFs) and creates a VNF-Graph. The rules are analyzed using object-oriented dependencies between the address space and actions of VNF rules. OOPC utilizes a compact VNF-Graph, which leads to a reduction in search complexity when analyzing new security policies. Our security policy composition in our framework OOPC achieves 37 percent lower latency in policy graph composition than previous work. The proposed solution performs 20 percent faster security policy conflict detection on a cloud network with 60k OpenFlow rules than prior frameworks that serve a similar purpose.

Enhancing 5G SDN/NFV Edge with P4 Data Plane Programmability

The 5G network revolution will be enabled by deep integration of Software Defined Networking (SDN) and Network Function Virtualization (NFV) to support multi-tenancy, per-user and per-application quality of service and experience. However, full softwarization and current SDN platforms may not be able to sustain the complexity and the heterogeneity of different requirements, for example, strict latency, jitter, high precision traffic and advanced monitoring. For such services, SDN/NFV needs to be boosted not only considering orchestration and control plane, but also data plane programmability. In this article, the potential of the P4 language is illustrated with the aim to show its disruptive novel functionalities at the data plane level currently not available in a SDN/NFV network, opening the way to new orchestration frameworks and enabling a novel autonomic and flexible network at the edge. Use cases, assessments and softwarized performance results are proposed and discussed in the edge and IoT scenario, targeting advanced traffic engineering, cyber security, multi-tenancy, 5G offloading, and telemetry, to demonstrate the feasibility of such an approach.

Climate Ambition and Sustainable Development for a New Decade: A Catalytic Framework

AbstractThis paper examines the Global Climate Action Agenda (GCAA) and discusses options to improve sub‐ and non‐state involvement in post‐2020 climate governance. A framework that stimulates sub‐ and non‐state action is a necessary complement to national governmental action, as the latter falls short of achieving low‐carbon and climate‐resilient development as envisaged in the Paris Agreement. Applying design principles for an ideal‐type orchestration framework, we review literature and gather expert judgements to assess whether the GCAA has been collaborative, comprehensive, evaluative and catalytic. Results show that there has been greater coordination among orchestrators, for instance in the organization of events. However, mobilization efforts remain event‐driven and too little effort is invested in understanding the progress of sub‐ and non‐state action. Data collection has improved, although more sophisticated indicators are needed to evaluate climate and sustainable development impacts. Finally, the GCAA has recorded more action, but relatively little by actors in developing countries. As the world seeks to recover from the COVID‐19 crisis and enters a new decade of climate action, the GCAA could make a vital contribution in challenging times by helping governments keep and enhance climate commitments; strengthening capacity for sub‐ and non‐state action; enabling accountability; and realizing sustainable development.

마이크로서비스의 특징을 반영한 컨테이너 오케스트레이션 프레임워크 배치 기법

본 논문에서 마이크로서비스(microservice) 구조의 어플리케이션을 구성할 때, 마이크로서비스의 약결합(loosely-coupling) 특성을 반영한 배치 명세를 제안한다. 또한, 제안된 배치 명세 기반의 마이크로서비스 어플리케이션을 컨테이너 오케스트레이션 프레임워크(container orchestration framework)에 배치하는 방안을 제안한다. 제안된 기법을 Kubernetes의 기본 템플릿을 활용한 배치와의 차이점을 비교하고, 그 성능을 측정하기 위하여 시뮬레이션을 통하여 마이크로서비스 응답 지연시간을 측정하였다. 그 결과, Kubernetes의 기본 템플릿을 활용한 경우는 마이크로서비스 구조를 반영하지 않아 무작위로 배치되는 경향을 보였으나 제안된 기법의 경우 응답 지연시간을 줄이는 적절한 위치에 마이크로서비스가 배치됨을 확인하였다. 그리고 기존의 방법에 비해 평균 응답 지연시간이 약 1.95ms 감소하였다.

The Evolution of Resource-Based Inquiry: A Review and Meta-Analytic Integration of the Strategic Resources–Actions–Performance Pathway

Understanding why some firms outperform others is central to strategy research. The resource-based view (RBV) suggests that competitive advantages arise due to possessing strategic resources (i.e., assets that are valuable, rare, nonsubstitutable, and inimitable), and researchers have extended this logic to explain performance differences. However, RBV is relatively silent about the actions managers could use to create or capitalize on a resource-based advantage. Enriching RBV, the resource orchestration framework describes specific managerial actions that use such resources to realize performance gains. After reviewing the conceptual evolution of these two literature streams as well as related streams, we use meta-analytic structural equation modeling to aggregate evidence from 255 samples involving 111,120 observations to answer outstanding research questions regarding the strategic resources–actions–performance pathway. The results show strong complementarity and interdependence between their logics. Additional inquiry drawing on their complementarity is a clear path toward enhancing scholars’ understanding of how and why some firms outperform others. We build on our findings to lay a foundation for such inquiry, including a call for theorizing centered on the interdependence of resources and actions, as well as new theoretical terrain that can help resource-based inquiry continue to evolve.

Regime complexity and managing financial data streams: The orchestration of trade reporting for derivatives

AbstractInternational regime complexity has become a prominent feature of the global economy and world politics. The international governance of derivatives markets is a notable case of this phenomenon in finance because a variety of post‐crisis rules have been issued by a multitude of international standard‐setting bodies. By combining the regime complexity and orchestration frameworks, we explain the precision and scope of international standards for derivatives trade reporting. We show how a collective orchestrator (the Group of Twenty) and a hub intermediary (the Financial Stability Board) managed regime complexity through the orchestration of the available intermediaries. We also seek to refine the orchestration framework by explaining how the mechanism of issue de‐linkage can be used to manage the partly diverging goals among states within the collective orchestrator. Our findings are relevant for the global regulation and governance of other policy areas characterized by a multiplicity of actors and interlinked issues, such as trade, energy, and environmental policy.

Software-Defined Workflows for Distributed Interoperable Closed-Loop Neuromodulation Control Systems

Closed-loop neuromodulation control systems facilitate regulating abnormal physiological processes by recording neurophysiological activities and modifying those activities through feedback loops. Designing such systems requires interoperable service composition, consisting of cycles. Workflow frameworks enable standard modular architectures, offering reproducible automated pipelines. However, those frameworks limit their support to executions represented by directed acyclic graphs (DAGs). DAGs need a pre-defined start and end execution step with no cycles, thus preventing the researchers from using the standard workflow languages as-is for closed-loop workflows and pipelines. In this paper, we present NEXUS, a workflow orchestration framework for distributed analytics systems. NEXUS proposes a Software-Defined Workflows approach, inspired by Software-Defined Networking (SDN), which separates the data flows across the service instances from the control flows. NEXUS enables creating interoperable workflows with closed loops by defining the workflows in a logically centralized approach, from microservices representing each execution step. The centralized NEXUS orchestrator facilitates dynamically composing and managing scientific workflows from the services and existing workflows, with minimal restrictions. NEXUS represents complex workflows as directed hypergraphs (DHGs) rather than DAGs. We illustrate a seamless execution of neuromodulation control systems by supporting loops in a workflow as the use case of NEXUS. Our evaluations highlight the feasibility, flexibility, performance, and scalability of NEXUS in modeling and executing closed-loop workflows.

Dynamic Service Function Chain Orchestration for NFV/MEC-Enabled IoT Networks: A Deep Reinforcement Learning Approach

Network function virtualization (NFV) and mobile-edge computing (MEC) have been introduced by Internet service providers (ISPs) to deal with various challenges, which hinder them from satisfying ambitious quality of experience demands of the Internet-of-Things (IoT) applications. In NFV/MEC-enabled IoT networks, any IoT service can be expressed as a service function chain (SFC) consisting of several strictly ordered virtual network functions (VNFs), which can be geographically placed onto edge clouds close to IoT terminals. However, regarding the large number of IoT terminals and constant dynamics of IoT networks, determining the placement of VNFs and routing service paths that optimize the end-to-end delays in the hybrid edge clouds is a challenging problem. This problem is also called SFC dynamic orchestration problem (SFC-DOP). To address the SFC-DOP, we are motivated to creatively present an SFC dynamic orchestration framework for IoT deep reinforcement learning (DRL). Also, a DRL-based algorithm for SFC-DOP with the actor–critic and the deterministic policy gradient scheme is provided, which can efficiently deal with the SFC-DOP in IoT networks. The obtained experimental results show the outstanding performance of our proposal compared with the current benchmarks.

A Fog Computing-Based Device-Driven Mobility Management Scheme for 5G Networks.

The fog computing-based device-driven network is a promising solution for high data rates in modern cellular networks. It is a unique framework to reduce the generated-data, data management overheads, network scalability challenges, and help us to provide a pervasive computation environment for real-time network applications, where the mobile data is easily available and accessible to nearby fog servers. It explores a new dimension of the next generation network called fog networks. Fog networks is a complementary part of the cloud network environment. The proposed network architecture is a part of the newly emerged paradigm that extends the network computing infrastructure within the device-driven 5G communication system. This work explores a new design of the fog computing framework to support device-driven communication to achieve better Quality of Service (QoS) and Quality of Experience (QoE). In particular, we focus on, how potential is the fog computing orchestration framework? How it can be customized to the next generation of cellular communication systems? Next, we propose a mobility management procedure for fog networks, considering the static and dynamic mobile nodes. We compare our results with the legacy of cellular networks and observed that the proposed work has the least energy consumption, delay, latency, signaling cost as compared to LTE/LTE-A networks.

A deep reinforcement learning-based algorithm for reliability-aware multi-domain service deployment in smart ecosystems

The transition towards full network virtualization will see services for smart ecosystems including smart metering, healthcare and transportation among others, being deployed as Service Function Chains (SFCs) comprised of an ordered set of virtual network functions. However, since such services are usually deployed in remote cloud networks, the SFCs may transcend multiple domains belonging to different Infrastructure Providers (InPs), possibly with differing policies regarding billing and Quality-of-service (QoS) guarantees. Therefore, efficiently allocating the exhaustible network resources to the different SFCs while meeting the stringent requirements of the services such as delay and QoS among others, remains a complex challenge, especially under limited information disclosure by the InPs. In this work, we formulate the SFC deployment problem across multiple domains focusing on delay constraints, and propose a framework for SFC orchestration which adheres to the privacy requirements of the InPs. Then, we propose a reinforcement learning (RL)-based algorithm for partitioning the SFC request across the different InPs while considering service reliability across the participating InPs. Such RL-based algorithms have the intelligence to infer undisclosed InP information from historical data obtained from past experiences. Simulation results, considering both online and offline scenarios, reveal that the proposed algorithm results in up to 10% improvement in terms of acceptance ratio and provisioning cost compared to the benchmark algorithms, with up to more than 90% saving in execution time for large networks. In addition, the paper proposes an enhancement to a state-of-the-art algorithm which results in up to 5% improvement in terms of provisioning cost.

MESON: Optimized Cross-Slice Communication for Edge Computing

Network slicing is set out to address crucial needs of 5G, including support for multi-service provisioning. Focusing on resource and performance isolation, as well as security concerns associated with multi-tenancy, existing management and orchestration (MANO) frameworks typically offer network slices in the form of isolated bundles of computing, storage, and network resources across the network infrastructure, including the edge. However, network slice instantiation in its prevailing form raises significant concerns related to performance and resource utilization, hindering potential business-to-business (B2B) synergies. In this article, we discuss a new aspect of network slicing, optimized cross-slice communication (CSC). We argue that multi-tenancy and service co-location open up unique opportunities for B2B interactions, inter-service communications, and service composition, especially in the domain of edge computing and location-based services. In this context, we present optimized edge slice orchestration (MESON), a MANO-based architecture for optimized CSC in edge clouds. We discuss the main architecture components and descriptors, as well as the steps required for the discovery of CSC-enabled services and the establishment of optimized CSC among co-located slices.

Cloud resource orchestration in the multi-cloud landscape: a systematic review of existing frameworks

The number of both service providers operating in the cloud market and customers consuming cloud-based services is constantly increasing, proving that the cloud computing paradigm has successfully delivered its potential. Nevertheless, the unceasing growth of the cloud market is posing hard challenges on its participants. On the provider side, the capability of orchestrating resources in order to maximise profits without failing customers’ expectations is a matter of concern. On the customer side, the efficient resource selection from a plethora of similar services advertised by a multitude of providers is an open question. In such a multi-cloud landscape, several research initiatives advocate the employment of software frameworks (namely, cloud resource orchestration frameworks - CROFs) capable of orchestrating the heterogeneous resources offered by a multitude of cloud providers in a way that best suits the customer’s need. The objective of this paper is to provide the reader with a systematic review and comparison of the most relevant CROFs found in the literature, as well as to highlight the multi-cloud computing open issues that need to be addressed by the research community in the near future.

Auto-3P: An autonomous VNF performance prediction & placement framework based on machine learning

We propose Auto-3P, an Autonomous module for Virtual Network Functions Performance Prediction and Placement at network cloud and edge facilities based on Machine Learning (ML). Auto-3P augments the autonomous placement capabilities of MANagement and Orchestration frameworks (MANOs) by considering both resource availability at hosting nodes and the implied impact of a VNF node placement decisions on the whole service level end-to-end performance. Unlike that, most existing placement methods take a rather myopic approach after manual rule-based decisions, and/or based exclusively on a host-centric view that focuses merely on node-local resource availability and network metrics. We evaluate and validate Auto-3P with real-field trials in the context of a well-defined Smart City Safety use case using a real end-to-end application over a real city-based testbed. We meticulously conduct repeated tests to assess (i) the accuracy of our adopted prediction models; and (ii) their placement performance against three other existing MANO approaches, namely, a “Traditional”, a “Latency-aware” and a “Random” one, as well as against a collection of well-known Time Series Forecasting (TSF) methods. Our results show that the accuracy of our ML models outperforms the one by TSF models, with the most prominent accuracy performances being exhibited by models such as K-Nearest Neighbors Regression (K-NNR), Decision Tree (DT), and Support Vector Regression (SVR). What is more, the resulted end-to-end service level performance of our approach outperforms “Traditional”, “Latency-aware”, and Random MANO placement. Last, Auto-3P achieves load balancing at selected VNF hosts without degrading end-to-end service level delay, and without a need for a (fixed) overload threshold check, unlike what is suggested by other works in the literature for coping with heavy system-wide load conditions.

Aloe: Fault-Tolerant Network Management and Orchestration Framework for IoT Applications

Internet of Things (IoT) platforms use a large number of low-cost resource constrained devices and generates millions of short-flows. In-network processing is gaining popularity day by day to handle IoT applications and services. However, traditional software-defined networking (SDN) based management systems are not suitable to handle the plug and play nature of such systems. In this paper, we propose Aloe, an auto-scalable SDN orchestration framework. Aloe exploits in-network processing framework by using multiple lightweight controller instances in place of service grade SDN controller applications. The proposed framework ensures the availability and significant reduction in flow-setup delay by deploying instances in the vicinity the resource constraint IoT devices dynamically. Aloe supports fault-tolerance with recovery from network partitioning by employing self-stabilizing placement of migration capable controller instances. Aloe also provides resource reservation for micro-controllers so that they can ensure the quality of services (QoS). The performance of the proposed system is measured by using an in-house testbed along with a large scale deployment in Amazon Web services (AWS) cloud platform. The experimental results from these two testbeds show significant improvement in response time for standard IoT based services. This improvement of performance is due to the reduction in flow-setup time. We found that Aloe can improve flow-setup time by around 10%-30% in comparison to one of the states of the art orchestration framework.

Semantic-Aware Security Orchestration in SDN/NFV-Enabled IoT Systems.

IoT systems can be leveraged by Network Function Virtualization (NFV) and Software-Defined Networking (SDN) technologies, thereby strengthening their overall flexibility, security and resilience. In this sense, adaptive and policy-based security frameworks for SDN/NFV-aware IoT systems can provide a remarkable added value for self-protection and self-healing, by orchestrating and enforcing dynamically security policies and associated Virtual Network Functions (VNF) or Virtual network Security Functions (VSF) according to the actual context. However, this security orchestration is subject to multiple possible inconsistencies between the policies to enforce, the already enforced management policies and the evolving status of the managed IoT system. In this regard, this paper presents a semantic-aware, zero-touch and policy-driven security orchestration framework for autonomic and conflict-less security orchestration in SDN/NFV-aware IoT scenarios while ensuring optimal allocation and Service Function Chaining (SFC) of VSF. The framework relies on Semantic technologies and considers the security policies and the evolving IoT system model to dynamically and formally detect any semantic conflict during the orchestration. In addition, our optimized SFC algorithm maximizes the QoS, security aspects and resources usage during VSF allocation. The orchestration security framework has been implemented and validated showing its feasibility and performance to detect the conflicts and optimally enforce the VSFs.