Orchestration Software Research Articles

Deep integration of low-cost liquid handling robots in an industrial pharmaceutical development environment

The pharmaceutical industry is increasingly embracing laboratory automation to enhance experimental efficiency and operational resilience, particularly through the integration of automated liquid handlers (ALHs). This paper explores the integration of the low-cost Opentrons OT-2 liquid handling robot with F. Hoffmann-La Roche AG's in-house workflow orchestration software, AutoLab, to overcome barriers to lab automation. By leveraging the OT-2′s development-oriented interfaces and AutoLab's modular architecture, we achieved a user-friendly, cost-efficient, and flexible automation solution that aligns with FAIR (findable, accessible, interoperable, reusable) data principles. We demonstrate an advanced workflow development methodology, utilizing the software architecture, that facilitates the creation of two flexible pipetting protocols and medium complexity assays. This deep integration approach diminishes the learning curve for novice users while simultaneously enhancing the overall efficiency and reliability of the experimental workflow. Our findings suggest that such integrations can significantly mitigate the challenges associated with lab automation, including cost, complexity, and adaptability, paving the way for more accessible and robust automated systems in pharmaceutical research.

The Virtual Research Environment: A multi-science analysis platform

One of the objectives of the ESCAPE collaboration, within the context of the implementation of the European Open Science Cloud, is integrating analysis workflows from Cosmology, Astrophysics and High Energy Physics into a common framework, making it easier for researchers to share data, software and analysis code. The development of the ESCAPE Science Projects (Dark Matter and Extreme Universe) relies on the implementation of the Virtual Research Environment, a prototype analysis platform deployed in the respect of FAIR data principles. The Virtual Research Environment’s main components are (1) an Authentication and Authorization layer, (2) a federated distributed storage solution, providing functionalities for data injection and replication through a Data Management and Orchestration software, (3) an interactive notebook interface that hides the complexity of the underlying infrastructure and facilitates browsing data catalogues and (4) computing processing power to run full end-to-end analyses through workflow schedulers and resource managers that also allow for pipelines’ preservation. This contribution will depict the collection of services and technologies that compose the Virtual Research Environment, hosted on a cloud infrastructure and managed by an open source, publicly available repository.

The yaq project: Standardized software enabling flexible instrumentation

Modern instrumentation development often involves the incorporation of many dissimilar hardware peripherals into a single unified instrument. The increasing availability of modular hardware has brought greater instrument complexity to small research groups. This complexity stretches the capability of traditional, monolithic orchestration software. In many cases, a lack of software flexibility leads creative researchers to feel frustrated, unable to perform experiments they envision. Herein, we describe Yet Another acQuisition (yaq), a software project defining a new standardized way of communicating with diverse hardware peripherals. yaq encourages a highly modular approach to experimental software development that is well suited to address the experimental flexibility needs of complex instruments. yaq is designed to overcome hardware communication barriers that challenge typical experimental software. A large number of hardware peripherals are already supported, with tooling available to expand support. The yaq standard enables collaboration among multiple research groups, increasing code quality while lowering development effort.

Strong Temporal Isolation Among Containers in OpenStack for NFV Services

In this paper, the problem of temporal isolation among containerized software components running in shared cloud infrastructures is tackled, proposing an approach based on hierarchical real-time CPU scheduling. This allows for reserving a precise share of the available computing power for each container deployed in a multi-core server, so to provide it with a stable performance, independently from the load of other co-located containers. The proposed technique enables the use of reliable modeling techniques for end-to-end service chains that are effective in controlling the application-level performance. An implementation of the technique within the well-known OpenStack cloud orchestration software is presented, focusing on a use-case framed in the context of network function virtualization. The modified OpenStack is capable of leveraging the special real-time scheduling features made available in the underlying Linux operating system through a patch to the in-kernel process scheduler. The effectiveness of the technique is validated by gathering performance data from two applications running in a real test-bed with the mentioned modifications to OpenStack and the Linux kernel. A performance model is developed that tightly models the application behavior under a variety of conditions. Extensive experimentation shows that the proposed mechanism is successful in guaranteeing isolation of individual containerized activities on the platform.

Improving datacenter utilization through containerized service-based architecture

The modern datacenter's computing capabilities have far outstripped the applications running within and have become a hidden cost of doing business due to how software is architected and deployed. Resources are over-allocated to monolithic applications that sit idle for large parts of the day. If applications were architected and deployed differently, shared services could be used for multiple applications as needed. When combined with powerful orchestration software, containerized microservices can both deploy and dynamically scale applications from very small to very large within moments—scaling the application not only across a single datacenter but across all datacenters where the application(s) are deployed.In this paper, we analyze data from an application(s) deployed both as a single monolithic codebase and as a containerized application using microservice-based architecture to calculate the performance and computing resource waste are both architected and deployed. A modern approach is offered as a solution as a path from how to go from a monolithic codebase to a more efficient, reliable, scalable, and less costly deployment model.

Metagenomics workflow for hybrid assembly, differential coverage binning, metatranscriptomics and pathway analysis (MUFFIN).

Metagenomics has redefined many areas of microbiology. However, metagenome-assembled genomes (MAGs) are often fragmented, primarily when sequencing was performed with short reads. Recent long-read sequencing technologies promise to improve genome reconstruction. However, the integration of two different sequencing modalities makes downstream analyses complex. We, therefore, developed MUFFIN, a complete metagenomic workflow that uses short and long reads to produce high-quality bins and their annotations. The workflow is written by using Nextflow, a workflow orchestration software, to achieve high reproducibility and fast and straightforward use. This workflow also produces the taxonomic classification and KEGG pathways of the bins and can be further used for quantification and annotation by providing RNA-Seq data (optionally). We tested the workflow using twenty biogas reactor samples and assessed the capacity of MUFFIN to process and output relevant files needed to analyze the microbial community and their function. MUFFIN produces functional pathway predictions and, if provided de novo metatranscript annotations across the metagenomic sample and for each bin. MUFFIN is available on github under GNUv3 licence: https://github.com/RVanDamme/MUFFIN.

Production Process Interoperability for Cyber-Physical Production Systems

Abstract Operating/maintaining a shop-floor and planning for the efficient utilization of said shop floor, are typically two different sides of a coin. While the operation side is focused on making the interaction between humans, machines and software run flawlessly (e.g. by utilizing orchestration software), the planning side is focused on finding the right order of doing things for good utilization of resources. Planning is mostly done according to a regular schedule (once a day, once a week). The production of individual products is intended to happen in the regularly pre-determined order, including some slack to account for minor problems. During production, shop-floor workers (or smart machinery) can deal with problems, but they are nonetheless intended to follow the pre-determined plan. The contribution of this paper is a case-study that explores the integration and interoperability problems that occur, when a flexible shop-floor orchestration software centurio.work is coupled with a high-performance scheduling software (NgMPSS) in order to allow for ad-hoc planning and re-planning of a production schedule. The idea behind this integration is, (1) to allow for a system that is more robust when dealing with problems, but also (2) allows for reaping additional overall time savings, because dynamic re-planning allows for minimal slack between tasks. This research has been conducted on the interface between the Center for Digital Production (CDP) and the Pro2Future (P2F) as part of a common research project.

Claims-Based VM Authorization on OpenStack Private Cloud using Blockchain

Abstract The complexity of providing secure access, protecting critical data and end-user privacy in cloud data centre is leading to a demand for a new approach in network and data security. Recently, blockchain technology is being used in claims management to provide a decentralized and secure solution. The issue of security is crucial in the Virtual machine authorization in the cloud data centers. In the traditional approach of VM authorization, SSH key and IP address is given to the user to log into virtual machines. This opens much vulnerability as it might get spoofed or sniffed from the network, leading to accesses of private data to intruder. In this work, we propose a method which intends to aid in the security of the VM authorization using claims-based authorization system in conjunction with Blockchain based decentralized storage. Furthermore, the proposed system automates the process of launching a VM in OpenStack orchestration software.

Cloud resource orchestration optimisation based on ARIMA

The problem of resources management in the cloud environment, focusing on the platform as a service (PaaS), satisfying the demands of users, and relieving the load on the server in high concurrency, requires attention. After analysing the resource orchestration technology on PaaS, this paper presents a dynamic orchestration optimisation framework based on autoregressive integrated moving average model (ARIMA) model. The architecture is based on an OpenStack infrastructure as a service (IaaS), and combines with Cloudify, a resource orchestration software on the PaaS. Adjustments may be made in advance by predicting the resource consumption in the next time period. Experiments showed that this architecture can effectively shorten the concurrent response time and improve the utilisation of memory.

Cloud resource orchestration optimisation based on ARIMA

The problem of resources management in the cloud environment, focusing on the platform as a service (PaaS), satisfying the demands of users, and relieving the load on the server in high concurrency, requires attention. After analysing the resource orchestration technology on PaaS, this paper presents a dynamic orchestration optimisation framework based on autoregressive integrated moving average model (ARIMA) model. The architecture is based on an OpenStack infrastructure as a service (IaaS), and combines with Cloudify, a resource orchestration software on the PaaS. Adjustments may be made in advance by predicting the resource consumption in the next time period. Experiments showed that this architecture can effectively shorten the concurrent response time and improve the utilisation of memory.

Fault-tolerance analyzer: A middle layer for pre-provision testing in OpenStack

OpenStack is arguably the most popular open-source cloud orchestration software available today. Its user-base spans from large corporations and large scale service providers to SMEs. OpenStack manages infrastructure resources e.g. compute, storage and networking. While customers get a fully functional cloud, an incumbent fault-tolerance analysis and suggestion system is absent from this otherwise robust cloud management system. In this work, we demonstrate a system for automated fault-tolerance (FT) analysis of network applications hosted on OpenStack. Faults are modeled by random server and/or service shutdowns. Aim of the FT Analyzer is to equip cloud tenants with information to determine optimum configuration for their applications before firing up the service. FT Analyzer helps tenants maintain maximum up-time for services and data and preemptively tests the virtual network based applications for any issues that can have a negative impact on performance in post deployment scenario.

Scheduling-free resource management

We investigate a resource management policy that allocates resources based on the number of available resources only. We formulate a condition on resource requesting processes, called solidity that guarantees successful termination. Processes that do not satisfy this condition can be modified to become solid. We investigate performance of the resource management policy proposed by comparing it to the theoretically found optimum for the special case. Our method can be applied in, for example, project management systems, orchestration software and workflow engines.