Workflow Description Language Research Articles

TheiaEuk: a species-agnostic bioinformatics workflow for fungal genomic characterization.

The clinical incidence of antimicrobial-resistant fungal infections has dramatically increased in recent years. Certain fungal pathogens colonize various body cavities, leading to life-threatening bloodstream infections. However, the identification and characterization of fungal isolates in laboratories remain a significant diagnostic challenge in medicine and public health. Whole-genome sequencing provides an unbiased and uniform identification pipeline for fungal pathogens but most bioinformatic analysis pipelines focus on prokaryotic species. To this end, TheiaEuk_Illumina_PE_PHB (TheiaEuk) was designed to focus on genomic analysis specialized to fungal pathogens. TheiaEuk was designed using containerized components and written in the workflow description language (WDL) to facilitate deployment on the cloud-based open bioinformatics platform Terra. This species-agnostic workflow enables the analysis of fungal genomes without requiring coding, thereby reducing the entry barrier for laboratory scientists. To demonstrate the usefulness of this pipeline, an ongoing outbreak of C. auris in southern Nevada was investigated. We performed whole-genome sequence analysis of 752 new C. auris isolates from this outbreak. Furthermore, TheiaEuk was utilized to observe the accumulation of mutations in the FKS1 gene over the course of the outbreak, highlighting the utility of TheiaEuk as a monitor of emerging public health threats when combined with whole-genome sequencing surveillance of fungal pathogens. A primary result of this work is a curated fungal database containing 5,667 unique genomes representing 245 species. TheiaEuk also incorporates taxon-specific submodules for specific species, including clade-typing for Candida auris (C. auris). In addition, for several fungal species, it performs dynamic reference genome selection and variant calling, reporting mutations found in genes currently associated with antifungal resistance (FKS1, ERG11, FUR1). Using genome assemblies from the ATCC Mycology collection, the taxonomic identification module used by TheiaEuk correctly assigned genomes to the species level in 126/135 (93.3%) instances and to the genus level in 131/135 (97%) of instances, and provided zero false calls. Application of TheiaEuk to actual specimens obtained in the course of work at a local public health laboratory resulted in 13/15 (86.7%) correct calls at the species level, with 2/15 called at the genus level. It made zero incorrect calls. TheiaEuk accurately assessed clade type of Candida auris in 297/302 (98.3%) of instances. TheiaEuk demonstrated effectiveness in identifying fungal species from whole genome sequence. It further showed accuracy in both clade-typing of C. auris and in the identification of mutations known to associate with drug resistance in that organism.

Sapporo: A workflow execution service that encourages the reuse of workflows in various languages in bioinformatics

The increased demand for efficient computation in data analysis encourages researchers in biomedical science to use workflow systems. Workflow systems, or so-called workflow languages, are used for the description and execution of a set of data analysis steps. Workflow systems increase the productivity of researchers, specifically in fields that use high-throughput DNA sequencing applications, where scalable computation is required. As systems have improved the portability of data analysis workflows, research communities are able to share workflows to reduce the cost of building ordinary analysis procedures. However, having multiple workflow systems in a research field has resulted in the distribution of efforts across different workflow system communities. As each workflow system has its unique characteristics, it is not feasible to learn every single system in order to use publicly shared workflows. Thus, we developed Sapporo, an application to provide a unified layer of workflow execution upon the differences of various workflow systems. Sapporo has two components: an application programming interface (API) that receives the request of a workflow run and a browser-based client for the API. The API follows the Workflow Execution Service API standard proposed by the Global Alliance for Genomics and Health. The current implementation supports the execution of workflows in four languages: Common Workflow Language, Workflow Description Language, Snakemake, and Nextflow. With its extensible and scalable design, Sapporo can support the research community in utilizing valuable resources for data analysis.

STAAR workflow: a cloud-based workflow for scalable and reproducible rare variant analysis.

We developed the variant-Set Test for Association using Annotation infoRmation (STAAR) workflow description language (WDL) workflow to facilitate the analysis of rare variants in whole genome sequencing association studies. The open-access STAAR workflow written in the WDL allows a user to perform rare variant testing for both gene-centric and genetic region approaches, enabling genome-wide, candidate and conditional analyses. It incorporates functional annotations into the workflow as introduced in the STAAR method in order to boost the rare variant analysis power. This tool was specifically developed and optimized to be implemented on cloud-based platforms such as BioData Catalyst Powered by Terra. It provides easy-to-use functionality for rare variant analysis that can be incorporated into an exhaustive whole genome sequencing analysis pipeline. The workflow is freely available from https://dockstore.org/workflows/github.com/sheilagaynor/STAAR_workflow. Supplementary data are available at Bioinformatics online.

"Escalibur"-A practical pipeline for the de novo analysis of nucleotide variation in nonmodel eukaryotes.

The revolution in genomics has enabled large‐scale population genetic investigations of a wide range of organisms, but there has been a relatively limited focus on improving analytical pipelines. To efficiently analyse large data sets, highly integrated and automated software pipelines, which are easy to use, efficient, reliable, reproducible and run in multiple computational environments, are required. A number of software workflows have been developed to handle and process such data sets for population genetic analyses, but effective, specialized pipelines for genetic and statistical analyses of nonmodel organisms are lacking. For most species, resources for variomes (sets of genetic variations found in populations of species) are not available, and/or genome assemblies are often incomplete and fragmented, complicating the selection of the most suitable reference genome when multiple assemblies are available. Additionally, the biological samples used often contain extraneous DNA from sources other than the species under investigation (e.g., microbial contamination), which needs to be removed prior to genetic analyses. For these reasons, we established a new pipeline, called Escalibur, which includes: functionalities, such as data trimming and mapping; selection of a suitable reference genome; removal of contaminating read data; recalibration of base calls; and variant‐calling. Escalibur uses a proven gatk variant caller and workflow description language (WDL), and is, therefore, a highly efficient and scalable pipeline for the genome‐wide identification of nucleotide variation in eukaryotes. This pipeline is available at https://gitlab.unimelb.edu.au/bioscience/escalibur (version 0.3‐beta) and is essentially applicable to any prokaryote or eukaryote.

Hummingbird: efficient performance prediction for executing genomic applications in the cloud.

A major drawback of executing genomic applications on cloud computing facilities is the lack of tools to predict which instance type is the most appropriate, often resulting in an over- or under- matching of resources. Determining the right configuration before actually running the applications will save money and time. Here, we introduce Hummingbird, a tool for predicting performance of computing instances with varying memory and CPU on multiple cloud platforms. Our experiments on three major genomic data pipelines, including GATK HaplotypeCaller, GATK Mutect2 and ENCODE ATAC-seq, showed that Hummingbird was able to address applications in command line specified in JSON format or workflow description language (WDL) format, and accurately predicted the fastest, the cheapest and the most cost-efficient compute instances in an economic manner. Hummingbird is available as an open source tool at: https://github.com/StanfordBioinformatics/Hummingbird. Supplementary data are available at Bioinformatics online.

Tibanna: software for scalable execution of portable pipelines on the cloud

We introduce Tibanna, an open-source software tool for automated execution of bioinformatics pipelines on Amazon Web Services (AWS). Tibanna accepts reproducible and portable pipeline standards including Common Workflow Language (CWL), Workflow Description Language (WDL) and Docker. It adopts a strategy of isolation and optimization of individual executions, combined with a serverless scheduling approach. Pipelines are executed and monitored using local commands or the Python Application Programming Interface (API) and cloud configuration is automatically handled. Tibanna is well suited for projects with a range of computational requirements, including those with large and widely fluctuating loads. Notably, it has been used to process terabytes of data for the 4D Nucleome (4DN) Network. Source code is available on GitHub at https://github.com/4dn-dcic/tibanna. Supplementary data are available at Bioinformatics online.

VCPA: genomic variant calling pipeline and data management tool for Alzheimer's Disease Sequencing Project.

SummaryWe report VCPA, our SNP/Indel Variant Calling Pipeline and data management tool used for the analysis of whole genome and exome sequencing (WGS/WES) for the Alzheimer’s Disease Sequencing Project. VCPA consists of two independent but linkable components: pipeline and tracking database. The pipeline, implemented using the Workflow Description Language and fully optimized for the Amazon elastic compute cloud environment, includes steps from aligning raw sequence reads to variant calling using GATK. The tracking database allows users to view job running status in real time and visualize >100 quality metrics per genome. VCPA is functionally equivalent to the CCDG/TOPMed pipeline. Users can use the pipeline and the dockerized database to process large WGS/WES datasets on Amazon cloud with minimal configuration.Availability and implementationVCPA is released under the MIT license and is available for academic and nonprofit use for free. The pipeline source code and step-by-step instructions are available from the National Institute on Aging Genetics of Alzheimer’s Disease Data Storage Site (http://www.niagads.org/VCPA).Supplementary information Supplementary data are available at Bioinformatics online.

Analysis Preservation and Systematic Reinterpretation within the ATLAS experiment

The LHC data analysis software used in order to derive and publish experimental results is an important asset that is necessary to preserve in order to fully exploit the scientific potential of a given measurement. An important use-case is the re-usability of the analysis procedure in the context of new scientific studies such as the reinterpretation of searches for new physics in terms of signal models that not studied in the original publication (RECAST). We present the usage of the graph-based workflow description language yadage to drive the reinterpretation of preserved HEP analyses. The analysis software is preserved using Docker containers, while the workflow structure is preserved using plain JSON documents. This allows the re-execution of complex analysis workflows on modern distributed container orchestration systems and enables a systematic reinterpretation service based on such preserved analysis.

DEFINING EARTH DATA BATCH PROCESSING TASKS BY MEANS OF A FLEXIBLE WORKFLOWDESCRIPTION LANGUAGE

Abstract. This paper aims to present some of the main features of the Workflow Description Language (WorDeL) and demonstrate their usage in defining Earth Data processing tasks. This description language is based on the flexible description of processing tasks as workflows, composed of basic processing operators. This approach allows the language to offer an intuitive way of representing processing tasks, without requiring programming expertise from its users. It also allows its users to employ and integrate existing functionality into their design, thereby reducing the design complexity and development effort of newly defined processing workflows. WorDeL supports the transparent adaptive parallelization of the processing tasks over high performance computation architectures, such as cloud-based solutions. Throughout the paper, we will exemplify this language’s use in creating flexible, reusable and easy-to-understand earth data processing descriptions, with an emphasis on satellite image processing.

DEFINING EARTH DATA BATCH PROCESSING TASKS BY MEANS OF A FLEXIBLE WORKFLOWDESCRIPTION LANGUAGE

This paper aims to present some of the main features of the Workflow Description Language (WorDeL) and demonstrate their usage in defining Earth Data processing tasks. This description language is based on the flexible description of processing tasks as workflows, composed of basic processing operators. This approach allows the language to offer an intuitive way of representing processing tasks, without requiring programming expertise from its users. It also allows its users to employ and integrate existing functionality into their design, thereby reducing the design complexity and development effort of newly defined processing workflows. WorDeL supports the transparent adaptive parallelization of the processing tasks over high performance computation architectures, such as cloud-based solutions. Throughout the paper, we will exemplify this language’s use in creating flexible, reusable and easy-to-understand earth data processing descriptions, with an emphasis on satellite image processing.

An open source infrastructure for quality assurance and preservation of a large digital book collection

This article presents an open source infrastructure for processing large collections of digital books available at the Austrian National Library with a special focus on quality assurance tasks in the context of the European project SCAPE (www.scapeproject-eu). It describes the cluster hardware and the software components used for building the experimental IT infrastructure.More concretely, a set of best practices for the data analysis of large document image collections on the basis of Apache Hadoop will be shown. Different types of Hadoop jobs (Hadoop-Streaming-API, Hadoop MapReduce, and Hive) are used as basic components, and the Taverna workflow description language and execution engine (www.taverna.org.uk) is used for orchestrating complex data processing tasks.

Workflow management for soft real-time interactive applications in virtualized environments

Many applications, especially the ones implementing multi-user collaborative environments, fall within the context of soft real-time systems in which only small deviations from timing constraints are allowed. The advancements in distributed computing have made it possible to follow a service oriented approach, taking advantage of the benefits this provides. In this context, applications consist of soft real-time critical application service components that interact with each other to provide the corresponding application functionality, forming application workflows. In this paper we present a new architectural design and implementation of a Workflow Management approach. This approach covers enacting soft real-time application service components according to a workflow description language, synchronizing the application components, monitoring the execution and reacting to events within a distributed virtualized environment. We also demonstrate the operation of the implemented mechanism and evaluate its effectiveness using an application scenario with soft real-time interactivity characteristics, namely Film post-production, under realistic settings.

ZFlow: Workflow for Cooperative Editing System

Enforcement of global cooperation urges the rapid growth of cooperative work. As a typical application of it, cooperative editing is widely used in many fields gradually, especially in knowledge intensive application. Workflow is a kind of automation technology by predefined process rules and tasks. It can bring us clarity in process control and management, which are two important facets in cooperative editing. Marriage of workflow and cooperation shows us an approach to achieve automation and convenient management as well as collaboration, thus gaining efficiency and regularity. We summarize the editing process flow as iterative phases including initializing, editing, coediting, reviewing and finalizing, which lays the foundation of incorporating workflow with cooperative editing. An activity-role based access control model, which uses activity to represent the editing task and uses role to effectively manage participants, is designed and applied in the workflow system. To get fine control grain, section, a logic part of the document, is used. We specially introduce the concept of subflow for section processing control and use section for cooperative editing control. To represent the workflow, we propose a workflow description language, which can be understood and processed by the workflow engine. As control logic in cooperative editing is quite different from that in individual editing, we put forward a novel workflow control algorithm, taking into account characteristics of cooperative editing. A temporal cooperative editing system applied the workflow system, showing it can deal with cooperative editing well, spare much human work, and offer clear process logic.

Dynamic Computational Workflows: Discovery, Optimisation and Scheduling

The Grid computing community is converging on a service-oriented architecture in which applications are composed from geographically-distributed, interacting web services, and are expressed in a workflow description language, usually based on XML. Such workflows are viewed as offering a useful representation of service-based applications or applications composed of standalone components that are to be run in a distributed environment. A workflow can be conveniently displayed as a directed graph in which nodes represent services and edges represent the flow of data and/or control between them. In some sense workflows represent “how the Grid is programmed”, and consequently much recent research has focussed on software and tools that support the use of workflows for grid computing. The semantic description of services, components and workflows, service and resource discovery and selection, and the scheduling of workflows present significant research challenges in making workflows of practical benefit to grid users. The solution of these challenges, in turn, depends on languages for workflow representation, enactment engines for workflow execution, and ontologies. This special issue arose mainly out of two international workshops:

웹 서비스 기반의 유비쿼터스 워크플로우 언어

Currently workflows in business processes and distributed computing environments have provided service automation by connecting many tasks with rules and/or orderings. The services in ubiquitous computing environments have to automatically provide users with adaptive services according to dynamically changing contort information, which is obtained from both the users and their environment. To adapt these workflows to ubiquitous computing, we must specify the situation information on the transition conditions. In this paper, we propose uWDL, Ubiquitous Workflow Description Language, based on Web Services to use the situation information on the transition constraints of workflow`s services. uWDL can select adaptive services according to contexts, profiles, and events information, which are situation information. Furthermore, we verified the execution of a scenario document described with uWDL using the parser and the context handler for uWDL. The scenario developers can use the uWDL scenario editor for a design of scenarios, and they can easily specify the transition condition of the services according the situation information of ubiquitous environments using the uWDL.

JISGA: A Jini-Based Service-Oriented Grid Architecture

This paper describes how to extend a Jini system into an OGSA-compliant infrastructure for Grid computing by introducing Web service techniques into the Jini system. The Jini-based Service-oriented Grid Architecture (JISGA) is presented, the functionality of the system is described, and the main services and components of the system are discussed. The paper shows how to present a Jini service as a Web service, and how an application composed of interacting services can be described using the service workflow description language (SWFL). The conversion of a composite application specified in an SWFL document into an executable Java code, and its subsequent execution, are also discussed. This latter capability, enabled by the SWFL2Java tool, permits the blocking and non-blocking submission of jobs to the Grid, and their processing in both sequential and parallel execution modes.