Reproducible Workflows Research Articles

3-D Reconstruction of Rock Samples via Structure-From-Motion for Virtual Reality Applications: A Methodological Proposal

This article presents a methodological proposal for the three-dimensional reconstruction of rock samples via structure-from-motion. The presented methodological steps aimed to provide a reproducible workflow to create virtual rock samples to be applied in virtual applications. The proposed methodology works as a how-to guide as well as a preemptive troubleshooting guide for the complete process. Four geologists with different scholar levels volunteered to test this methodological proposal, applying it to three rock samples as the methodology steps were provided in an inverse-proportional manner to the graduate level. When analyzing the results of the performed reconstructions, all analyzed elements presented a proportional reduction due to the lack of information provided. An initial questionnaire was applied to verify the difficulties encountered, and subsequently, all volunteers received the complete methodology. In the second reconstruction, the results were equivalent to those obtained initially with the complete methodology. A technology acceptance model questionnaire was applied to determine the perception of utility and ease of use of the presented methodology. In both cases the results presented themselves in a positive way, indicating that the methodology was able to solve the problems found simply and objectively through a repeatable workflow.

Model structure and ensemble size: Implications for predictions of groundwater age

This paper examines the influence of simplified vertical discretization using 50- to four- layer models and ensemble size on history matching and predictions of groundwater age for a national scale model of New Zealand (approximately 265,000 km2). A reproducible workflow using a combination of opensource tools and custom python scripts is used to generate three models that use the same model domain and underlying data with only the vertical discretization changing between the models. The iterative ensemble smoother approach is used for history matching each model to the same synthetic dataset. The results show that: 1) the ensemble based mean objective function is not a good indicator of model predictive ability, 2) predictive failure from model structural errors in the simplified models are compounded by history matching, especially when small (<100 member) ensembles are used, 3) predictive failure rates increase with iteration, 4) predictive failure rates for the simplified model reach 30–65% using 50-member ensembles, but stabilize at relatively low values (<10%) using the 300 member ensemble, 5) small (50 member) ensembles contribute to predictive failure of 22–30% after six iterations even in structurally “perfect” models, 6) correlation-based localization methods can help reduce prediction failure associated with small ensembles by up to 45%, 7) the deleterious effects of model simplification and ensemble size are problem specific. Systematic investigation of these issues is an important part of the model design, and this investigation process benefits greatly from a scripted, reproducible workflow using flexible, opensource tools.

Workflow for health-related and brain data lifecycle.

Poor lifestyle leads potentially to chronic diseases and low-grade physical and mental fitness. However, ahead of time, we can measure and analyze multiple aspects of physical and mental health, such as body parameters, health risk factors, degrees of motivation, and the overall willingness to change the current lifestyle. In conjunction with data representing human brain activity, we can obtain and identify human health problems resulting from a long-term lifestyle more precisely and, where appropriate, improve the quality and length of human life. Currently, brain and physical health-related data are not commonly collected and evaluated together. However, doing that is supposed to be an interesting and viable concept, especially when followed by a more detailed definition and description of their whole processing lifecycle. Moreover, when best practices are used to store, annotate, analyze, and evaluate such data collections, the necessary infrastructure development and more intense cooperation among scientific teams and laboratories are facilitated. This approach also improves the reproducibility of experimental work. As a result, large collections of physical and brain health-related data could provide a robust basis for better interpretation of a person's overall health. This work aims to overview and reflect some best practices used within global communities to ensure the reproducibility of experiments, collected datasets and related workflows. These best practices concern, e.g., data lifecycle models, FAIR principles, and definitions and implementations of terminologies and ontologies. Then, an example of how an automated workflow system could be created to support the collection, annotation, storage, analysis, and publication of findings is shown. The Body in Numbers pilot system, also utilizing software engineering best practices, was developed to implement the concept of such an automated workflow system. It is unique just due to the combination of the processing and evaluation of physical and brain (electrophysiological) data. Its implementation is explored in greater detail, and opportunities to use the gained findings and results throughout various application domains are discussed.

Three levels of reproducible workflow remove barriers for archaeologists and increase accessibility

Three levels of reproducible workflow remove barriers for archaeologists and increase accessibility

Spatial consistency in drivers of population dynamics of a declining migratory bird.

Many migratory species are in decline across their geographical ranges. Single-population studies can provide important insights into drivers at a local scale, but effective conservation requires multi-population perspectives. This is challenging because relevant data are often hard to consolidate, and state-of-the-art analytical tools are typically tailored to specific datasets. We capitalized on a recent data harmonization initiative (SPI-Birds) and linked it to a generalized modelling framework to identify the demographic and environmental drivers of large-scale population decline in migratory pied flycatchers (Ficedula hypoleuca) breeding across Britain. We implemented a generalized integrated population model (IPM) to estimate age-specific vital rates, including their dependency on environmental conditions, and total and breeding population size of pied flycatchers using long-term (34-64 years) monitoring data from seven locations representative of the British breeding range. We then quantified the relative contributions of different vital rates and population structure to changes in short- and long-term population growth rate using transient life table response experiments (LTREs). Substantial covariation in population sizes across breeding locations suggested that change was the result of large-scale drivers. This was supported by LTRE analyses, which attributed past changes in short-term population growth rates and long-term population trends primarily to variation in annual survival and dispersal dynamics, which largely act during migration and/or nonbreeding season. Contributions of variation in local reproductive parameters were small in comparison, despite sensitivity to local temperature and rainfall within the breeding period. We show that both short- and long-term population changes of British breeding pied flycatchers are likely linked to factors acting during migration and in nonbreeding areas, where future research should be prioritized. We illustrate the potential of multi-population analyses for informing management at (inter)national scales and highlight the importance of data standardization, generalized and accessible analytical tools, and reproducible workflows to achieve them.

Tools and Recommendations for Reproducible Teaching

It is recommended that teacher-scholars of data science adopt reproducible workflows in their research as scholars and teach reproducible workflows to their students. In this article, we propose a third dimension to reproducibility practices and recommend that regardless of whether they teach reproducibility in their courses or not, data science instructors adopt reproducible workflows for their own teaching. We consider computational reproducibility, documentation, and openness as three pillars of reproducible teaching framework. We share tools, examples, and recommendations for the three pillars.

Enhancing RDM in Galaxy by integrating RO-Crate

Enhancing RDM in Galaxy by integrating RO-Crate

LUSTRE: An Online Data Management and Student Project Resource

We advocate for greater emphasis in training students about data management, within the context of supporting experience in reproducible workflows. We introduce the Lancaster University STatistics REsources (LUSTRE) package, used to manage student research project data in psychology and build capacity with respect to data acumen. LUSTRE provides a safe space to engage students with open research practices—by making tangible different phases of the reproducible research pipeline, while emphasizing its value as a transferable skill. It is an open-source online data catalogue that captures key data management information about a student research project of potential relevance to data scientists. Embedded within a taught programme, it also highlights concepts and examples of data management processes. We document a portfolio of open teaching resources for LUSTRE, and consider how others can implement or adapt them to facilitate data management and open research. We discuss the role of LUSTRE as a; (a) resource and set of activities for promoting good data management practices; (b) framework to enable the delivery of key concepts in open research; (c) an online system to organize and showcase project work.

Passive Acoustic Monitoring as a Tool to Investigate the Spatial Distribution of Invasive Alien Species

Invasive alien species (IAS) are a threat to biodiversity and ecosystem function worldwide. Unfortunately, researchers, agencies, and other management groups face the unresolved challenge of effectively detecting and monitoring IAS at large spatial and temporal scales. To improve the detection of soniferous IAS, we introduced a pipeline for large-scale passive acoustic monitoring (PAM). Our main goal was to illustrate how PAM can be used to rapidly provide baseline information on soniferous IAS. To that aim, we collected acoustic data across Puerto Rico from March to June 2021 and used single-species occupancy models to investigate species distribution of species in the archipelago and to assess the peak of vocal activity. Overall, we detected 16 IAS (10 birds, 3 mammals, and 3 frogs) and 79 native species in an extensive data set with 1,773,287 1-min recordings. Avian activity peaked early in the morning (between 5 a.m. and 7 a.m.), while amphibians peaked between 1 a.m. and 5 a.m. Occupancy probability for IAS in Puerto Rico ranged from 0.002 to 0.67. In general, elevation and forest cover older than 54 years were negatively associated with IAS occupancy, corroborating our expectation that IAS occurrence is related to high levels of human disturbance and present higher occupancy probabilities in places characterized by more intense human activities. The work presented here demonstrates that PAM is a workable solution for monitoring vocally active IAS over a large area and provides a reproducible workflow that can be extended to allow for continued monitoring over longer timeframes.

PING‐Mapper: Open‐Source Software for Automated Benthic Imaging and Mapping Using Recreation‐Grade Sonar

AbstractThe characterization of benthic habitats is essential for aquatic ecosystem science and management, but is frequently limited by waterbody visibility and depth. Recreation‐grade side‐scan sonar systems are increasingly used to aid scientific inquiries in aquatic environments due to their relative low‐cost, ease of operation, low‐weight, and ease of mounting on a variety of vessels. However, existing procedures and software for post‐processing these data are either limited, closed source, or fail on data from new sonar models, limiting the development of reproducible workflows. Here, we present PING‐Mapper, an open‐source and freely available side‐scan sonar post‐processing toolset for processing and mapping sonar recordings from popular Humminbird instruments. The modular software automatically: (a) decodes sonar recordings from any Humminbird system, (b) exports ping attributes from every sonar channel, (c) uses sonar sensor depth for water column removal, and (d) exports sonogram tiles and georectified mosaics. Sonar channels are processed in parallel for quick decoding and metadata extraction. Major processing wokflows, including georectification and image export, are optimized to scale with computing resources. The software has been extensively tested using data from several rivers of varying character and distribution of depths, but could also be used in estuarine and lacustrine environments. Usage of PING‐Mapper is illustrated in three case studies focused on mapping large woody debris, bathymetric mapping, and visual interpretation and mapping of substrates for selected reaches of the Pearl and Pascagoula river systems in Mississippi.

Curation at the point of measurement and traceability of measurement workflows

In this paper we introduce a method that can digitally capture machine actionable metadata, tag them to the associated measurement data, and upload to a curated database. Our method is packaged as a tool to enable scientists to capture and store curated data at the point of measurement. By ‘data’ we include the primary measurement and any associated information such as calibration data, processing/analysis scripts, multi-modal data, etc. Combining the associated data together enhances re-usability through metadata and confidence though calibration data. We extend this process by adding new data at each stage of the data capture and analysis workflow to develop a completely traceable data processing pipeline. We achieve this by cumulatively updating the ‘data’ at each stage and by using versioning in our database for complete generality. Here each version is a self-contained curated container of all relevant data and codes providing a reproducible ‘snapshot’ in a traceable analytical pipeline. Within each ‘snapshot’ we store the outputs from the relevant data analysis (figures, models, hypothesis tests, etc), the raw data, and each step (codes, converters, etc) between them, resulting in a fully transparent and reproducible workflow. The ‘snapshots’ are updated along the analytical pipeline and we demonstrate this with several steps including: at the point of measurement; conversion to an open format; pre-processing (feature selection, noise reduction, etc); and data analysis. We demonstrate our method with a large cohort of mass spectrometry imaging experiments as an exemplar case study.

TidyMass an object-oriented reproducible analysis framework for LC–MS data

Reproducibility, traceability, and transparency have been long-standing issues for metabolomics data analysis. Multiple tools have been developed, but limitations still exist. Here, we present the tidyMass project (https://www.tidymass.org/), a comprehensive R-based computational framework that can achieve the traceable, shareable, and reproducible workflow needs of data processing and analysis for LC-MS-based untargeted metabolomics. TidyMass is an ecosystem of R packages that share an underlying design philosophy, grammar, and data structure, which provides a comprehensive, reproducible, and object-oriented computational framework. The modular architecture makes tidyMass a highly flexible and extensible tool, which other users can improve and integrate with other tools to customize their own pipeline.

Reproducible workflow used for " Synechococcus and heterotrophic bacteria show inherent tendency towards mutualism during long-term coexistence despite environmental interference"article in Science Advances

Reproducible workflow used for " Synechococcus and heterotrophic bacteria show inherent tendency towards mutualism during long-term coexistence despite environmental interference"article in Science Advances

Reproducible workflow used for " Synechococcus and heterotrophic bacteria show inherent tendency towards mutualism during long-term coexistence despite environmental interference"article in Science Advances

Reproducible workflow used for " Synechococcus and heterotrophic bacteria show inherent tendency towards mutualism during long-term coexistence despite environmental interference"article in Science Advances

A Journey from Wild to Textbook Data to Reproducibly Refresh the Wages Data from the National Longitudinal Survey of Youth Database

Textbook data is essential for teaching statistics and data science methods because it is clean, allowing the instructor to focus on methodology. Ideally textbook datasets are refreshed regularly, especially when they are subsets taken from an ongoing data collection. It is also important to use contemporary data for teaching, to imbue the sense that the methodology is relevant today. This article describes the trials and tribulations of refreshing a textbook dataset on wages, extracted from the National Longitudinal Survey of Youth (NLSY79) in the early 1990s. The data is useful for teaching modeling and exploratory analysis of longitudinal data. Subsets of NLSY79, including the wages data, can be found in supplementary materials from numerous textbooks and research articles. The NLSY79 database has been continually updated through to 2018, so new records are available. Here we describe our journey to refresh the wages data, and document the process so that the data can be regularly updated into the future. Our journey was difficult because the steps and decisions taken to get from the raw data to the wages textbook subset have not been clearly articulated. We have been diligent to provide a reproducible workflow for others to follow, which also hopefully inspires more attempts at refreshing data for teaching. Three new datasets and the code to produce them are provided in the open source R package called yowie. Supplementary materials for this article are available online.

Abstract 1912: A versatile computational pipeline for the preprocessing of cell-free DNA fragmentation data

Abstract Cell-free DNA (cfDNA) emerges as a promising liquid biopsy biomarker for cancer diagnosis and patient monitoring. Complementing mutation-based assays, cfDNA carries information about epigenetic modifications from decaying cells. This information is encoded in the shape of the cfDNA fragments. Specifically, fragments from cancer tend to be shorter than those originating from other adult cells, enabling a distinction between cancer patients and healthy individuals. Additional cfDNA features such as fragment end motifs and information on nucleosome positioning provide further insight into cancer biology. These cfDNA measures are typically inferred from low-pass whole genome sequencing and subsequent bioinformatics processing. A key bioinformatics step is the alignment of DNA sequencing reads to the reference genome, which critically depends on preprocessing steps such as read trimming and alignment filters. A good understanding of preprocessing settings is thus crucial to derive accurate information from cfDNA. We therefore investigated to what extent preprocessing choices affect cfDNA analysis. To this end, we have built a robust bioinformatics pipeline that evaluates a range of possible preprocessing settings. The pipeline is implemented in a bioinformatics workflow engine, enabling a scalable and reproducible workflow. We are currently evaluating the effect of preprocessing on global and regional fragmentation patterns, detection of nucleosome positioning and identification of fragment end motifs. The different preprocessing settings are also evaluated for their ability to distinguish cancer from control samples using cfDNA fragmentation information. We are benchmarking the pipeline using cfDNA datasets from multiple centres to ensure our findings are generalizable over different platforms and experimental procedures. Our investigations will allow us to build a versatile bioinformatics preprocessing pipeline for the analysis of cell-free DNA fragmentation data. Citation Format: Ivna Ivankovic, Zsolt Balázs, Todor Gitchev, Daniel Trejo Banos, Norbert Moldovan, Panagiotis Balermpas, Jonas Willmann, Nicolaus Andratschke, Florent Moulière, Michael Krauthammer. A versatile computational pipeline for the preprocessing of cell-free DNA fragmentation data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1912.

Abstract 1900: Rapid variant detection and annotations from next generation sequencing data using a GPU accelerated framework

Abstract Introduction Next Generation Sequencing (NGS) played a crucial role in revolutionizing the field of human genetics to become an integral part of research and clinical management. Some examples are diagnosis and treatment of cancer patients, identifying syndromes in the NICU (Neonatal Intensive Care Units), insights into neuro-degenerative disease to name a few. NGS integrated with multi-omics data provides greater insight to underlying molecular mechanisms and helps to identify alterations specific to the disease condition. Accurate and robust detection of somatic and germline genomic alterations from whole genome sequencing (WGS) data is complex where the datasets tend to be very large, and they require highly compute intensive operations that often take days to weeks to complete. By leveraging the Graphical Processing Unit (GPU) accelerated framework, it is possible to analyze the WGS data and identify clinically actionable variants with fast turnaround times without compromising on the accuracy. In this study, we implemented a GPU accelerated framework for most used variant callers ie. Mutect2, MuSE, Somatic Sniper and LoFreq with significantly faster run times in comparison to their Central Processing Unit (CPU) counterpart. Methods Commonly used CPU based somatic variant calling algorithms Mutect2, MuSE, Somatic Sniper and LoFreq, were selected and subsequently accelerated to run on GPUs. The goal of this work is to generate results that are equivalent to the CPU version of these algorithms, but at much faster run times. As a control data set, we utilize the SEQC2 consortium WGS dataset consisting of ~50x tumor and normal samples to assess the performance of individual CPU variant callers to their GPU equivalents. Results The GPU accelerated framework overall provides 10-66x faster run times compared to their CPU-based counterparts. Individually, CPU vs GPU comparisons are, Mutect2 achieved a maximum acceleration of 66x, followed by MuSE, LoFreq both 11x and Somatic Sniper 10x acceleration respectively. The results from accelerated MuSE, LoFreq, Somatic sniper are identical in comparison to the CPU-based somatic variant callers, For Mutect2 we achieve 99.9% sensitivity and 99.85% precision. Discussion Recently, machine-learning based variant detection algorithms exhibited better performance compared to the statistics based heuristic approaches. Here, we have presented the results for an ensemble of four variant callers to be used as the starting point for an ongoing effort to develop a deep learning-based approach for somatic variant detection and smart filtering algorithms. The goal is to enable improved variant calling and provide a GPU computing framework for faster, better, and reproducible workflows to analyze large scale NGS datasets. Citation Format: Pankaj Vats, Ankit Sethia, Mehrzad Samadi, Timothy T. Harkins. Rapid variant detection and annotations from next generation sequencing data using a GPU accelerated framework [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1900.

VisU-HydRA: A Computational Toolbox for Groundwater Contaminant Transport to Support Risk-Based Decision Making

Obtaining accurate and deterministic predictions of the risks associated with the presence of contaminants in aquifers is an illusive goal given the presence of heterogeneity in hydrological properties and limited site characterization data. For such reasons, a probabilistic framework is needed to quantify the risks in groundwater systems. In this work, we present a computational toolbox VisU-HydRA that aims to statistically characterize and visualize metrics that are relevant in risk analysis with the ultimate goal of supporting decision making. The VisU-HydRA computational toolbox is an open-source Python package that can be linked to a series of existing codes such as MODFLOW and PAR2, a GPU-accelerated transport simulator. To illustrate the capabilities of the computational toolbox, we simulate flow and transport in a heterogeneous aquifer within a Monte Carlo framework. The computational toolbox allows to compute the probability of a contaminant’s concentration exceeding a safe threshold value as well as the uncertainty associated with the loss of resilience of the aquifer. To ensure consistency and a reproducible workflow, a step-by-step tutorial is provided and available on a GitHub repository.

ScAmpi-A versatile pipeline for single-cell RNA-seq analysis from basics to clinics.

Single-cell RNA sequencing (scRNA-seq) has emerged as a powerful technique to decipher tissue composition at the single-cell level and to inform on disease mechanisms, tumor heterogeneity, and the state of the immune microenvironment. Although multiple methods for the computational analysis of scRNA-seq data exist, their application in a clinical setting demands standardized and reproducible workflows, targeted to extract, condense, and display the clinically relevant information. To this end, we designed scAmpi (Single Cell Analysis mRNA pipeline), a workflow that facilitates scRNA-seq analysis from raw read processing to informing on sample composition, clinically relevant gene and pathway alterations, and in silico identification of personalized candidate drug treatments. We demonstrate the value of this workflow for clinical decision making in a molecular tumor board as part of a clinical study.

Can Forel–Ule Index Act as a Proxy of Water Quality in Temperate Waters? Application of Plume Mapping in Liverpool Bay, UK

The use of ocean colour classification algorithms, linked to water quality gradients, can be a useful tool for mapping river plumes in both tropical and temperate systems. This approach has been applied in operational water quality programs in the Great Barrier Reef to map river plumes and assess trends in marine water composition and ecosystem health during flood periods. In this study, we used the Forel–Ule colour classification algorithm for Sentinel-3 OLCI imagery in an automated process to map monthly, annual and long-term plume movement in the temperate coastal system of Liverpool Bay (UK). We compared monthly river plume extent to the river flow and in situ water quality data between 2017–2020. The results showed a strong positive correlation (Spearman’s rho = 0.68) between the river plume extent and the river flow and a strong link between the FUI defined waterbodies and nutrients, SPM, turbidity and salinity, hence the potential of the Forel–Ule index to act as a proxy for water quality in the temperate Liverpool Bay water. The paper discusses how the Forel–Ule index could be used in operational water quality programs to better understand river plumes and the land-based inputs to the coastal zones in UK waters, drawing parallels with methods that have been developed in the GBR and Citclops project. Overall, this paper provides the first insight into the systematic long-term river plume mapping in UK coastal waters using a fast, cost-effective, and reproducible workflow. The study created a novel water assessment typology based on the common physical, chemical and biological ocean colour properties captured in the Forel–Ule index, which could replace the more traditional eutrophication assessment regions centred around strict geographic and political boundaries. Additionally, the Forel–Ule assessment typology is particularly important since it identifies areas of the greatest impact from the land-based loads into the marine environment, and thus potential risks to vulnerable ecosystems.