Open-source Software Infrastructure Research Articles

Software Application Profile: ShinyDataSHIELD—an R Shiny application to perform federated non-disclosive data analysis in multicohort studies

Abstract Motivation DataSHIELD is an open-source software infrastructure enabling the analysis of data distributed across multiple databases (federated data) without leaking individuals’ information (non-disclosive). It has applications in many scientific domains, ranging from biosciences to social sciences and including high-throughput genomic studies. R is the language used to interact with (and build) DataSHIELD. This creates difficulties for researchers who do not have experience writing R code or lack the time to learn how to use the DataSHIELD functions. To help new researchers use the DataSHIELD infrastructure and to improve the user-friendliness for experienced researchers, we present ShinyDataSHIELD. Implementation ShinyDataSHIELD is a web application with an R backend that serves as a graphical user interface (GUI) to the DataSHIELD infrastructure. General features The version of the application presented here includes modules to perform: (i) exploratory analysis through descriptive summary statistics and graphical representations (scatter plots, histograms, heatmaps and boxplots); (ii) statistical modelling (generalized linear fixed and mixed-effects models, survival analysis through Cox regression); (iii) genome-wide association studies (GWAS); and (iv) omic analysis (transcriptomics, epigenomics and multi-omic integration). Availability ShinyDataSHIELD is publicly hosted online [https://datashield-demo.obiba.org/], the source code and user guide are deposited on Zenodo DOI 10.5281/zenodo.6500323, freely available to non-commercial users under ‘Commons Clause’ License Condition v1.0. Docker images are also available [https://hub.docker.com/r/brgelab/shiny-data-shield].

Converting Numeral Text in Bulgarian into Digit Number Using GATE

The Internet serves billions of users providing a variety of information resources whereby a lot of the information is presented in natural human language and needs an efficient approach to be processed. Natural language processing (NLP) refers to the ability of computers to analyze and understand the structure of human language. By utilizing NLP this linguistic knowledge is transformed into algorithms for solving specific problems. GATE is widely used, open-source software infrastructure that provides a framework and components for solving NLP tasks. The available GATE tools can be adapted to other languages and text processing tasks. This article will present an approach for converting numeric data, written as words in Bulgarian, into digit numbers. For this case, a relevant configuration file for Bulgarian has been integrated into the general tool set in the open source software for natural language processing GATE. The aim of this survey is to determine the exact numeric value of Bulgarian text numeric data, which can be used as a starting point for producing more complex annotations, such as monetary measurement units, etc.

Fast-transient Searches in Real Time with ZTFReST: Identification of Three Optically Discovered Gamma-Ray Burst Afterglows and New Constraints on the Kilonova Rate

Abstract The most common way to discover extragalactic fast transients, which fade within a few nights in the optical, is via follow-up of gamma-ray burst and gravitational-wave triggers. However, wide-field surveys have the potential to identify rapidly fading transients independently of such external triggers. The volumetric survey speed of the Zwicky Transient Facility (ZTF) makes it sensitive to objects as faint and fast fading as kilonovae, the optical counterparts to binary neutron star mergers, out to almost 200 Mpc. We introduce an open-source software infrastructure, the ZTF REaltime Search and Triggering, ZTFReST, designed to identify kilonovae and fast transients in ZTF data. Using the ZTF alert stream combined with forced point-spread-function photometry, we have implemented automated candidate ranking based on their photometric evolution and fitting to kilonova models. Automated triggering, with a human in the loop for monitoring, of follow-up systems has also been implemented. In 13 months of science validation, we found several extragalactic fast transients independently of any external trigger, including two supernovae with post-shock cooling emission, two known afterglows with an associated gamma-ray burst (ZTF20abbiixp, ZTF20abwysqy), two known afterglows without any known gamma-ray counterpart (ZTF20aajnksq, ZTF21aaeyldq), and three new fast-declining sources (ZTF20abtxwfx, ZTF20acozryr, ZTF21aagwbjr) that are likely associated with GRB200817A, GRB201103B, and GRB210204A. However, we have not found any objects that appear to be kilonovae. We constrain the rate of GW170817-like kilonovae to R < 900 Gpc−3 yr−1 (95% confidence). A framework such as ZTFReST could become a prime tool for kilonova and fast-transient discovery with the Vera Rubin Observatory.

"They Can Only Ever Guide"

Unlike in commercial software development, open source software (OSS) projects do not generally have managers with direct control over how developers spend their time, yet for projects with large, diverse sets of contributors, the need exists to focus and steer development in a particular direction in a coordinated way. This is especially important for "infrastructure" projects, such as critical libraries and programming languages that many other people depend on. Some projects have taken the approach of borrowing planning tools that originated in commercial development, despite the fact that these techniques were designed for very different contexts, e.g. strong top-down control and profit motives. Little research has been done to understand how these practices are adapted to a new context. In this paper, we examine the Rust project's use of roadmaps: how has an important OSS infrastructure project adapted an inherently top-down tool to the freewheeling world of OSS? We find that because Rust's roadmaps are built in part by summarizing what motivated developers most prefer to work on, they are in some ways more a description of the motivated labor available than they are a directive that the community move in a particular direction. They allow the community to avoid wasting time on unpopular proposals by revealing that there will be little help in building them, and encouraging work on popular features by making visible the amount of consensus in those features. Roadmaps generate a collective focus without limiting the full scope of what developers work on: roadmap issues consume proportionally more effort than other issues, but constitute a minority of the work done (i.e issues and pull requests made) by both central and peripheral participants. They also create transparency among and beyond the community into what central contributors' plans are, and allow more rational decision-making by providing a way for evidence about community needs to be linked to decision-making.

Sustainable Interoperability among Biological Collections and their Computing Platforms

Integration and interoperability have been persistent drivers of biocollections software innovation for over 40 years—both within the biodiversity community and to the outside with external linkages to broader computing initiatives. Currently, global-scale "real-time" data integration is an aspirational dream, mostly constrained by cost and by being technically much more demanding than can generally be supported with current levels of investment and expertise. Today the collections community shuffles along relying on web portals for data discovery, and multi-step, batch transfers of species occurrence data for aggregation. With sufficient resources and collaboration, much more seamless integration and data fluidity are possible. The digital specimen architecture being planned by DiSSCo (The Distributed System of Scientific Collections) promises to re-invent the ways in which institutions curate, improve, and share their collections' data, by transforming the pipelines and methods with which specimen and related information will frictionlessly course among them. With robust integration of applications running on local systems (desktops, laptops, phones), that architecture will ultimately greatly advance the ease of rapidly assembling and updating custom distributed databases for research, education, and outreach. This presentation will touch on the future of collections community data integration and interoperability from the perspective of sustainability with allusions to the experience we have had with Specify Collections Consortium. Long-term revenue support models, actual financial stakeholders, widespread implementation of robust services behind Application Programming Interfaces (APIs), co-dependent collaborations on open source software infrastructure, unprecedented levels of technical coordination, and last but not least, quickly-evolving data standards will all be key resources needed to trek out of our current data landscape to reach the next plateau of biocollections computing.

Traffic Woes, Metro Manila, and Collaborative Problem-Solving: A Case Study of Computer-mediated, Collaboratively Built Information Infrastructure in the Field of Transportation in the Philippines

The chaotic traffic situation in Metro Manila has been characterized as a major roadblock to the country’s economic development and has turned into an important discussion point in political debates. In this paper three traffic related web services, aimed at helping their users gain an insight into the traffic situation of Metro Manila and beyond are analyzed in regards to their use of cooperation using Benkler’s concept of Collaborative Peer Production. The three web services differ starkly from each other in their concept – PH-Commute.com is a blog, Taxikick.com is a service for short messages pertaining to misbehaviors of taxi drivers, Sakay.ph is a navigation service. As I conclude, all three however share in common that they are indeed highly dependent on cooperation on different layers. Determined by the underlying concept of each of the websites, they incorporate inputs from their users, but they might also let their users help them in developing their software by publishing their source code, and they rely on community-created, open-source software infrastructure to be able to run their own.

Fully turbulent discrete adjoint solver for non-ideal compressible flow applications

Abstract Non-Ideal Compressible Fluid-Dynamics (NICFD) has recently been established as a sector of fluid mechanics dealing with the flows of dense vapors, supercritical fluids, and two-phase fluids, whose properties significantly depart from those of the ideal gas. The flow through an Organic Rankine Cycle (ORC) turbine is an exemplary application, as stators often operate in the supersonic and transonic regime, and are affected by NICFD effects. Other applications are turbomachinery using supercritical CO2 as working fluid or other fluids typical of the oil and gas industry, and components of air conditioning and refrigeration systems. Due to the comparably lower level of experience in the design of this fluid machinery, and the lack of experimental information on NICFD flows, the design of the main components of these processes (i.e., turbomachinery and nozzles) may benefit from adjoint-based automated fluid-dynamic shape optimization. Hence, this work is related to the development and testing of a fully-turbulent adjoint method capable of treating NICFD flows. The method was implemented within the SU2 open-source software infrastructure. The adjoint solver was obtained by linearizing the discretized flow equations and the fluid thermodynamic models by means of advanced Automatic Differentiation (AD) techniques. The new adjoint solver was tested on exemplary turbomachinery cases. Results demonstrate the method effectiveness in improving simulated fluid-dynamic performance, and underline the importance of accurately modeling non-ideal thermodynamic and viscous effects when optimizing internal flows influenced by NICFD phenomena.

Examining organizations’ continued use of open source technologies

PurposeOur knowledge of why organizations continue to use open source software (OSS) infrastructure technologies is relatively limited, and existing models appear inadequate to explain this continuance phenomenon given that they are set at the individual level and also do not take into account the unique characteristics of OSS. The paper aims to discuss these issues.Design/methodology/approachUsing an institutional perspective, this paper posits that coercive (business value of IT) and normative (open source ideology (IDEO)) factors may be credited with sustaining the continued use of OSS technologies. The study argues that organizations that subscribe to IDEO are more likely to continue using OSS technologies. Survey data are collected from organizations that have implemented an OSS infrastructure technology and a moderated multiple regression analysis is performed to test the proposed hypotheses.FindingsIn addition to the business value provided by OSS technologies, adherence to IDEO also impacts decisions to continue using those technologies. The results suggest that once an OSS is implemented in an organization, IDEO can enhance organizations’ intentions to continue using such technologies, directly, as well as indirectly, by amplifying the impact of the perceived business value of the technology.Originality/valueMuch of extant literature on continued use focuses on end-user technologies. This paper is one of the first to focus on infrastructure technologies and examine organizations’ intentions to continue using those technologies by developing a parsimonious theory-driven model for examining organizations’ continued use intentions toward infrastructure IT. Additionally, much of open source research to date has been inwardly focused, and this paper is one of few empirical studies to focus on the demand or consumption side of OSS technologies.

Adoption of OSS components: A goal-oriented approach

Open Source Software (OSS) has become a strategic asset for a number of reasons, such as short time-to-market software delivery, reduced development and maintenance costs, and its customization capabilities. Therefore, organizations are increasingly becoming OSS adopters, either as a result of a strategic decision or because it is almost unavoidable nowadays, given the fact that most commercial software also relies at some extent in OSS infrastructure. The way in which organizations adopt OSS affects and shapes their businesses. Therefore, knowing the impact of different OSS adoption strategies in the context of an organization may help improving the processes undertaken inside this organization and ultimately pave the road to strategic moves. In this paper, we propose to model OSS adoption strategies using a goal-oriented notation, in which different actors state their objectives and dependencies on each other. These models describe the consequences of adopting one such strategy or another: which are the strategic and operational goals that are supported, which are the resources that emerge, etc. The models rely on an OSS ontology, built upon a systematic literature review, which comprises the activities and resources that characterize these strategies. Different OSS adoption strategy models arrange these ontology elements in diverse ways. In order to assess which is the OSS adoption strategy that better fits the organization needs, the notion of model coverage is introduced, which allows to measure the degree of concordance among every strategy with the model of the organization by comparing the respective models. The approach is illustrated with an example of application in a big telecommunications company.

Enabling systematic, harmonised and large-scale biofilms data computation: The Biofilms Experiment Workbench

Background and objectiveBiofilms are receiving increasing attention from the biomedical community. Biofilm-like growth within human body is considered one of the key microbial strategies to augment resistance and persistence during infectious processes. The Biofilms Experiment Workbench is a novel software workbench for the operation and analysis of biofilms experimental data. The goal is to promote the interchange and comparison of data among laboratories, providing systematic, harmonised and large-scale data computation. MethodsThe workbench was developed with AIBench, an open-source Java desktop application framework for scientific software development in the domain of translational biomedicine. Implementation favours free and open-source third-parties, such as the R statistical package, and reaches for the Web services of the BiofOmics database to enable public experiment deposition. ResultsFirst, we summarise the novel, free, open, XML-based interchange format for encoding biofilms experimental data. Then, we describe the execution of common scenarios of operation with the new workbench, such as the creation of new experiments, the importation of data from Excel spreadsheets, the computation of analytical results, the on-demand and highly customised construction of Web publishable reports, and the comparison of results between laboratories. ConclusionsA considerable and varied amount of biofilms data is being generated, and there is a critical need to develop bioinformatics tools that expedite the interchange and comparison of microbiological and clinical results among laboratories. We propose a simple, open-source software infrastructure which is effective, extensible and easy to understand. The workbench is freely available for non-commercial use at http://sing.ei.uvigo.es/bew under LGPL license.

Data harmonization and federated analysis of population-based studies: the BioSHaRE project

sBackgroundIndividual-level data pooling of large population-based studies across research centres in international research projects faces many hurdles. The BioSHaRE (Biobank Standardisation and Harmonisation for Research Excellence in the European Union) project aims to address these issues by building a collaborative group of investigators and developing tools for data harmonization, database integration and federated data analyses.MethodsEight population-based studies in six European countries were recruited to participate in the BioSHaRE project. Through workshops, teleconferences and electronic communications, participating investigators identified a set of 96 variables targeted for harmonization to answer research questions of interest. Using each study’s questionnaires, standard operating procedures, and data dictionaries, harmonization potential was assessed. Whenever harmonization was deemed possible, processing algorithms were developed and implemented in an open-source software infrastructure to transform study-specific data into the target (i.e. harmonized) format. Harmonized datasets located on server in each research centres across Europe were interconnected through a federated database system to perform statistical analysis.ResultsRetrospective harmonization led to the generation of common format variables for 73% of matches considered (96 targeted variables across 8 studies). Authenticated investigators can now perform complex statistical analyses of harmonized datasets stored on distributed servers without actually sharing individual-level data using the DataSHIELD method.ConclusionNew Internet-based networking technologies and database management systems are providing the means to support collaborative, multi-center research in an efficient and secure manner. The results from this pilot project show that, given a strong collaborative relationship between participating studies, it is possible to seamlessly co-analyse internationally harmonized research databases while allowing each study to retain full control over individual-level data. We encourage additional collaborative research networks in epidemiology, public health, and the social sciences to make use of the open source tools presented herein.

A Generic Open-Source Software Framework Supporting Scenario Simulations in Bioterrorist Crises

Since the 2001 anthrax attack in the United States, awareness of threats originating from bioterrorism has grown. This led internationally to increased research efforts to improve knowledge of and approaches to protecting human and animal populations against the threat from such attacks. A collaborative effort in this context is the extension of the open-source Spatiotemporal Epidemiological Modeler (STEM) simulation and modeling software for agro- or bioterrorist crisis scenarios. STEM, originally designed to enable community-driven public health disease models and simulations, was extended with new features that enable integration of proprietary data as well as visualization of agent spread along supply and production chains. STEM now provides a fully developed open-source software infrastructure supporting critical modeling tasks such as ad hoc model generation, parameter estimation, simulation of scenario evolution, estimation of effects of mitigation or management measures, and documentation. This open-source software resource can be used free of charge. Additionally, STEM provides critical features like built-in worldwide data on administrative boundaries, transportation networks, or environmental conditions (eg, rainfall, temperature, elevation, vegetation). Users can easily combine their own confidential data with built-in public data to create customized models of desired resolution. STEM also supports collaborative and joint efforts in crisis situations by extended import and export functionalities. In this article we demonstrate specifically those new software features implemented to accomplish STEM application in agro- or bioterrorist crisis scenarios.

The National Alliance for Medical Image Computing, a roadmap initiative to build a free and open source software infrastructure for translational research in medical image analysis

The National Alliance for Medical Image Computing (NA-MIC), is a multi-institutional, interdisciplinary community of researchers, who share the recognition that modern health care demands improved technologies to ease suffering and prolong productive life. Organized under the National Centers for Biomedical Computing 7 years ago, the mission of NA-MIC is to implement a robust and flexible open-source infrastructure for developing and applying advanced imaging technologies across a range of important biomedical research disciplines. A measure of its success, NA-MIC is now applying this technology to diseases that have immense impact on the duration and quality of life: cancer, heart disease, trauma, and degenerative genetic diseases. The targets of this technology range from group comparisons to subject-specific analysis.

Next generation simulation tools: the Systems Biology Workbench and BioSPICE integration.

Researchers in quantitative systems biology make use of a large number of different software packages for modelling, analysis, visualization, and general data manipulation. In this paper, we describe the Systems Biology Workbench (SBW), a software framework that allows heterogeneous application components--written in diverse programming languages and running on different platforms--to communicate and use each others' capabilities via a fast binary encoded-message system. Our goal was to create a simple, high performance, opensource software infrastructure which is easy to implement and understand. SBW enables applications (potentially running on separate, distributed computers) to communicate via a simple network protocol. The interfaces to the system are encapsulated in client-side libraries that we provide for different programming languages. We describe in this paper the SBW architecture, a selection of current modules, including Jarnac, JDesigner, and SBWMeta-tool, and the close integration of SBW into BioSPICE, which enables both frameworks to share tools and compliment and strengthen each others capabilities.

The ERATO Systems Biology Workbench: enabling interaction and exchange between software tools for computational biology.

Researchers in computational biology today make use of a large number of different software packages for modeling, analysis, and data manipulation and visualization. In this paper, we describe the ERATO Systems Biology Workbench (SBW), a software framework that allows these heterogeneous application components--written in diverse programming languages and running on different platforms--to communicate and use each others' data and algorithmic capabilities. Our goal is to create a simple, open-source software infrastructure which is effective, easy to implement and easy to understand. SBW uses a broker-based architecture and enables applications (potentially running on separate, distributed computers) to communicate via a simple network protocol. The interfaces to the system are encapsulated in client-side libraries that we provide for different programming languages. We describe the SBW architecture and the current set of modules, as well as alternative implementation technologies.