Open Source License Research Articles

Open-source Cindex: what it means for indexers

Cindex professional indexing software was recently released under an open-source license. In this article, Maria Sullivan, who has been providing user support for Cindex since 1988, first explains what open-source software is. After outlining the background to the establishment of the Open Cindex Project, she then explains what this means for the indexing community and how Cindex can continue to be used – and improved – well into the future.

Leveraging the All of Us Database for Primary Care Research with Large Datasets.

The National Institutes of Health (NIH) are supporting the All of Us research program, a large multicenter initiative to accelerate precision medicine. The All of Us database contains information on greater than 400,000 individuals spanning thousands of medical conditions, drug exposure types, and laboratory test types. These data can be correlated with genomic information and with survey data on social and environmental factors which influence health. A core principle of the All of Us program is that participants should reflect the diversity present in the United States population.The All of Us database has advanced many areas of medicine but is currently underutilized by primary care and public health researchers. In this Special Communication article, I seek to reduce the "barrier to entry" for primary care researchers to develop new projects within the All of Us Researcher Workbench. This Special Communication discusses (1) obtaining access to the database, (2) using the database securely and responsibly, (3) the key design concepts of the Researcher Workbench, and (4) details of data set extraction and analysis in the cloud computing environment. Fully documented, tutorial R statistical programming language and Python programs are provided alongside this article, which researchers may freely adapt under the open-source MIT license. The primary care research community should use the All of Us database to accelerate innovation in primary care research, make epidemiologic discoveries, promote community health, and further the infrastructure-building strategic priority of the family medicine 2024 to 2030 National Research Strategy.

An Open-Source Julia Package for RMS Time-Domain Simulations of Power Systems

This paper presents RMSPowerSims.jl, an open-source Julia package for the time-domain simulation of power systems. The package is designed to be used in conjunction with PowerModels.jl, a widely used Julia package for power system optimization. RMSPowerSims.jl provides a framework for the simulation of power systems in the time domain, allowing for the study of transient stability, frequency stability, and other dynamic phenomena. The package is designed to be intuitive and flexible, allowing users to easily define custom models for network components and disturbances, while also providing a range of pre-constructed models for common power system components. RMSPowerSims.jl simplifies the process of performing RMS simulations on power system models developed using the PowerModels.jl ecosystem, and provides an easy-to-use modeling that reduces the barrier to entry for new users wishing to perform RMS simulations. The accuracy of the package is verified against DIgSILENT PowerFactory for short-circuit and load-increase disturbances, using the New England 39-bus system. The active power generation delivered by several generators in the network, and the voltage magnitudes of selected busbars are analyzed and noted to be in close agreement with those obtained using PowerFactory. The computational performance of the package is compared to that of PowerFactory and is found to be comparable for load-step simulations; however, PowerFactory is found to be considerably faster for short-circuit simulations. As computational performance is not a priority at this stage of development, this is expected, and speed optimization is planned for future work. RMSPowerSims.jl is available under an open-source license and can be downloaded from GitHub.

Self-supervised learning for chest computed tomography: training strategies and effect on downstream applications.

Self-supervised pre-training can reduce the amount of labeled training data needed by pre-learning fundamental visual characteristics of the medical imaging data. We investigate several self-supervised training strategies for chest computed tomography exams and their effects on downstream applications. We benchmark five well-known self-supervision strategies (masked image region prediction, next slice prediction, rotation prediction, flip prediction, and denoising) on 15M chest computed tomography (CT) slices collected from four sites of the Mayo Clinic enterprise, United States. These models were evaluated for two downstream tasks on public datasets: pulmonary embolism (PE) detection (classification) and lung nodule segmentation. Image embeddings generated by these models were also evaluated for prediction of patient age, race, and gender to study inherent biases in models' understanding of chest CT exams. The use of pre-training weights especially masked region prediction-based weights, improved performance, and reduced computational effort needed for downstream tasks compared with task-specific state-of-the-art (SOTA) models. Performance improvement for PE detection was observed for training dataset sizes as large as with a maximum gain of 5% over SOTA. The segmentation model initialized with pre-training weights learned twice as fast as the randomly initialized model. While gender and age predictors built using self-supervised training weights showed no performance improvement over randomly initialized predictors, the race predictor experienced a 10% performance boost when using self-supervised training weights. We released self-supervised models and weights under an open-source academic license. These models can then be fine-tuned with limited task-specific annotated data for a variety of downstream imaging tasks, thus accelerating research in biomedical imaging informatics.

RNApysoforms: Fast rendering interactive visualization of RNA isoform structure and expression in Python.

Alternative splicing generates multiple RNA isoforms from a single gene, enriching genetic diversity and impacting gene function. Effective visualization of these isoforms and their expression patterns is crucial but challenging due to limitations in existing tools. Traditional genome browsers lack programmability, while other tools offer limited customization, produce static plots, or cannot simultaneously display structures and expression levels. RNApysoforms was developed to address these gaps by providing a Python-based package that enables concurrent visualization of RNA isoform structures and expression data. Leveraging plotly and polars libraries, it offers an interactive, customizable, and faster-rendering framework suitable for web applications, enhancing the analysis and dissemination of RNA isoform research. RNApysoforms is a Python package available at ( https://github.com/UK-SBCoA-EbbertLab/RNApysoforms ) via an open-source MIT license. It can be easily installed using the piip package installer for Python. Thorough documentation and usage vignettes are available at: https://rna-pysoforms.readthedocs.io/en/latest/ .

Assessing the Carbon Intensity of e-fuels Production in European Countries: A Temporal Analysis

The transport sector heavily relies on the use of fossil fuels, which are causing major environmental concerns. Solutions relying on the direct or indirect use of electricity through e-fuel production are emerging to power the transport sector. To ensure environmental benefits are achieved over this transition, an accurate estimation of the impact of the use of electricity is needed. This requires a high temporal resolution to capture the high variability of electricity. This paper presents a previously unseen temporal analysis of the carbon intensity of e-fuels using grid electricity in countries that are members of the European Network of Transmission System Operators (ENTSO-E). It also provides an estimation of the potential load factor for producing low-carbon e-fuels according to the European Union legislative framework. This was achieved by building on top of the existing EcoDynElec tool to develop EcoDynElec_xr, a python tool enabling—with an hourly time resolution—the calculation, visualisation, and analysis of the historical time-series of electricity mixing from the ENTSO-E. The results highlight that, in 2023, very few European countries were reaching low carbon intensity for electricity that enables the use of grid electricity for the production of green electrolytic hydrogen. The methodological assumptions consider the consumption of the electricity mix instead of the production mix, and the considered time step is of paramount importance and drastically impacts the potential load factor of green hydrogen production. The developed tools are released under an open-source license to ensure transparency, result reproducibility, and reuse regarding newer data for other territories or for other purposes.

Beyond The Basics: A Detailed Survey of Advanced Python Applications and Innovations

Abstract: Python is the general purpose, high-level, scriptable, interpreted, object oriented and easy to learn language was developed by Guido van Rossum. Python is a popular programming language because of its simplicity, ease of use, open-source licensing, and accessibility. It provides various libraries and frameworks for learning and real world programming. Python’s versatility and active community make it an ideal language for Machine Learning, Data Science, Artificial Intelligence, Networking, Game Development, Web Development and various other domains. In this paper, we will provide an introduction to the main Python programming software tools used for Web development, Data science, Machine learning techniques, Educational Development, etc. In this paper will first introduce Python as a language, and give introduction about Web Development, Data science, Machine learning, Educational Development & Financial Analysis and then describe packages that are popular in that sector

Spectral cluster supertree: fast and statistically robust merging of rooted phylogenetic trees.

The algorithms for phylogenetic reconstruction are central to computational molecular evolution. The relentless pace of data acquisition has exposed their poor scalability and the conclusion that the conventional application of these methods is impractical and not justifiable from an energy usage perspective. Furthermore, the drive to improve the statistical performance of phylogenetic methods produces increasingly parameter-rich models of sequence evolution, which worsens the computational performance. Established theoretical and algorithmic results identify supertree methods as critical to divide-and-conquer strategies for improving scalability of phylogenetic reconstruction. Of particular importance is the ability to explicitly accommodate rooted topologies. These can arise from the more biologically plausible non-stationary models of sequence evolution. We make a contribution to addressing this challenge with Spectral Cluster Supertree, a novel supertree method for merging a set of overlapping rooted phylogenetic trees. It offers significant improvements over Min-Cut supertree and previous state-of-the-art methods in terms of both time complexity and overall topological accuracy, particularly for problems of large size. We perform comparisons against Min-Cut supertree and Bad Clade Deletion. Leveraging two tree topology distance metrics, we demonstrate that while Bad Clade Deletion generates more correct clades in its resulting supertree, Spectral Cluster Supertree's generated tree is generally more topologically close to the true model tree. Over large datasets containing 10,000 taxa and 500 source trees, where Bad Clade Deletion usually takes 2 h to run, our method generates a supertree in on average 20 s. Spectral Cluster Supertree is released under an open source license and is available on the python package index as sc-supertree.

Optimizing fountain codes for DNA data storage

Fountain codes, originally developed for reliable multicasting in communication networks, are effectively applied in various data transmission and storage systems. Their recent use in DNA data storage systems has unique challenges, since the DNA storage channel deviates from the traditional Gaussian white noise erasure model considered in communication networks and has several restrictions as well as special properties. Thus, optimizing fountain codes to address these challenges promises to improve their overall usability in DNA data storage systems. In this article, we present several methods for optimizing fountain codes for DNA data storage. Apart from generally applicable optimizations for fountain codes, we propose optimization algorithms to create tailored distribution functions of fountain codes, which is novel in the context of DNA data storage. We evaluate the proposed methods in terms of various metrics related to the DNA storage channel. Our evaluation shows that optimizing fountain codes for DNA data storage can significantly enhance the reliability and capacity of DNA data storage systems. The developed methods represent a step forward in harnessing the full potential of fountain codes for DNA-based data storage applications. The new coding schemes and all developed methods are available under a free and open-source software license.

Creating a biomedical knowledge base by addressing GPT inaccurate responses and benchmarking context.

We created GNQA, a generative pre-trained transformer (GPT) knowledge base driven by a performant retrieval augmented generation (RAG) with a focus on aging, dementia, Alzheimer's and diabetes. We uploaded a corpus of three thousand peer reviewed publications on these topics into the RAG. To address concerns about inaccurate responses and GPT 'hallucinations', we implemented a context provenance tracking mechanism that enables researchers to validate responses against the original material and to get references to the original papers. To assess the effectiveness of contextual information we collected evaluations and feedback from both domain expert users and 'citizen scientists' on the relevance of GPT responses. A key innovation of our study is automated evaluation by way of a RAG assessment system (RAGAS). RAGAS combines human expert assessment with AI-driven evaluation to measure the effectiveness of RAG systems. When evaluating the responses to their questions, human respondents give a "thumbs-up" 76% of the time. Meanwhile, RAGAS scores 90% on answer relevance on questions posed by experts. And when GPT-generates questions, RAGAS scores 74% on answer relevance. With RAGAS we created a benchmark that can be used to continuously assess the performance of our knowledge base. Full GNQA functionality is embedded in the free GeneNetwork.org web service, an open-source system containing over 25 years of experimental data on model organisms and human. The code developed for this study is published under a free and open-source software license at https://git.genenetwork.org/gn-ai/tree/README.md.

Cluster-efficient pangenome graph construction with nf-core/pangenome.

Pangenome graphs offer a comprehensive way of capturing genomic variability across multiple genomes. However, current construction methods often introduce biases, excluding complex sequences or relying on references. The PanGenome Graph Builder (PGGB) addresses these issues. To date, though, there is no state-of-the-art pipeline allowing for easy deployment, efficient and dynamic use of available resources, and scalable usage at the same time. To overcome these limitations, we present nf-core/pangenome, a reference-unbiased approach implemented in Nextflow following nf-core's best practices. Leveraging biocontainers ensures portability and seamless deployment in High-Performance Computing (HPC) environments. Unlike PGGB, nf-core/pangenome distributes alignments across cluster nodes, enabling scalability. Demonstrating its efficiency, we constructed pangenome graphs for 1000 human chromosome 19 haplotypes and 2146 Escherichia coli sequences, achieving a two to threefold speedup compared to PGGB without increasing greenhouse gas emissions. nf-core/pangenome is released under the MIT open-source license, available on GitHub and Zenodo, with documentation accessible at https://nf-co.re/pangenome/docs/usage.

An open-source time-domain wave-based room acoustic software in Python based on the nodal discontinuous Galerkin method

In this work, we introduce an open-source implementation of a time-domain wave-based room acoustic modeling software package, named DG_RoomAcoustics. In this software, the linear acoustic equations are spatially discretized by the nodal discontinuous Galerkin method, and are integrated in time by either the explicit Runge-Kutta or the arbitrary high-order derivatives (ADER) integration schemes. Following the principles of object-oriented programming paradigm, the software is structured to ensure generic applicability and to facilitate future extensions with additional functionalities (e.g., different time integration schemes, boundary conditions). A comprehensive presentation of the physical and numerical aspects of the problem is provided. A detailed exposition of the code structure and components is presented, all of which are released under an open-source license, fostering community feedback and collaborative contributions for ongoing improvements. A brief overview of the current capabilities of the software is introduced. Future work regarding possible functionality extensions and performance optimizing are discussed.

Pangenome-Informed Language Models for Privacy-Preserving Synthetic Genome Sequence Generation.

The public availability of genome datasets, such as The Human Genome Project (HGP), The 1000 Genomes Project, The Cancer Genome Atlas, and the International HapMap Project, has significantly advanced scientific research and medical understanding. Here our goal is to share such genomic information for downstream analysis while protecting the privacy of individuals through Differential Privacy (DP). We introduce synthetic DNA data generation based on pangenomes in combination with Pretrained-Language Models (PTLMs). We introduce two novel tokenization schemes based on pangenome graphs to enhance the modeling of DNA. We evaluated these tokenization methods, and compared them with classical single nucleotide and k-mer tokenizations. We find k-mer tokenization schemes, indicating that our tokenization schemes boost the model's performance consistency with long effective context length (covering longer sequences with the same number of tokens). Additionally, we propose a method to utilize the pangenome graph and make it comply with DP privacy standards. We assess the performance of DP training on the quality of generated sequences with discussion of the trade-offs between privacy and model accuracy. The source code for our work will be published under a free and open source license soon.

Higher-order correction of persistent batch effects in correlation networks.

Systems biology analyses often use correlations in gene expression profiles to infer co-expression networks that are then used as input for gene regulatory network inference or to identify functional modules of co-expressed or putatively co-regulated genes. While systematic biases, including batch effects, are known to induce spurious associations and confound differential gene expression analyses (DE), the impact of batch effects on gene co-expression has not been fully explored. Methods have been developed to adjust expression values, ensuring conditional independence of mean and variance from batch or other covariates for each gene, resulting in improved fidelity of DE analysis. However, such adjustments do not address the potential for spurious differential co-expression (DC) between groups. Consequently, uncorrected, artifactual DC can skew the correlation structure, leading to the identification of false, non-biological associations, even when the input data are corrected using standard batch correction. In this work, we demonstrate the persistence of confounders in covariance after standard batch correction using synthetic and real-world gene expression data examples. We then introduce Co-expression Batch Reduction Adjustment (COBRA), a method for computing a batch-corrected gene co-expression matrix based on estimating a conditional covariance matrix. COBRA estimates a reduced set of parameters expressing the co-expression matrix as a function of the sample covariates, allowing control for continuous and categorical covariates. COBRA is computationally efficient, leveraging the inherently modular structure of genomic data to estimate accurate gene regulatory associations and facilitate functional analysis for high-dimensional genomic data. COBRA is available under the GLP3 open source license in R and Python in netZoo (https://netzoo.github.io).

Feature selection by replicate reproducibility and non-redundancy.

A fundamental step in many analyses of high-dimensional data is dimension reduction. Two basic approaches are introduction of new synthetic coordinates and selection of extant features. Advantages of the latter include interpretability, simplicity, transferability, and modularity. A common criterion for unsupervized feature selection is variance or dynamic range. However, in practice, it can occur that high-variance features are noisy, that important features have low variance, or that variances are simply not comparable across features because they are measured in unrelated numeric scales or physical units. Moreover, users may want to include measures of signal-to-noise ratio and non-redundancy into feature selection. Here, we introduce the RNR algorithm, which selects features based on (i) the reproducibility of their signal across replicates and (ii) their non-redundancy, measured by linear dependence. It takes as input a typically large set of features measured on a collection of objects with two or more replicates per object. It returns an ordered list of features, i1,i2,…,ik, where feature i1 is the one with the highest reproducibility across replicates, i2 that with the highest reproducibility across replicates after projecting out the dimension spanned by i1, and so on. Applications to microscopy-based imaging of cells and proteomics highlight benefits of the approach. The RNR method is available via Bioconductor (Huber W, Carey VJ, Gentleman R et al. (Orchestrating high-throughput genomic analysis with bioconductor. Nat Methods 2015;12:115-21.) in the R package FeatSeekR. Its source code is also available at https://github.com/tcapraz/FeatSeekR under the GPL-3 open source license.

InteractOA: Showcasing the representation of knowledge from scientific literature in Wikidata

Knowledge generated during the scientific process is still mostly stored in the form of scholarly articles. This lack of machine-readability hampers efforts to find, query, and reuse such findings efficiently and contributes to today’s information overload. While attempts have been made to semantify journal articles, widespread adoption of such approaches is still a long way off. One way to demonstrate the usefulness of such approaches to the scientific community is by showcasing the use of freely available, open-access knowledge graphs such as Wikidata as sustainable storage and representation solutions. Here we present an example from the life sciences in which knowledge items from scholarly literature are represented in Wikidata, linked to their exact position in open-access articles. In this way, they become part of a rich knowledge graph while maintaining clear ties to their origins. As example entities, we chose small regulatory RNAs (sRNAs) that play an important role in bacterial and archaeal gene regulation. These post-transcriptional regulators can influence the activities of multiple genes in various manners, forming complex interaction networks. We stored the information on sRNA molecule interaction taken from open-access articles in Wikidata and built an intuitive web interface called InteractOA, which makes it easy to visualize, edit, and query information. The tool also links information on small RNAs to their reference articles from PubMed Central on the statement level. InteractOA encourages researchers to contribute, save, and curate their own similar findings. InteractOA is hosted at https://interactoa.zbmed.de and its code is available under a permissive open source licence. In principle, the approach presented here can be applied to any other field of research.

Electronic data capture in resource-limited settings using the lightweight clinical data acquisition and recording system

Our prototype system designed for clinical data acquisition and recording of studies is a novel electronic data capture (EDC) software for simple and lightweight data capture in clinical research. Existing software tools are either costly or suffer from very limited features. To overcome these shortcomings, we designed an EDC software together with a mobile client. We aimed at making it easy to set-up, modifiable, scalable and thereby facilitating research. We wrote the software in R using a modular approach and implemented existing data standards along with a meta data driven interface and database structure. The prototype is an adaptable open-source software, which can be installed locally or in the cloud without advanced IT-knowledge. A mobile web interface and progressive web app for mobile use and desktop computers is added. We show the software’s capability, by demonstrating four clinical studies with over 1600 participants and 679 variables per participant. We delineate a simple deployment approach for a server-installation and indicate further use-cases. The software is available under the MIT open-source license. Conclusively the software is versatile, easily deployable, highly modifiable, and extremely scalable for clinical studies. As an open-source R-software it is accessible, open to community-driven development and improvement in the future.

Cell Simulation as Cell Segmentation.

Single-cell spatial transcriptomics promises a highly detailed view of a cell's transcriptional state and microenvironment, yet inaccurate cell segmentation can render this data murky by misattributing large numbers of transcripts to nearby cells or conjuring nonexistent cells. We adopt methods from ab initio cell simulation to rapidly infer morphologically plausible cell boundaries that preserve cell type heterogeneity. Benchmarking applied to datasets generated by three commercial platforms show superior performance and computational efficiency of this approach compared with existing methods. We show that improved accuracy in cell segmentation aids greatly in detection of difficult to accurately segment tumor infiltrating immune cells such as neutrophils and T cells. Lastly, through improvements in our ability to delineate subsets of tumor infiltrating T cells, we show that CXCL13-expressing CD8+ T cells tend to be more closely associated with tumor cells than their CXCL13-negative counterparts in data generated from renal cell carcinoma patient samples.

TSDSystem: a framework to collect, archive and share time series data at volcanological observatories

This paper presents a framework designed to collect, archive, and share time series data coming from sensor networks at Istituto Nazionale di Geofisica e Vulcanologia, Osservatorio Etneo (Italy), which we have developed and called Time Series Database management System (TSDSystem). The framework proposes a flexible database model for the standardization of sensor networks data and implements an optimized technology for storage and retrieval of acquired time series data. It is designed for the implementation of multiparametric databases and then suitable for development in volcanological observatories worldwide. The proposed framework provides a web service to perform writing and reading data via a standard web communication protocol, which easily enables interaction with other instruments or automatic systems. All results provided by the TSDSystem web service are represented using common data formats in the context of online services. In particular, the station networks metadata representation follows a schema inspired by the International Federation of Digital Seismograph Networks, widely known in seismology. A web GUI (graphical user interface) is provided to test and document the web service. Additionally, basic built-in web applications are supplied with the web GUI to perform joint and synchronized time series data visualization as well as representation of stations on a geographical map. The web GUI also offers administration tools for data access policy management, creation of monitoring dashboards and data publication through web pages. The framework implements an authorization system that can be used to restrict both writing or reading operations. The TSDSystem can also be a useful tool for engineering surveillance systems. The implementing code of the framework is available with an open source license on a public repository together with a user manual.

Pangenome graph layout by Path-Guided Stochastic Gradient Descent.

The increasing availability of complete genomes demands for models to study genomic variability within entire populations. Pangenome graphs capture the full genomic similarity and diversity between multiple genomes. In order to understand them, we need to see them. For visualization, we need a human-readable graph layout: a graph embedding in low (e.g. two) dimensional depictions. Due to a pangenome graph's potential excessive size, this is a significant challenge. In response, we introduce a novel graph layout algorithm: the Path-Guided Stochastic Gradient Descent (PG-SGD). PG-SGD uses the genomes, represented in the pangenome graph as paths, as an embedded positional system to sample genomic distances between pairs of nodes. This avoids the quadratic cost seen in previous versions of graph drawing by SGD. We show that our implementation efficiently computes the low-dimensional layouts of gigabase-scale pangenome graphs, unveiling their biological features. We integrated PG-SGD in ODGI which is released as free software under the MIT open source license. Source code is available at https://github.com/pangenome/odgi.