Pure Python Research Articles

DockQ v2: improved automatic quality measure for protein multimers, nucleic acids, and small molecules.

It is important to assess the quality of modeled biomolecules to benchmark and assess the performance of different prediction methods. DockQ has emerged as the standard tool for assessing the quality of protein interfaces in model structures against given references. However, as predictions of large multimers with multiple chains become more common, DockQ needs to be updated with more functionality for robustness and speed. Moreover, as the field progresses and more methods are released to predict interactions between proteins and other types of molecules, such as nucleic acids and small molecules, it becomes necessary to have a tool that can assess all types of interactions. Here, we present a complete reimplementation of DockQ in pure Python. The updated version of DockQ is more portable, faster and introduces novel functionalities, such as automatic DockQ calculations for multiple interfaces and automatic chain mapping with multi-threading. These enhancements are designed to facilitate comparative analyses of protein complexes, particularly large multi-chain complexes. Furthermore, DockQ is now also able to score interfaces between proteins, nucleic acids, and small molecules. DockQ v2 is available online at: https://wallnerlab.org/DockQ.

Poincaré and SimBio: a versatile and extensible Python ecosystem for modeling systems.

Chemical reaction networks (CRNs) play a pivotal role in diverse fields such as systems biology, biochemistry, chemical engineering, and epidemiology. High-level definitions of CRNs enables to use various simulation approaches, including deterministic and stochastic methods, from the same model. However, existing Python tools for simulation of CRN typically wrap external C/C++ libraries for model definition, translation into equations and/or numerically solving them, limiting their extensibility and integration with the broader Python ecosystem. In response, we developed Poincaré and SimBio, two novel Python packages for simulation of dynamical systems and CRNs. Poincaré serves as a foundation for dynamical systems modeling, while SimBio extends this functionality to CRNs, including support for the Systems Biology Markup Language (SBML). Poincaré and SimBio are developed as pure Python packages enabling users to easily extend their simulation capabilities by writing new or leveraging other Python packages. Moreover, this does not compromise the performance, as code can be just-in-time compiled with Numba. Our benchmark tests using curated models from the BioModels repository demonstrate that these tools may provide a potentially superior performance advantage compared to other existing tools. In addition, to ensure a user-friendly experience, our packages use standard typed modern Python syntax that provides a seamless integration with integrated development environments. Our Python-centric approach significantly enhances code analysis, error detection, and refactoring capabilities, positioning Poincaré and SimBio as valuable tools for the modeling community. Poincaré and SimBio are released under the MIT license. Their source code is available on GitHub (https://github.com/maurosilber/poincare and https://github.com/hgrecco/simbio) and can be installed from PyPI or conda-forge.

Alrecon: computed tomography reconstruction web application based on Solara

Synchrotron X-ray computed tomography is a non-destructive 3D imaging technique that offers the possibility to study the internal microstructure of samples with high spatial and temporal resolution. Given its unmatched image quality and acquisition speed, and the possibility to preserve the specimens, there is an increasing demand for this technique, from scientific users from innumerable disciplines. Computed tomography reconstruction is the computational process by which experimental radiographs are converted to a meaningful 3-dimensional image after the scan. The procedure involves pre-processing steps for image background and artifact correction on raw data, a reconstruction step approximating the inverse Radon-transform, and writing of the reconstructed volume image to disk. Several open-source Python packages exist to help scientists in the process of tomography reconstruction, by offering efficient implementations of reconstruction algorithms exploiting central or graphics processing unit (CPU and GPU, respectively), and by automating significant portions of the data processing pipeline. A further increase in productivity is attained by scheduling and parallelizing demanding reconstructions on high performance computing (HPC) clusters. Nevertheless, visual inspection and interactive selection of optimal reconstruction parameters remain crucial steps that are often performed in close interaction with the end-user of the data. As a result, the reconstruction task involves more than one software. Graphical user interfaces are provided to the user for fast inspection and optimization of reconstructions, while HPC resources are often accessed through scripts and command line interface. We propose Alrecon, a pure Python web application for tomographic reconstruction built using Solara. Alrecon offers users an intuitive and reactive environment for exploring data and customizing reconstruction pipelines. By leveraging upon popular 3D image visualization tools, and by providing a user-friendly interface for reconstruction scheduling on HPC resources, Alrecon guarantees productivity and efficient use of resources for any type of beamline user.

GeoTaichi: A Taichi-powered high-performance numerical simulator for multiscale geophysical problems

This study introduces GeoTaichi, an open-source high-performance numerical simulator designed for addressing multiscale geophysical problems. By leveraging the power of the Taichi parallel language, GeoTaichi maximizes the utilization of modern computer resources on multicore CPU and GPU architectures. It offers robust and reliable modules for the discrete element method (DEM), material point method (MPM), and coupled material point-discrete element method (MPDEM). These modules enable efficient solving of large-scale problems while being implemented in pure Python. The design philosophy of GeoTaichi focuses on creating a framework that is readable, extensible, and user-friendly. This paper highlights the coupling procedure of MPDEM, the code structures, and the most important features of GeoTaichi. Rigorous benchmark tests have been conducted to verify the validity and robustness of GeoTaichi. Additionally, the performance of GeoTaichi is compared with similar software tools in the field, underscoring a notable improvement in both computational efficiency and memory savings when compared to existing alternatives. Program summaryProgram title: GeoTaichiCPC Library link to program files:https://doi.org/10.17632/858bmcf7j6.1Developer's repository link:https://github.com/Yihao-Shi/GeoTaichiLicensing provisions: GNU General Public License v3.0Programming language: PythonNature of problem: The simulations of large-deformation geophysical flows and their interaction with structures play a crucial role in the field of geophysics. To address the complexities of these nonlinear problems, the discrete element method (DEM), material point method (MPM), and their coupling (MPDEM) have proven to be highly suitable numerical schemes. However, these schemes impose substantial computational demands, necessitating the development of an efficient framework that can harness modern computer resources on multicore CPU and GPU architectures.Solution method: The open-source code GeoTaichi implements the DEM, MPM, and coupled MPDEM, encompassing a range of constitutive models and contact laws for different geologic materials. The clump particle model is also introduced in DEM to solve granular mechanics involving complex-shaped particles. One significant advantage of GeoTaichi is its utilization of the Taichi parallel language, which is designed to be user-friendly and easily extensible for customized applications.

Likelihood-based Jump Detection and Cosmic Ray Rejection for Detectors Read Out Up-the-ramp

This paper implements likelihood-based jump detection for detectors read out up-the-ramp, using the entire set of reads to compute likelihoods. The approach compares the χ 2 value of a fit with and without a jump for every possible jump location. I show that this approach can be substantially more sensitive than one that only uses the difference between sequential groups of reads, especially for long ramps and for jumps that occur in the middle of a group of reads. It can also be implemented for a computational cost that is linear in the number of resultants. I provide and describe a pure Python implementation that can process a 10-resultant ramp on a 4096 × 4096 detector in ≈20 s, including iterative cosmic ray detection and removal, on a single core of a 2020 Macbook Air. This Python implementation, together with tests and a tutorial notebook, are available at https://github.com/t-brandt/fitramp. I also provide tests and demonstrations of the full ramp fitting and cosmic ray rejection approach on data from the JWST.

Optimal Fitting and Debiasing for Detectors Read Out Up-the-Ramp

This paper derives the optimal fit to a pixel’s count rate in the case of an ideal detector read out nondestructively in the presence of both read and photon noise. The approach is general for any readout scheme, provides closed-form expressions for all quantities, and has a computational cost that is linear in the number of resultants (groups of reads). I also derive the bias of the fit from estimating the covariance matrix and show how to remove it to first order. The ramp-fitting algorithm I describe provides the χ 2 value of the fit of a line to the accumulated counts, which can be interpreted as a goodness-of-fit metric. I provide and describe a pure Python implementation of these algorithms that can process a 10-resultant ramp on a 4096 × 4096 detector in ≈8 s with bias removal on a single core of a 2020 Macbook Air. This Python implementation, together with tests and a tutorial notebook, are available at https://github.com/t-brandt/fitramp. A companion paper describes a jump detection algorithm based on hypothesis testing of ramp fits and demonstrates all algorithms on data from JWST.

Alrecon: computed tomography reconstruction web application based on Solara.

Synchrotron X-ray computed tomography is a non-destructive 3D imaging technique that offers the possibility to study the internal microstructure of samples with high spatial and temporal resolution. Given its unmatched image quality and acquisition speed, and the possibility to preserve the specimens, there is an increasing demand for this technique, from scientific users from innumerable disciplines. Computed tomography reconstruction is the computational process by which experimental radiographs are converted to a meaningful 3-dimensional image after the scan. The procedure involves pre-processing steps for image background and artifact correction on raw data, a reconstruction step approximating the inverse Radon-transform, and writing of the reconstructed volume image to disk. Several open-source Python packages exist to help scientists in the process of tomography reconstruction, by offering efficient implementations of reconstruction algorithms exploiting central or graphics processing unit (CPU and GPU, respectively), and by automating significant portions of the data processing pipeline. A further increase in productivity is attained by scheduling and parallelizing demanding reconstructions on high performance computing (HPC) clusters. Nevertheless, visual inspection and interactive selection of optimal reconstruction parameters remain crucial steps that are often performed in close interaction with the end-user of the data. As a result, the reconstruction task involves more than one software. Graphical user interfaces are provided to the user for fast inspection and optimization of reconstructions, while HPC resources are often accessed through scripts and command line interface. We propose Alrecon, a pure Python web application for tomographic reconstruction built using Solara. Alrecon offers users an intuitive and reactive environment for exploring data and customizing reconstruction pipelines. By leveraging upon popular 3D image visualization tools, and by providing a user-friendly interface for reconstruction scheduling on HPC resources, Alrecon guarantees productivity and efficient use of resources for any type of beamline user.

Dias: Dynamic Rewriting of Pandas Code

In recent years, dataframe libraries, such as pandas have exploded in popularity. Due to their flexibility, they are increasingly used in ad-hoc exploratory data analysis (EDA) workloads. These workloads are diverse, including custom functions which can span libraries or be written in pure Python. The majority of systems available to accelerate EDA workloads focus on bulk-parallel workloads, which contain vastly different computational patterns, typically within a single library. As a result, they can introduce excessive overheads for ad-hoc EDA workloads due to their expensive optimization techniques. Instead, we identify source-to-source, external program rewriting as a lightweight technique which can optimize across representations, and offer substantial speedups while also avoiding slowdowns. We implemented Dias, which rewrites notebook cells to be more efficient for ad-hoc EDA workloads. We develop techniques for efficient rewrites in Dias, including checking the preconditions under which rewrites are correct, dynamically, at fine-grained program points. We show that Dias can rewrite individual cells to be 57× faster compared to pandas and 1909× faster compared to optimized systems such as modin. Furthermore, Dias can accelerate whole notebooks by up to 3.6× compared to pandas and 27.1× compared to modin.

Polyglot Jet Finding

The evaluation of new computing languages for a large community, like HEP, involves comparison of many aspects of the languages’ behaviour, ecosystem and interactions with other languages. In this paper we compare a number of languages using a common, yet non-trivial, HEP algorithm: the anti- kT clustering algorithm used for jet finding. We compare specifically the algorithm implemented in Python (pure Python and accelerated with numpy and numba), and Julia, with respect to the reference implementation in C++, from Fastjet. As well as the speed of the implementation we describe the ergonomics of the language for the coder, as well as the efforts required to achieve the best performance, which can directly impact on code readability and sustainability.

СРАВНЕНИЕ МЕТОДОВ НАХОЖДЕНИЯ РАВНОВЕСИЯ В ТЕОРИИ ИГР В БИБЛИОТЕКЕ NashPу

The article compares the performance and reliability of different methods for finding Nash equilibrium situations in antagonistic and bimatrix games. The NashPy library is used, which implements a convenient interface for searching for equilibrium situations. All methods are implemented in pure Python language. The comparison showed that the support_enumeration method finds all equilibrium situations, but its running time grows exponentially. Therefore, when the dimension is greater than 15, it is recommended to use the vertex_enumeration method, which also finds all equilibrium situations, but does it faster. If you need to find any one equilibrium, it is better to use the lemke_howson_enumeration method, which works very quickly, but returns only one solution. The article clearly presents graphs of the time spent searching for equilibrium depending on the dimension of the problem.

PyMsOfa: A Python Package for the Standards of Fundamental Astronomy (SOFA) Service

The Standards of Fundamental Astronomy (SOFA) is a service provided by the International Astronomical Union that offers algorithms and software for astronomical calculations, which was released in two versions for FORTRAN 77 and ANSI C, respectively. In this work, we implement the Python package PyMsOfa for SOFA service by three ways: (1) a Python wrapper package based on a foreign function library for Python (ctypes), (2) a Python wrapper package with the foreign function interface for Python calling C code (cffi) and (3) a Python package directly written in pure Python codes from SOFA subroutines. The package PyMsOfa has fully implemented 247 functions of the original SOFA routines released on 2023 October 11. In addition, PyMsOfa is also extensively examined, which is exactly consistent with those test examples given by the original SOFA. This Python package can be suitable to not only the astrometric detection of habitable planets from the Closeby Habitable Exoplanet Survey mission, but also for the frontier themes of black holes and dark matter related to astrometric calculations and other fields. The source codes are available via http://pypi.org/project/PyMsOfa/ and https://github.com/CHES2023/PyMsOfa.

Mzapy: An Open-Source Python Library Enabling Efficient Extraction and Processing of Ion Mobility Spectrometry-Mass Spectrometry Data in the MZA File Format.

Analysis of ion mobility spectrometry (IMS) data has been challenging and limited the full utility of these measurements. Unlike liquid chromatography-mass spectrometry, where a plethora of tools with well-established algorithms exist, the incorporation of the additional IMS dimension requires upgrading existing computational pipelines and developing new algorithms to fully exploit the advantages of the technology. We have recently reported MZA, a new and simple mass spectrometry data structure based on the broadly supported HDF5 format and created to facilitate software development. While this format is inherently supportive of application development, the availability of core libraries in popular programming languages with standard mass spectrometry utilities will facilitate fast software development and broader adoption of the format. To this end, we present a Python package, mzapy, for efficient extraction and processing of mass spectrometry data in the MZA format, especially for complex data containing ion mobility spectrometry dimension. In addition to raw data extraction, mzapy contains supporting utilities enabling tasks including calibration, signal processing, peak finding, and generating plots. Being implemented in pure Python and having minimal and largely standardized dependencies makes mzapy uniquely suited to application development in the multiomics domain. The mzapy package is free and open-source, includes comprehensive documentation, and is structured to support future extension to meet the evolving needs of the MS community. The software source code is freely available at https://github.com/PNNL-m-q/mzapy.

JAX-FEM: A differentiable GPU-accelerated 3D finite element solver for automatic inverse design and mechanistic data science

This paper introduces JAX-FEM, an open-source differentiable finite element method (FEM) library. Constructed on top of Google JAX, a rising machine learning library focusing on high-performance numerical computing, JAX-FEM is implemented with pure Python while scalable to efficiently solve problems with moderate to large sizes. For example, in a 3D tensile loading problem with 7.7 million degrees of freedom, JAX-FEM with GPU achieves around 10× acceleration compared to a commercial FEM code depending on platform. Beyond efficiently solving forward problems, JAX-FEM employs the automatic differentiation technique so that inverse problems are solved in a fully automatic manner without the need to manually derive sensitivities. Examples of 3D topology optimization of nonlinear materials are shown to achieve optimal compliance. Finally, JAX-FEM is an integrated platform for machine learning-aided computational mechanics. We show an example of data-driven multi-scale computations of a composite material where JAX-FEM provides an all-in-one solution from microscopic data generation and model training to macroscopic FE computations. The source code of the library and these examples are shared with the community to facilitate computational mechanics research. Program summaryProgram Title: JAX-FEMCPC Library link to program files:https://doi.org/10.17632/hgwshjbcw6.1Developer's repository link:https://github.com/tianjuxue/jax-am/tree/main/jax_am/femLicensing provisions: GPLv3Programming language: PythonNature of problem: Implementation of the finite element method (FEM) with several appealing features that classic FEM software typically does not have: realized with pure Python; running on CPU/GPU; differentiable for solving PDE-constrained optimization problems; seamless integration with machine learning.Solution method: Our framework JAX-FEM is based on Google JAX, a high-performance numerical computing library with automatic differentiation features and supporting both CPU/GPU. Unlike classic FEM software written in Fortran or C/C++, JAX-FEM is implemented with pure Python and can easily be installed as a Python package. We demonstrate our software by solving problems including forward PDE prediction, inverse design/optimization, and data-driven analysis.

Pygenomics: manipulating genomic intervals and data files in Python.

We present pygenomics, a Python package for working with genomic intervals and bioinformatic data files. The package implements interval operations, provides both API and CLI, and supports reading and writing data in widely used bioinformatic formats, including BAM, BED, GFF3, and VCF. The source code of pygenomics is provided with in-source documentation and type annotations and adheres to the functional programming paradigm. These features facilitate seamless integration of pygenomics routines into scripts and pipelines. The package is implemented in pure Python using its standard library only and contains the property-based testing framework. Comparison of pygenomics with other Python bioinformatic packages with relation to features and performance is presented. The performance comparison covers operations with genomic intervals, read alignments, and genomic variants and demonstrates that pygenomics is suitable for computationally effective analysis. The source code is available at https://gitlab.com/gtamazian/pygenomics.

Mdapy: A flexible and efficient analysis software for molecular dynamics simulations

The mdapy is a library for pre- and postprocessing molecular dynamics simulation data. Benefitting from the just-in-time compile technology of TaiChi mdapy can be written in pure Python while possessing similar speed to those written in C++. mdapy is designed with highly paralleled and makes full advantages of modern computer resources on both multicore CPU and GPU architecture. The package implements a fast module to find the neighbors of particles in both free and periodic boundaries, based on which it offers a wide variety of methods to analyze atomic environments, such as standard centrosymmetry parameters, radial distribution function and newer methods, such as atomic entropy fingerprint. In addition, mdapy can be used to create the geometric structure of polycrystals with metallic or graphene grain boundaries by Voronoi diagram. mdapy can directly read the DUMP and DATA format defined in LAMMPS code, and, in practice, it accepts any other format by converting it into NumPy ndarray format. This design philosophy enables seamless integration with abundant scientific ecosystems in the Python community and easy cooperation with other analysis codes like OVITO or freud. Program summaryProgram Title: mdapyCPC Library link to program files:https://doi.org/10.17632/dtdkxvcsc9.1Developer's repository link:https://github.com/mushroomfire/mdapyCode Ocean capsule:https://codeocean.com/capsule/5271472Licensing provisions: BSD 3-clauseProgramming language: Python, C++Nature of problem: Atomic environment analysis and generation of the initial structure are important in molecular dynamics simulations. Many analysis methods relying on particle neighbors, such as radial distribution functions and atomic entropy, are computationally intensive and need to be carefully implemented to scale to large systems. Traditional code is often written in C++ or Fortran to guarantee performance while causing difficulty in installation and secondary development due to the complexity of the programming language.Solution method: mdapy is written in parallel to quickly perform neighbor finding, provides a set of analysis methods and creates an atomic geometry structure on multicore CPU and GPU. Over 95 percent of mdapy is written in Python with a uniform API to call. All data are stored in NumPy ndarray, making users easy to install, use and secondary develop.Additional comments including restrictions and unusual features:1. mdapy provides fast parallel implementations of neighbor finding in periodic and free boundary system.2. mdapy can generate a polycrystalline model with graphene grain boundaries.3. mdapy is helpful for analyzing the melting process by the mean squared displacement and Lindemann index at the atomic level.4. mdapy is very easy to install via pip install mdapy without any compile steps and can be run on Windows, Linux and Mac OS with Python 3.7-3.10.5. mdapy has detailed documentation to make it easier to use.

DQPB: software for calculating disequilibrium U–Pb ages

Abstract. Initial radioactive disequilibrium amongst intermediate nuclides of the U decay chains can have a significant impact on the accuracy of U–Pb ages, especially in young samples. For samples that can reasonably be assumed to have attained radioactive equilibrium at the time of analysis, a relatively straightforward correction may be applied. However, in younger materials where this assumption is unreasonable, it is necessary to replace the familiar U–Pb age equations with more complete expressions that account for growth and decay of intermediate nuclides through time. DQPB is software for calculating U–Pb ages while accounting for the effects of radioactive disequilibrium among intermediate nuclides of the U decay chains. The software is written in Python and distributed as both a pure Python package and a stand-alone graphical user interface (GUI) application that integrates with standard Microsoft Excel spreadsheets. The software implements disequilibrium U–Pb equations to compute ages using various approaches, including concordia intercept ages on a Tera–Wasserburg diagram, U–Pb isochron ages, Pb*/U ages based on single aliquots, and 207Pb-corrected ages. While these age-calculation approaches are tailored toward young samples that cannot reasonably be assumed to have attained radioactive equilibrium at the time of analysis, they may also be applied to older materials where disequilibrium is no longer analytically resolvable. The software allows users to implement a variety of regression algorithms based on both classical and robust statistical approaches, compute weighted average ages and construct customisable, publication-ready plots of U–Pb age data. The regression and weighted average algorithms implemented in DQPB may also be applicable to other (i.e. non-U–Pb) geochronological datasets.

PyHomogeneity: A Python Package for Homogeneity Test of Time Series Data

The pyHomogeneity package is a pure python package for performing homogeneity tests. The homogeneity test is applied to detect one (or more) change-point/breakpoint in the time series or sequential data. It is one of the essential tests for time series data (e.g. financial time series, hydrology, climate study, etc.). However, till now, there is no Python package available for the homogeneity test. This is where the pyHomogeneity package comes in. Currently, Pettitt’s Test, four variants of Buishand’s Test, and SNHT Test can be performed using this package. The package is freely available for public use.

Forecasting the Detection Capabilities of Third-generation Gravitational-wave Detectors Using GWFAST

We introduce GWFAST, a novel Fisher-matrix code for gravitational-wave studies, tuned toward third-generation gravitational-wave detectors such as Einstein Telescope (ET) and Cosmic Explorer (CE). We use it to perform a comprehensive study of the capabilities of ET alone, and of a network made by ET and two CE detectors, as well as to provide forecasts for the forthcoming O4 run of the LIGO-Virgo-KAGRA (LVK) collaboration. We consider binary neutron stars, binary black holes, and neutron star–black hole binaries, and compute basic metrics such as the distribution of signal-to-noise ratio (S/N), the accuracy in the reconstruction of various parameters (including distance, sky localization, masses, spins, and, for neutron stars, tidal deformabilities), and the redshift distribution of the detections for different thresholds in S/N and different levels of accuracy in localization and distance measurement. We examine the expected distribution and properties of golden events, with especially large values of the S/N. We also pay special attention to the dependence of the results on astrophysical uncertainties and on various technical details (such as choice of waveforms, or the threshold in S/N), and we compare with other Fisher codes in the literature. In the companion paper Iacovelli et al., we discuss the technical aspects of the code. Together with this paper, we publicly release the code GWFAST, (https://github.com/CosmoStatGW/gwfast) and the library WF4Py (https://github.com/CosmoStatGW/WF4Py) implementing state-of-the-art gravitational-wave waveforms in pure Python.

Icepool: Efficient Computation of Dice Pool Probabilities

Mechanics involving the roll of multiple dice---a "dice pool"---commonly appear in tabletop board games and role-playing games. Existing general-purpose dice pool probability calculators resort to exhaustive enumeration of all possible sorted sequences of rolls, which can quickly become computationally intractable. We propose a dynamic programming algorithm that can efficiently compute probabilities for a wide variety of dice pool mechanics while limiting the need for bespoke optimization. We also present Icepool, a pure Python implementation of the algorithm combined with a library of common dice operations.

Codebase release 3.4 for NetKet

We introduce version 3 of NetKet, the machine learning toolbox for many-body quantum physics. NetKet is built around neural quantum states and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature is the possibility to define arbitrary neural network ansätze in pure Python code using the concise notation of machine-learning frameworks, which allows for just-in-time compilation as well as the implicit generation of gradients thanks to automatic differentiation. NetKet 3 also comes with support for GPU and TPU accelerators, advanced support for discrete symmetry groups, chunking to scale up to thousands of degrees of freedom, drivers for quantum dynamics applications, and improved modularity, allowing users to use only parts of the toolbox as a foundation for their own code.