Python Interface Research Articles

Extendible and Efficient Python Framework for Solving Evolution Equations with Stabilized Discontinuous Galerkin Methods

This paper discusses a Python interface for the recently published Dune-Fem-DG module which provides highly efficient implementations of the discontinuous Galerkin (DG) method for solving a wide range of nonlinear partial differential equations (PDEs). Although the C++ interfaces of Dune-Fem-DG are highly flexible and customizable, a solid knowledge of C++ is necessary to make use of this powerful tool. With this work, easier user interfaces based on Python and the unified form language are provided to open Dune-Fem-DG for a broader audience. The Python interfaces are demonstrated for both parabolic and first-order hyperbolic PDEs.

RPG: A low-cost, open-source, high-performance solution for displaying visual stimuli

BackgroundThe development of new high throughput approaches for neuroscience such as high-density silicon probes and 2-photon imaging have led to a renaissance in visual neuroscience. However, generating the stimuli needed to evoke activity in the visual system still represents a non-negligible difficulty for experimentalists. While several widely used software toolkits exist to deliver such stimuli, they all suffer from some shortcomings. Primarily, the hardware needed to effectively display such stimuli comes at a significant financial cost, and secondly, triggering and/or timing the stimuli such that it can be accurately synchronized with other devices requires the use of legacy hardware, further hardware, or bespoke solutions. ResultsHere we present RPG (Raspberry Pi Gratings), a Python package written for the Raspberry Pi, which overcomes these issues. Specifically, the Raspberry Pi is a low-cost, credit card sized computer with general purpose input/output pins, allowing RPG to be triggered to deliver stimuli and to provide real-time feedback on stimulus timing. RPG delivers stimuli at 60 frames per second and the feedback of frame timings is accurate to 10s of microseconds. Comparison with existing method(s)With respect to the accuracy of frame timings, the performance of RPG is at least as accurate as commonly used packages. However, the inbuilt ability to trigger stimuli and the real-time feedback of frame timings will be extremely useful for certain experiments. ConclusionsRPG provides a simple to use Python interface that is capable of generating drifting sine wave gratings, Gabor patches and displaying raw images/video.

Olympus: a benchmarking framework for noisy optimization and experiment planning

Research challenges encountered across science, engineering, and economics can frequently be formulated as optimization tasks. In chemistry and materials science, recent growth in laboratory digitization and automation has sparked interest in optimization-guided autonomous discovery and closed-loop experimentation. Experiment planning strategies based on off-the-shelf optimization algorithms can be employed in fully autonomous research platforms to achieve desired experimentation goals with the minimum number of trials. However, the experiment planning strategy that is most suitable to a scientific discovery task is a priori unknown while rigorous comparisons of different strategies are highly time and resource demanding. As optimization algorithms are typically benchmarked on low-dimensional synthetic functions, it is unclear how their performance would translate to noisy, higher-dimensional experimental tasks encountered in chemistry and materials science. We introduce Olympus, a software package that provides a consistent and easy-to-use framework for benchmarking optimization algorithms against realistic experiments emulated via probabilistic deep-learning models. Olympus includes a collection of experimentally derived benchmark sets from chemistry and materials science and a suite of experiment planning strategies that can be easily accessed via a user-friendly Python interface. Furthermore, Olympus facilitates the integration, testing, and sharing of custom algorithms and user-defined datasets. In brief, Olympus mitigates the barriers associated with benchmarking optimization algorithms on realistic experimental scenarios, promoting data sharing and the creation of a standard framework for evaluating the performance of experiment planning strategies.

Designing a High-Performance Boundary Element Library With OpenCL and Numba

The Bempp boundary element library is a well-known library for the simulation of a range of electrostatic, acoustic, and electromagnetic problems in homogeneous bounded and unbounded domains. It originally started as a traditional C++ library with a Python interface. Over the last two years, we have completely redesigned Bempp as a native Python library, called Bempp-cl, that provides computational backends for OpenCL (using PyOpenCL) and Numba. The OpenCL backend implements kernels for GPUs and CPUs with SIMD optimization. In this article, we discuss the design of Bempp-cl, provide performance comparisons on different compute devices, and discuss the advantages and disadvantages of OpenCL as compared to Numba.

ClusTCR: a python interface for rapid clustering of large sets of CDR3 sequences with unknown antigen specificity.

The T-cell receptor (TCR) determines the specificity of a T-cell towards an epitope. As of yet, the rules for antigen recognition remain largely undetermined. Current methods for grouping TCRs according to their epitope specificity remain limited in performance and scalability. Multiple methodologies have been developed, but all of them fail to efficiently cluster large datasets exceeding 1 million sequences. To account for this limitation, we developed ClusTCR, a rapid TCR clustering alternative that efficiently scales up to millions of CDR3 amino acid sequences, without knowledge about their antigen specificity. Benchmarking comparisons revealed similar accuracy of ClusTCR as compared to other TCR clustering methods, as measured by cluster retention, purity and consistency. ClusTCR offers a drastic improvement in clustering speed, which allows the clustering of millions of TCR sequences in just a few minutes through ultraefficient similarity searching and sequence hashing. ClusTCR was written in Python 3. It is available as an anaconda package (https://anaconda.org/svalkiers/clustcr) and on github (https://github.com/svalkiers/clusTCR). Supplementary data are available at Bioinformatics online.

NetPyNE Implementation and Scaling of the Potjans-Diesmann Cortical Microcircuit Model.

The Potjans-Diesmann cortical microcircuit model is a widely used model originally implemented in NEST. Here, we reimplemented the model using NetPyNE, a high-level Python interface to the NEURON simulator, and reproduced the findings of the original publication. We also implemented a method for scaling the network size that preserves first- and second-order statistics, building on existing work on network theory. Our new implementation enabled the use of more detailed neuron models with multicompartmental morphologies and multiple biophysically realistic ion channels. This opens the model to new research, including the study of dendritic processing, the influence of individual channel parameters, the relation to local field potentials, and other multiscale interactions. The scaling method we used provides flexibility to increase or decrease the network size as needed when running these CPU-intensive detailed simulations. Finally, NetPyNE facilitates modifying or extending the model using its declarative language; optimizing model parameters; running efficient, large-scale parallelized simulations; and analyzing the model through built-in methods, including local field potential calculation and information flow measures.

The cosmic dispersion measure in the EAGLE simulations

ABSTRACT The dispersion measure (DM) of fast radio bursts (FRBs) provides a unique way to probe ionized baryons in the intergalactic medium (IGM). Cosmological models with different parameters lead to different DM–redshift (DM–z) relations. Additionally, the over/underdense regions in the IGM and the circumgalactic medium of intervening galaxies lead to scatter around the mean DM–z relations. We have used the Evolution and Assembly of GaLaxies and their Environments (EAGLE) simulations to measure the mean DM–z relation and the scatter around it using over 1 billion lines of sight at redshifts 0 < z < 3. We investigated two techniques to estimate line-of-sight DM: pixel scrambling and box transformations. We find that using box transformations (a technique from the literature) causes strong correlations due to repeated replication of structure. Comparing a linear and a non-linear model, we find that the non-linear model with a dependence on cosmological parameters provides a better fit to the DM–z relation. The differences between these models are the most significant at low redshifts (z < 0.5). The scatter around the DM–z relation is highly asymmetric, especially at low redshift (z < 0.5), and becomes more Gaussiana as redshift approaches z = 3, the limit of this study. The increase in Gaussianity with redshift is indicative of the large-scale structure that is better sampled with longer lines of sight. The DM–z relation measured in EAGLE is available with an easy-to-use python interface in the open-source FRB redshift estimation package fruitbat.

GeoGraph

In many applications of graph processing, the input data is often generated from an underlying geometric point data set. However, existing high-performance graph processing frameworks assume that the input data is given as a graph. Therefore, to use these frameworks, the user must write or use external programs based on computational geometry algorithms to convert their point data set to a graph, which requires more programming effort and can also lead to performance degradation. In this paper, we present our ongoing work on the Geo- Graph framework for shared-memory multicore machines, which seamlessly supports routines for parallel geometric graph construction and parallel graph processing within the same environment. GeoGraph supports graph construction based on k-nearest neighbors, Delaunay triangulation, and b-skeleton graphs. It can then pass these generated graphs to over 25 graph algorithms. GeoGraph contains highperformance parallel primitives and algorithms implemented in C++, and includes a Python interface. We present four examples of using GeoGraph, and some experimental results showing good parallel speedups and improvements over the Higra library. We conclude with a vision of future directions for research in bridging graph and geometric data processing.

Code Generation for Productive, Portable, and Scalable Finite Element Simulation in Firedrake

Creating scalable, high-performance PDE-based simulations requires an appropriate combination of models, discretizations, and solvers. The required combination changes with the application and with the available hardware, yet software development time is a severely limited resource for most scientists and engineers. Here we demonstrate that generating simulation code from a high-level Python interface provides an effective mechanism for creating high-performance simulations from very few lines of user code. We demonstrate that moving from one supercomputer to another can require significant algorithmic changes to achieve scalable performance, but that the code generation approach enables these algorithmic changes to be achieved with minimal development effort.

BEC2HPC: A HPC spectral solver for nonlinear Schrödinger and rotating Gross-Pitaevskii equations. Stationary states computation

We present BEC2HPC which is a parallel HPC spectral solver for computing the ground states of the nonlinear Schrödinger equation and the Gross-Pitaevskii equation (GPE) modeling rotating Bose-Einstein condensates (BEC). Considering a standard pseudo-spectral discretization based on Fast Fourier Transforms (FFTs), the method consists in finding the numerical solution of the energy functional minimization problem under normalization constraint by using a preconditioned nonlinear conjugate gradient method. We present some numerical simulations and scalability results for the 2D and 3D problems to obtain the stationary states of BEC with fast rotation and large nonlinearities. The code takes advantage of existing HPC libraries and can itself be leveraged to implement other numerical methods like e.g. for the dynamics of BECs. Program summaryProgram title: BEC2HPCCPC Library link to program files:https://doi.org/10.17632/mdzpw4dr4t.1Licensing provisions: GPLv2Programming language: C++, PythonNature of problem: This software computes the stationary states of rotating Bose–Einstein condensates (BEC) modeled by the Gross–Pitaevskii equation (GPE). It implements a numerical method that is particularly effective for BEC with fast rotation and large nonlinearities. The parallel implementation allows to perform large-scale simulations of 2D or 3D problems on parallel computing platforms.Solution method: The stationary states are computed using an iterative pseudo-spectral method based on Fast Fourier Transforms. The computation takes the form of a constrained minimization problem solved using a preconditioned nonlinear conjugate gradient method. This solver is implemented in distributed memory using MPI and a decomposition of the computational domain.Additional comments including restrictions and unusual features: The algorithms are implemented in C++ and MPI but a Python interface is provided for defining the physics of the problem. Results can be exported to HDF5 files and visualized with external tools such as ParaView. The code can be used to implement other spectral methods in parallel or to solve problems related to the dynamics of BECs.

A Python-Based Interface Design for Electric Power System Education.

Python, which is widely used today, is one of the open-source programming languages. In the Python program, its interface is done through the PyQt5 library. With PyQt5, it is possible to develop object-based software that takes up less space on the disk and runs faster. In this study, an educational calculation and analysis program (ITUPSA) was designed for the Electric power systems course using the Python interface. Electric power systems are one of the fundamental courses in electrical engineering. This main course is taught in three sub-course groups as energy transmission-distribution, energy flow and energy economy. The energy transmission-distribution course package constitutes the first and most comprehensive part of the electric power systems group. In the energy transmission-distribution, the creation of the transmission line model and the calculation of the line parameters form an important part of the training. This section contains very detailed mathematical calculations and it is very important that students get the correct result as a result of these calculations. This can be possible with a lot of practice. The interface design was made by students and faculty to teach practical examples correctly. This developed (ITUPSA) python-based interface program is very suitable for calculation and analysis exercises in both undergraduate and graduate courses in the Electrical Engineering department. The purpose of this study is to create a python-based interface in order to solve the problems related to these lines, to make the types of short, medium and long power transmission lines correct and understandable. In the study, after selecting the type of energy transmission line and all parameters related to the line, the necessary mathematical analysis can be made and also the analysis results can be plotted. The analysis program is available in two languages, English and Turkish.

MoSDeF Cassandra: A complete Python interface for the Cassandra Monte Carlo software.

We introduce a new Python interface for the Cassandra Monte Carlo software, molecular simulation design framework (MoSDeF) Cassandra. MoSDeF Cassandra provides a simplified user interface, offers broader interoperability with other molecular simulation codes, enables the construction of programmatic and reproducible molecular simulation workflows, and builds the infrastructure necessary for high-throughput Monte Carlo studies. Many of the capabilities of MoSDeF Cassandra are enabled via tight integration with MoSDeF. We discuss the motivation and design of MoSDeF Cassandra and proceed to demonstrate both simple use-cases and more complex workflows, including adsorption in porous media and a combined molecular dynamics - Monte Carlo workflow for computing lateral diffusivity in graphene slit pores. The examples presented herein demonstrate how even relatively complex simulation workflows can be reduced to, at most, a few files of Python code that can be version-controlled and shared with other researchers. We believe this paradigm will enable more rapid research advances and represents the future of molecular simulations.

ZMCintegral-v5.1: Support for multi-function integrations on GPUs

In this new version of ZMCintegral, we have added the functionality of multi-function integrations, i.e. the ability to integrate more than 103 different functions on GPUs. The Python API remains the similar as the previous versions. For integrands less than 5 dimensions, it usually takes less than 10 minutes to finish the evaluation of 103 integrations on one Tesla v100 card. The performance scales linearly with the increasing of the GPUs. New version program summaryProgram Title: ZMCintegralCPC Library link to program files:https://doi.org/10.17632/p7wc7k6mpp.3Licensing provisions: Apache-2.0Programming language: PythonJournal reference of previous version: Hong-Zhong Wu, Jun-Jie Zhang, Long-Gang Pang, Qun Wang, Comput. Phys. Commun. 248 (2020) 106962 and Jun-Jie Zhang, Hong-Zhong Wu, Comput. Phys. Commun. 251 (2020) 107240Does the new version supersede the previous version?: YesReasons for the new version: When solving the Boltzmann equation with radiations [1], one encounters different collision integrals for different energy beams. In the relativistic QED plasma, the collision terms involve various Feynman graphs [2] and usually the contribution from each graph is of great interest. In these circumstances one needs to integrate many functions of different forms simultaneously. In our previous versions [3,4], we focused on single integration of high dimensions and functions with parameters. Therefore, it is necessary to include the functionality for integrating many functions which have different dimensions, forms and integration domains.Summary of revisions:• Multi-function integrationsSuppose we have a series of integrations defined as(1)fn(x)=ancos(kn⋅x)+bnsin(kn⋅x), where n=1,2,3,...,100. The above integration series can be treated as a set of Harmonic bases if one wishes to evaluate the contribution of each Harmonic mode. In our previous versions, these series cannot be manipulated in a convenient and efficient way. It is worth noting that the integration domains or dimensions can be different, for example(2)gn(x1,x2)=an|x1+x2|for 0<n<50gn(x1,x2,x3)=bn|x1+x2−x3|for 50≤n≤100. The support of multi-functions gives the users full flexibility to integrate as many different functions as possible.• Test on GPUsAs an illustrative example, we report the solution of Eq. (1) with x=(x1,x2,x3,x4), an=bn=1. The ranges for all components are taken to be [0,1] and kn=(n+502π,n+502π,n+502π,n+502π) such that the integration is highly fluctuating around the zero line. The hardware condition in this case is taken to be: Intel(R) Xeon(R) Silver 4110 CPU@2.10GHz CPU with 10 processors + one Nvidia Tesla V100 GPU. The results are shown in Fig. 1.Nature of problem: ZMCintegral is an easy to use Python package for doing high dimensional integrations on distributed GPU clusters. Using the Python libraries Numba [5] and Ray [6], as well as the NVIDIA CUDA [7] capability, ZMCintegral offers a succinct Python interface to evaluate numerical integrations for physical problems. In this updated version, we mainly focus on the problems where the users have various functions to integrate. These integrations can take different forms and domains.Solution method: This new version contains three Python classes. ZMCintegral_normal utilizes the stratified-sampling and heuristic-tree-search techniques, while ZMCintegral_functional and ZMCintegral_multifunctions use the direct-Monte Carlo method for each integrand which benefits mainly from the heavily distributed GPU clusters.Additional comments including restrictions and unusual features: If the integrations are high-dimensional (e.g. dimensionality of 8-12), users are encouraged to use ZMCintegral_normal. If the integrations are middle-dimensional (e.g. dimensionality of 1-7) but with a large parameter space, we suggest users to try ZMCintegral_functional. If the integrations contain many different integrands and domains (e.g. 104 different integrations), then ZMCintegral_multifunctions is suggested. The detailed instructions can be found here: [8].

PyFstat: a Python package for continuous gravitational-wave data analysis

Gravitational waves in the sensitivity band of ground-based detectors can be emitted by a number of astrophysical sources, including not only binary coalescences, but also individual spinning neutron stars. The most promising signals from such sources, although not yet detected, are long-lasting, quasi-monochromatic Continuous Waves (CWs). The PyFstat package provides tools to perform a range of CW data-analysis tasks. It revolves around the F-statistic, a matched-filter detection statistic for CW signals that has been one of the standard methods for LIGO-Virgo CW searches for two decades. PyFstat is built on top of established routines in LALSuite but through its more modern Python interface it enables a flexible approach to designing new search strategies. Hence, it serves a dual function of (i) making LALSuite CW functionality more easily accessible through a Python interface, thus facilitating the new user experience and, for developers, the exploratory implementation of novel methods; and (ii) providing a set of production-ready search classes for use cases not yet covered by LALSuite itself, most notably for MCMC-based followup of promising candidates from wide-parameter-space searches.

Light-curve calculations for triple microlensing systems

ABSTRACT We present a method to compute the magnification of a finite source star lensed by a triple lens system based on the image boundary (contour integration) method. We describe a new procedure to obtain continuous image boundaries from solutions of the tenth-order polynomial obtained from the lens equation. Contour integration is then applied to calculate the image areas within the image boundaries, which yields the magnification of a source with uniform brightness. We extend the magnification calculation to limb-darkened stars approximated with a linear profile. In principle, this method works for all multiple lens systems, not just triple lenses. We also include an adaptive sampling and interpolation method for calculating densely covered light curves. The C++ source code and a corresponding python interface are publicly available.

Multiple Stellar Evolution: a population synthesis algorithm to model the stellar, binary, and dynamical evolution of multiple-star systems

ABSTRACT In recent years, observations have shown that multiple-star systems such as hierarchical triple and quadruple-star systems are common, especially among massive stars. They are potential sources of interesting astrophysical phenomena such as compact object mergers, leading to supernovae, and gravitational wave events. However, many uncertainties remain in their often complex evolution. Here, we present the population synthesis code Multiple Stellar Evolution (mse), designed to rapidly model the stellar, binary, and dynamical evolution of multiple-star systems. mse includes a number of new features not present in previous population synthesis codes: (1) an arbitrary number of stars, as long as the initial system is hierarchical, (2) dynamic switching between secular and direct N-body integration for efficient computation of the gravitational dynamics, (3) treatment of mass transfer in eccentric orbits, which occurs commonly in multiple-star systems, (4) a simple treatment of tidal, common envelope, and mass transfer evolution in which the accretor is a binary instead of a single star, (5) taking into account planets within the stellar system, and (6) including gravitational perturbations from passing field stars. mse, written primarily in the C++ language, will be made publicly available and has few prerequisites; a convenient python interface is provided. We give a detailed description of MSE and illustrate how to use the code in practice. We demonstrate its operation in a number of examples.

Loos, a Better Tool to Analyze Molecular Dynamics Simulations

The enormous improvements in computational power in recent years means that we can routinely runs simulations of biologically interesting systems out to the microsecond scale or longer. However, in many cases their scientific impact is limited by an inability to extract useful information from this wealth of data. We created LOOS to unlock this previously unrealized potential. Molecular dynamics is unique among biophysical techniques in that the raw data can be interrogated in a multitude of ways; getting the most out of the data usually requires exploration and iterative refinement. LOOS is designed to make this process easy and efficient, to minimize the friction between a user and her data. LOOS can be viewed in two ways: 1) It is a suite of well-validated, easy-to-use tools for analyzing molecular dynamics simulations. These tools include common tasks, such as trajectory manipulation, as well as more advanced techniques such as analysis of simulation convergence, 3D histograms, and Voronoi decomposition, with a particular focus on membranes and membrane-bound proteins. 2) It is a tool for making tools. At its heart, LOOS is a C++ library covered with a Python interface, for the optimal mix of high performance and ease of development. LOOS is designed for rapid application development, so that most common tasks (reading files, selecting atoms, aligning structures, computing distances, etc) are one-line operations. LOOS is licensed under the GPLv3 license and available via GitHub (https://github.com/GrossfieldLab/loos).

Ab initio determination of the shape of membrane proteins in a nanodisc

New software, called Marbles, is introduced that employs SAXS intensities to predict the shape of membrane proteins embedded into membrane nanodiscs. To gain computational speed and efficient convergence, the strategy is based on a hybrid approach that allows one to account for the contribution of the nanodisc to the SAXS intensity through a semi-analytical model, while the embedded membrane protein is treated as a set of beads, similarly to as in well known ab initio methods. The reliability and flexibility of this approach is proved by benchmarking the code, implemented in C++ with a Python interface, on a toy model and two proteins with very different geometry and size.

Techno-Economic Assessment of Grid-Level Battery Energy Storage Supporting Distributed Photovoltaic Power

Centralised, front-of-the-meter battery energy storage systems are an option to support and add flexibility to distribution networks with increasing distributed photovoltaic systems, which generate renewable energy locally and help decarbonise the power sector. However, the provision of specific services at distribution level remains under development for real applications in industry. To this end, this paper presents an exhaustive techno-economic analysis of the role of front-of-the-meter battery energy storage systems in primary distribution networks with presence of distributed PV covering: (i) the siting decision for storage systems using multi-objective genetic algorithm optimisation; (ii) the response when smart capabilities for PV inverters (e.g., volt-var control) are present; and (iii) the quantification of revenue streams and compensation schemes that would bring positive profitability when providing distribution-specific services based on the supply of real and reactive power. The performance of grid-level battery energy storage technology is evaluated in the IEEE 34-bus system particularised to the distribution code of Northern Ireland, UK. The techno-economic assessment covers one year of simulations at 1-minute resolution performed with the simulation tool EPRI OpenDSS and its Python communication interface. The results show that, besides the technical benefits of centralised battery storage systems, the economic compensation based on traditional transmission-level services is not sufficient for the profitability of these projects. Several ideas are explored to improve the profitability of these systems considering the local value and decarbonisation they add. The findings presented can direct system operators and regulators towards developing schemes to incentivise centralised battery energy storage projects in distribution networks in the context of distribution-level services.

SeqRepo: A system for managing local collections of biological sequences.

MotivationAccess to biological sequence data, such as genome, transcript, or protein sequence, is at the core of many bioinformatics analysis workflows. The National Center for Biotechnology Information (NCBI), Ensembl, and other sequence database maintainers provide methods to access sequences through network connections. For many users, the convenience and currency of remotely managed data are compelling, and the network latency is non-consequential. However, for high-throughput and clinical applications, local sequence collections are essential for performance, stability, privacy, and reproducibility.ResultsHere we describe SeqRepo, a novel system for building a local, high-performance, non-redundant collection of biological sequences. SeqRepo enables clients to use primary database identifiers and several digests to identify sequences and sequence alises. SeqRepo provides a native Python interface and a REST interface, which can run locally and enables access from other programming languages. SeqRepo also provides an alternative REST interface based on the GA4GH refget protocol.SeqRepo provides fast random access to sequence slices. We provide results that demonstrate that a local SeqRepo sequence collection yields significant performance benefits of up to 1300-fold over remote sequence collections. In our use case for a variant validation and normalization pipeline, SeqRepo improved throughput 50-fold relative to use with remote sequences. SeqRepo may be used with any species or sequence type. Regular snapshots of Human sequence collections are available.It is often convenient or necessary to use a computed digest as a sequence identifier. For example, a digest-based identifier may be used to refer to proprietary reference genomes or segments of a graph genome, for which conventional identifiers will not be available. Here we also introduce a convention for the application of the SHA-512 hashing algorithm with Base64 encoding to generate URL-safe identifiers. This convention, sha512t24u, combines a fast digest mechanism with a space-efficient representation that can be used for any object. Our report includes an analysis of timing and collision probabilities for sha512t24u. SeqRepo enables clients to use sha512t24u as identifiers, thereby seamlessly integrating public and private sequence sets.AvailabilitySeqRepo is released under the Apache License 2.0 and is available on github and PyPi. Docker images and database snapshots are also available. See https://github.com/biocommons/biocommons.seqrepo.