Python Implementation Research Articles

Machine learning-based meta-analysis of colorectal cancer and inflammatory bowel disease.

Colorectal cancer (CRC) is a major global health concern, resulting in numerous cancer-related deaths. CRC detection, treatment, and prevention can be improved by identifying genes and biomarkers. Despite extensive research, the underlying mechanisms of CRC remain elusive, and previously identified biomarkers have not yielded satisfactory insights. This shortfall may be attributed to the predominance of univariate analysis methods, which overlook potential combinations of variants and genes contributing to disease development. Here, we address this knowledge gap by presenting a novel multivariate machine-learning strategy to pinpoint genes associated with CRC. Additionally, we applied our analysis pipeline to Inflammatory Bowel Disease (IBD), as IBD patients face substantial CRC risk. The importance of the identified genes was substantiated by rigorous validation across numerous independent datasets. Several of the discovered genes have been previously linked to CRC, while others represent novel findings warranting further investigation. A Python implementation of our pipeline can be accessed publicly at https://github.com/AriaSar/CRCIBD-ML.

Structural Outlier Detection and Zernike-Canterakis Moments for Molecular Surface Meshes-Fast Implementation in Python.

Object retrieval systems measure the degree of similarity of the shape of 3D models. They search for the elements of the 3D model databases that resemble the query model. In structural bioinformatics, the query model is a protein tertiary/quaternary structure and the objective is to find similarly shaped molecules in the Protein Data Bank. With the ever-growing size of the PDB, a direct atomic coordinate comparison with all its members is impractical. To overcome this problem, the shape of the molecules can be encoded by fixed-length feature vectors. The distance of a protein to the entire PDB can be measured in this low-dimensional domain in linear time. The state-of-the-art approaches utilize Zernike-Canterakis moments for the shape encoding and supply the retrieval process with geometric data of the input structures. The BioZernike descriptors are a standard utility of the PDB since 2020. However, when trying to calculate the ZC moments locally, the issue of the deficiency of libraries readily available for use in custom programs (i.e., without relying on external binaries) is encountered, in particular programs written in Python. Here, a fast and well-documented Python implementation of the Pozo-Koehl algorithm is presented. In contrast to the more popular algorithm by Novotni and Klein, which is based on the voxelized volume, the PK algorithm produces ZC moments directly from the triangular surface meshes of 3D models. In particular, it can accept the molecular surfaces of proteins as its input. In the presented PK-Zernike library, owing to Numba's just-in-time compilation, a mesh with 50,000 facets is processed by a single thread in a second at the moment order 20. Since this is the first time the PK algorithm is used in structural bioinformatics, it is employed in a novel, simple, but efficient protein structure retrieval pipeline. The elimination of the outlying chain fragments via a fast PCA-based subroutine improves the discrimination ability, allowing for this pipeline to achieve an 0.961 area under the ROC curve in the BioZernike validation suite (0.997 for the assemblies). The correlation between the results of the proposed approach and of the 3D Surfer program attains values up to 0.99.

Clinamen2: Functional-style evolutionary optimization in Python for atomistic structure searches

Clinamen2 is a versatile functional-style Python implementation of the covariance matrix adaptation evolution strategy (CMA-ES) utilizing Cholesky decomposition. On top of a problem-agnostic core algorithm, the software package offers a suite of utilities and library code enabling applications to important atomistic structure searches. Features include massively distributed computation and the BI-Population restart scheme. This article details the general code structure and introduces examples that illustrate some relevant applications for the materials science and chemistry worlds, including interfacing to density-functional-theory codes and machine-learned surrogate models. The functional design renders the code modular and adaptable, and makes the creation of interfaces to other atomistic software straightforward. Program summaryProgram Title: Clinamen2CPC Library link to program files:https://doi.org/10.17632/x7syr2txsd.1Developer's repository link:https://github.com/Madsen-s-research-group/clinamen2-public-releasesCode Ocean capsule:https://codeocean.com/capsule/4950229Licensing provisions: Apache-2.0Programming language: PythonSupplementary material:Nature of problem: Find optimal atomistic structures retaining full flexibility in the choice of the optimization target, the methodological approach and its implementation (e.g. CPU- vs. GPU-heavy calculations). Enable interfacing with relevant software, including but not limited to density-functional-theory (DFT) codes and machine-learning (ML) solutions.Solution method: The covariance matrix adaptation evolution strategy (CMA-ES) algorithm is implemented using Cholesky decompositions for efficiency. The core algorithm and application examples for specific problems are implemented in functional-style Python free of side effects, with data classes to keep track of the state of the evolution. Dask is used for job control to enable highly distributed workflows. Advanced strategies like BI-Population CMA-ES are easy to implement and illustrated in the examples.

Omnivariant Generalized Least Squares regression: Theory, geochronological applications, and making the case for reconciled Δ47 calibrations

Least-squares regression methods are mathematically powerful, conceptually and computationally simple, and widely used in many fields. However, none of the commonly-used flavors of least-squares regression, such as York regression or Generalized Least Squares (GLS), take into account the full set of covariances between all observed (x, y) values. Here we describe the Omnivariant Generalized Least Squares (OGLS) method to fit a model of the form y = f(x), accounting for the full error correlation structure of the (x, y) data, based on a first-order linear propagation of the uncertainties in all variables into errors in y residuals, followed by minimizing the vector of y residuals with respect to the Mahalanobis norm defined by its covariance matrix. This approach may be described as a generalization of both York regression and GLS. It is mathematically exact for straight-line fits, and is also suitable for many non-linear models. Here we describe the principles of OGLS regression and discuss its properties, caveats, and practical use, and provide two consistent open-source implementations in Python and R. To illustrate how various fields of geochronology and stable-isotope geochemistry may benefit from this new method, we discuss how OGLS may specifically apply to 40Ar/39Ar dating and how it provides robust mathematical evidence that Δ47 carbonate calibrations in the recently defined I-CDES metrological scale are statistically indistinguishable, effectively solving long-standing methodological discrepancies.

Phar-LSTM: a pharmacological representation-based LSTM network for drug-drug interaction extraction.

Pharmacological drug interactions are among the most common causes of medication errors. Many different methods have been proposed to extract drug-drug interactions from the literature to reduce medication errors over the last few years. However, the performance of these methods can be further improved. In this paper, we present a Pharmacological representation-based Long Short-Term Memory (LSTM) network named Phar-LSTM. In this method, a novel embedding strategy is proposed to extract pharmacological representations from the biomedical literature, and the information related to the target drug is considered. Then, an LSTM-based multi-task learning scheme is introduced to extract features from the different but related tasks according to their corresponding pharmacological representations. Finally, the extracted features are fed to the SoftMax classifier of the corresponding task. Experimental results on the DDIExtraction 2011 and DDIExtraction 2013 corpuses show that the performance of Phar-LSTM is competitive compared with other state-of-the-art methods. Our Python implementation and the corresponding data of Phar-LSTM are available by using the DOI 10.5281/zenodo.8249384.

PyComBat, a Python tool for batch effects correction in high-throughput molecular data using empirical Bayes methods

BackgroundVariability in datasets is not only the product of biological processes: they are also the product of technical biases. ComBat and ComBat-Seq are among the most widely used tools for correcting those technical biases, called batch effects, in, respectively, microarray and RNA-Seq expression data.ResultsIn this technical note, we present a new Python implementation of ComBat and ComBat-Seq. While the mathematical framework is strictly the same, we show here that our implementations: (i) have similar results in terms of batch effects correction; (ii) are as fast or faster than the original implementations in R and; (iii) offer new tools for the bioinformatics community to participate in its development. pyComBat is implemented in the Python language and is distributed under GPL-3.0 (https://www.gnu.org/licenses/gpl-3.0.en.html) license as a module of the inmoose package. Source code is available at https://github.com/epigenelabs/inmoose and Python package at https://pypi.org/project/inmoose.ConclusionsWe present a new Python implementation of state-of-the-art tools ComBat and ComBat-Seq for the correction of batch effects in microarray and RNA-Seq data. This new implementation, based on the same mathematical frameworks as ComBat and ComBat-Seq, offers similar power for batch effect correction, at reduced computational cost.

Fitting quantum noise models to tomography data

The presence of noise is currently one of the main obstacles to achieving large-scale quantum computation. Strategies to characterise and understand noise processes in quantum hardware are a critical part of mitigating it, especially as the overhead of full error correction and fault-tolerance is beyond the reach of current hardware. Non-Markovian effects are a particularly unfavourable type of noise, being both harder to analyse using standard techniques and more difficult to control using error correction. In this work we develop a set of efficient algorithms, based on the rigorous mathematical theory of Markovian master equations, to analyse and evaluate unknown noise processes. In the case of dynamics consistent with Markovian evolution, our algorithm outputs the best-fit Lindbladian, i.e., the generator of a memoryless quantum channel which best approximates the tomographic data to within the given precision. In the case of non-Markovian dynamics, our algorithm returns a quantitative and operationally meaningful measure of non-Markovianity in terms of isotropic noise addition. We provide a Python implementation of all our algorithms, and benchmark these on a range of 1- and 2-qubit examples of synthesised noisy tomography data, generated using the Cirq platform. The numerical results show that our algorithms succeed both in extracting a full description of the best-fit Lindbladian to the measured dynamics, and in computing accurate values of non-Markovianity that match analytical calculations.

Stress-Constrained Topology Optimization for Commercial Software: A Python Implementation for ABAQUS®

Topology optimization has evidenced its capacity to provide new optimal designs in many different disciplines. However, most novel methods are difficult to apply in commercial software, limiting their use in the academic field and hindering their application in the industry. This article presents a new open methodology for solving geometrically complex non-self-adjoint topology optimization problems, including stress-constrained and stress minimization formulations, using validated FEM commercial software. The methodology was validated by comparing the sensitivity analysis with the results obtained through finite differences and solving two benchmark problems with the following optimizers: Optimality Criteria, Method of Moving Asymptotes, Sequential Least-Squares Quadratic Programming (SLSQP), and Trust-constr optimization algorithms. The SLSQP and Trust-constr optimization algorithms obtained better results in stress-minimization problem statements than the methodology available in ABAQUS®. A Python implementation of this methodology is proposed, working in conjunction with the commercial software ABAQUS® 2023 to allow a straightforward application to new problems while benefiting from a graphic user interface and validated finite element solver.

АВТОМАТИЗАЦИЯ КОНТРОЛЯ КАЧЕСТВА НА ПРОИЗВОДСТВЕ С ИСПОЛЬЗОВАНИЕМ МЕТОДОВ МАШИННОГО ОБУЧЕНИЯ И КОМПЬЮТЕРНОГО ЗРЕНИЯ: РАЗРАБОТКА И АНАЛИЗ ЭФФЕКТИВНОСТИ СИСТЕМЫ

This article discusses methods for automating the quality control process in industrial enterprises using machine learning and computer vision. Special attention is paid to convolutional neural networks and their use for detecting defects in images. Specific examples of quality control system implementation in Python are given, describing the main steps in developing and using convolutional neural networks, as well as methods for evaluating system effectiveness. Overall, automating the quality control process using machine learning and computer vision is an effective way to improve product quality and optimize production processes.

ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions

Outlier detection refers to the identification of data points that deviate from a general data distribution. Existing unsupervised approaches often suffer from high computational cost, complex hyperparameter tuning, and limited interpretability, especially when working with large, high-dimensional datasets. To address these issues, we present a simple yet effective algorithm called ECOD (Empirical-Cumulative-distribution-based Outlier Detection), which is inspired by the fact that outliers are often the "rare events" that appear in the tails of a distribution. In a nutshell, ECOD first estimates the underlying distribution of the input data in a nonparametric fashion by computing the empirical cumulative distribution per dimension of the data. ECOD then uses these empirical distributions to estimate tail probabilities per dimension for each data point. Finally, ECOD computes an outlier score of each data point by aggregating estimated tail probabilities across dimensions. Our contributions are as follows: (1) we propose a novel outlier detection method called ECOD, which is both parameter-free and easy to interpret; (2) we perform extensive experiments on 30 benchmark datasets, where we find that ECOD outperforms 11 state-of-the-art baselines in terms of accuracy, efficiency, and scalability; and (3) we release an easy-to-use and scalable (with distributed support) Python implementation for accessibility and reproducibility.

Implementation of Apriori Algorithm for Determining Spare Parts Product Recommendation Packages

The aim of this research is to determine recommended product packages for spare parts from an automotive parts supplier. Shop owners have faced challenges in meeting customer demands over the past few months, experiencing frequent stockouts of spare parts due to a manual transaction recording system and a manual checking system for spare parts storage. This inefficiency and lack of accuracy in managing in-demand spare parts prompted the application of the apriori algorithm, a data mining method. Data was collected from the total sales over the past three months, subsequently cleaned and transformed for manual and Python-based apriori calculations. The results, obtained through both manual and Python implementations of apriori, indicate that the two frequently occurring item sets are oil filters with a confidence value of 68% and air filters with a confidence value of 63%. Based on these findings, the study recommends spare parts stores to maintain higher stock levels of oil filters and air filters compared to other spare parts.

Integrated reliability analysis procedure for spatially variable soil slopes using recursive FORM algorithm enhanced by importance sampling: Abaqus via Python implementations

ABSTRACT This paper proposes an efficient geotechnical reliability analysis framework that integrates the advantages of the first-order reliability method (FORM) and the Karhunen-Loève expansion (KL) to evaluate the reliability of spatially variable soil slopes. The improved Hasofer-Lind-Rackwitz-Fiessler (iHLRF) recursive algorithm is adopted to implement the FORM procedure when coupled with finite element codes for slope stability analysis, and the KL is used to reduce the dimension of random variables representing the soil random fields. To enhance the accuracy of the failure probability estimated by KL-iHLRF, the importance sampling (IS) is also conducted in this study, resulting in an innovative slope reliability analysis framework, KL-iHLRF-IS. It is found that although the FORM analysis by itself may not be able to confidently estimate the failure probability of spatially variable soil slopes because of the very high dimension of random variables and the strong nonlinearity of the limit state function, it provides useful information on the design point values for KL-iHLRF-IS to rapidly compute the failure probability. The efficiency of KL-iHLRF-IS for slope reliability analysis is illustrated by three example slope case studies. The proposed framework is particularly useful when fast yet accurate slope reliability analysis using the finite element method is needed.

Inferring cellular and molecular processes in single-cell data with non-negative matrix factorization using Python, R and GenePattern Notebook implementations of CoGAPS.

Non-negative matrix factorization (NMF) is an unsupervised learning method well suited to high-throughput biology. However, inferring biological processes from an NMF result still requires additional post hoc statistics and annotation for interpretation of learned features. Here, we introduce a suite of computational tools that implement NMF and provide methods for accurate and clear biological interpretation and analysis. A generalized discussion of NMF covering its benefits, limitations and open questions is followed by four procedures for the Bayesian NMF algorithm Coordinated Gene Activity across Pattern Subsets (CoGAPS). Each procedure will demonstrate NMF analysis to quantify cell state transitions in a public domain single-cell RNA-sequencing dataset. The first demonstrates PyCoGAPS, our new Python implementation that enhances runtime for large datasets, and the second allows its deployment in Docker. The third procedure steps through the same single-cell NMF analysis using our R CoGAPS interface. The fourth introduces a beginner-friendly CoGAPS platform using GenePattern Notebook, aimed at users with a working conceptual knowledge of data analysis but without a basic proficiency in the R or Python programming language. We also constructed a user-facing website to serve as a central repository for information and instructional materials about CoGAPS and its application programming interfaces. The expected timing to setup the packages and conduct a test run is around 15 min, and an additional 30 min to conduct analyses on a precomputed result. The expected runtime on the user's desired dataset can vary from hours to days depending on factors such as dataset size or input parameters.

Self-Calibration of UAV Thermal Imagery Using Gradient Descent Algorithm

Unmanned aerial vehicle (UAV) thermal imagery offers several advantages for environmental monitoring, as it can provide a low-cost, high-resolution, and flexible solution to measure the temperature of the surface of the land. Limitations related to the maximum load of the drone lead to the use of lightweight uncooled thermal cameras whose internal components are not stabilized to a constant temperature. Such cameras suffer from several unwanted effects that contribute to the increase in temperature measurement error from ±0.5 °C in laboratory conditions to ±5 °C in unstable flight conditions. This article describes a post-processing procedure that reduces the above unwanted effects. It consists of the following steps: (i) devignetting using the single image vignette correction algorithm, (ii) georeferencing using image metadata, scale-invariant feature transform (SIFT) stitching, and gradient descent optimisation, and (iii) inter-image temperature consistency optimisation by minimisation of bias between overlapping thermal images using gradient descent optimisation. The solution was tested in several case studies of river areas, where natural water bodies were used as a reference temperature benchmark. In all tests, the precision of the measurements was increased. The root mean square error (RMSE) on average was reduced by 39.0% and mean of the absolute value of errors (MAE) by 40.5%. The proposed algorithm can be called self-calibrating, as in contrast to other known solutions, it is fully automatic, uses only field data, and does not require any calibration equipment or additional operator effort. A Python implementation of the solution is available on GitHub.

Single elimination tournament design using dynamic programming algorithm

Finding the best single-elimination tournament design is important in scientific inquiry because it can have major financial implications for event organizers and participants. This research aims to create an optimal single-elimination tournament design using binary tree modeling with dummy techniques. Dynamic programming algorithms have been used to compute optimal single-elimination designs to overcome this effectively. This research method uses various implementations of sub-optimal algorithms and then compares their performance in terms of runtime and optimality as a solution to measure the comparison of sub-algorithms. This research shows that the difference in relative costs produced by various sub-algorithms with the same input is quite low. This is expected because quotes are generated as integer values from a small interval 1, ≤ 9, whereas costs tend to reach much higher values. From the comparison of these sub-algorithms, the best results among the sub-optimal algorithms were obtained in the Sub Optimal algorithm 3. We present the experimental findings achieved using the Python implementation of the suggested algorithm, with a focus on the best single-elimination tournament design solution.

A Survey Paper on Moving Object Detection Using Deep Learning

Moving object detection in Python using deep learning is a powerful technique for accurately identifying and localizing moving objects in images or videos. By leveraging pre-trained models like YOLO or SSD, developers can implement this task efficiently. The Python implementation allows for customization and extension to handle real-time video streams and complex scenarios. This approach is valuable for researchers, practitioners, and enthusiasts interested in moving object detection using deep learning. Deep Convolution Neural Networks are leveraged to detect more precise coordinates and identify the category of objects. This survey paper provides study of various methodologies for object detection. This paper provides systematic analysis of various existing object detection techniques with precise and arranged representation.

Implementation and empirical evaluation of a quantum machine learning pipeline for local classification.

In the current era, quantum resources are extremely limited, and this makes difficult the usage of quantum machine learning (QML) models. Concerning the supervised tasks, a viable approach is the introduction of a quantum locality technique, which allows the models to focus only on the neighborhood of the considered element. A well-known locality technique is the k-nearest neighbors (k-NN) algorithm, of which several quantum variants have been proposed; nevertheless, they have not been employed yet as a preliminary step of other QML models. Instead, for the classical counterpart, a performance enhancement with respect to the base models has already been proven. In this paper, we propose and evaluate the idea of exploiting a quantum locality technique to reduce the size and improve the performance of QML models. In detail, we provide (i) an implementation in Python of a QML pipeline for local classification and (ii) its extensive empirical evaluation. Regarding the quantum pipeline, it has been developed using Qiskit, and it consists of a quantum k-NN and a quantum binary classifier, both already available in the literature. The results have shown the quantum pipeline's equivalence (in terms of accuracy) to its classical counterpart in the ideal case, the validity of locality's application to the QML realm, but also the strong sensitivity of the chosen quantum k-NN to probability fluctuations and the better performance of classical baseline methods like the random forest.

OpenFEPOPS: A Python implementation of the FEPOPS molecular similarity technique

OpenFEPOPS: A Python implementation of the FEPOPS molecular similarity technique

Research on the Application of Python in Chinese High School Education: Challenges and Solutions

Python programming courses have been gradually introduced into the high school education stage in China since 2013. This paper explores the application of Python in the Chinese education system, discusses the challenges faced, and presents corresponding solutions. Python facilitates students learning and problem-solving abilities while aligning with the demands of the era. Despite positive progress, challenges such as resource constraints and teacher training need to be addressed. Also, several problems encountered during the implementation of Python in practice are identified, including scheduling prioritization, insufficient qualifications of teachers, and challenges in remote areas. Solutions are proposed, such as incorporating Python into the curriculum evaluation and supervision, improving teacher qualifications through training and collaborations with experts, and providing hardware support in remote areas. Therefore, this paper presents the application status, challenges, and solutions of Python in Chinese high school education. It emphasizes the importance of promoting collaboration, practical abilities, and learning confidence among students. Future research can further explore the impact of Python on education and analyze specific contexts to enhance the localization and specialization of Python in China.

Simultaneous application of enzyme and thermodynamic constraints to metabolic models using an updated Python implementation of GECKO.

The metabolism of biological cells is an intricate network of reactions that interconvert chemical compounds, gathering energy, and using that energy to grow. The static analysis of these metabolic networks can be turned into a computational model that can efficiently output the distribution of fluxes in the network. With the inclusion of enzymes in the network, we can also interpret the role and concentrations of the metabolic proteins. However, the models and the experimental data often clash, resulting in a network that cannot grow. Here, we tackle this situation with a suite of relaxation algorithms in a package called geckopy. Geckopy also integrates with other software to allow for adding thermodynamic and metabolomic constraints. In addition, to ensure that enzyme-constrained models follow the community standards, a format for the proteins is postulated. We hope that the package and algorithms presented here will be useful for the constraint-based modeling community.