Python Tool Research Articles

Feature Selection Algorithms as One of the Python Data Analytical Tools

With the current trend of rapidly growing popularity of the Python programming language for machine learning applications, the gap between machine learning engineer needs and existing Python tools increases. Especially, it is noticeable for more classical machine learning fields, namely, feature selection, as the community attention in the last decade has mainly shifted to neural networks. This paper has two main purposes. First, we perform an overview of existing open-source Python and Python-compatible feature selection libraries, show their problems, if any, and demonstrate the gap between these libraries and the modern state of feature selection field. Then, we present new open-source scikit-learn compatible ITMO FS (Information Technologies, Mechanics and Optics University feature selection) library that is currently under development, explain how its architecture covers modern views on feature selection, and provide some code examples on how to use it with Python and its performance compared with other Python feature selection libraries.

Machine learning meets pK a

We present a small molecule pK a prediction tool entirely written in Python. It predicts the macroscopic pK a value and is trained on a literature compilation of monoprotic compounds. Different machine learning models were tested and random forest performed best given a five-fold cross-validation (mean absolute error=0.682, root mean squared error=1.032, correlation coefficient r 2 =0.82). We test our model on two external validation sets, where our model performs comparable to Marvin and is better than a recently published open source model. Our Python tool and all data is freely available at https://github.com/czodrowskilab/Machine-learning-meets-pKa.

In silico approach to predict pancreatic β-cells classically secreted proteins.

Pancreatic β-cells, residents of the islets of Langerhans, are the unique insulin-producers in the body. Their physiology is a topic of intensive studies aiming to understand the biology of insulin production and its role in diabetes pathology. However, investigations about these cells’ subset of secreted proteins, the secretome, are surprisingly scarce and a list describing islet/β-cell secretome upon glucose-stimulation is not yet available. In silico predictions of secretomes are an interesting approach that can be employed to forecast proteins likely to be secreted. In this context, using the rationale behind classical secretion of proteins through the secretory pathway, a Python tool capable of predicting classically secreted proteins was developed. This tool was applied to different available proteomic data (human and rodent islets, isolated β-cells, β-cell secretory granules, and β-cells supernatant), filtering them in order to selectively list only classically secreted proteins. The method presented here can retrieve, organize, search and filter proteomic lists using UniProtKB as a central database. It provides analysis by overlaying different sets of information, filtering out potential contaminants and clustering the identified proteins into functional groups. A range of 70–92% of the original proteomes analyzed was reduced generating predicted secretomes. Islet and β-cell signal peptide-containing proteins, and endoplasmic reticulum-resident proteins were identified and quantified. From the predicted secretomes, exemplary conservational patterns were inferred, as well as the signaling pathways enriched within them. Such a technique proves to be an effective approach to reduce the horizon of plausible targets for drug development or biomarkers identification.

Machine learning meets pK a.

We present a small molecule pK a prediction tool entirely written in Python. It predicts the macroscopic pK a value and is trained on a literature compilation of monoprotic compounds. Different machine learning models were tested and random forest performed best given a five-fold cross-validation (mean absolute error=0.682, root mean squared error=1.032, correlation coefficient r 2 =0.82). We test our model on two external validation sets, where our model performs comparable to Marvin and is better than a recently published open source model. Our Python tool and all data is freely available at https://github.com/czodrowskilab/Machine-learning-meets-pKa.

Team-Based Learning for Scientific Computing and Automated Experimentation: Visualization of Colored Reactions

The increasing integration of software and automation in modern chemical laboratories prompts special emphasis on two important skills in the chemistry classroom. First, students need to learn the technical skills involved in modern scientific computing and automation. Second, applying these techniques in practice requires effective collaboration in teams. This work aims at developing a teaching module to help students gain both skills. In particular, we describe a modular and collaborative approach for introducing undergraduate students to scientific computing in the context of automated and autonomous chemical laboratories. Using online collaboration tools, students work in parallel teams to develop central components of an automated computer vision system that monitors color changes in ongoing chemical reactions. These components include three different aspects: image capture, communication, and data visualization. The image capture team collects and stores the images of the chemical reaction, the communication team processes the images, and the visualization team develops the tools for analyzing the processed image data. Using this educational framework, students built an open-source Python tool called AutoVis that enables the automated tracking of color and intensity changes in a liquid. The software is tested by simulating chemical reactions with dilute solutions of food coloring in water. It is shown that the system reliably tracks color and intensity, providing feedback to the experimentalist and enabling further computational analysis. Over the course of the project, students gain proficiency in scientific computing using Python and collaborate on software development using GitHub. In this way, they learn the role of software in chemical laboratories of the future.

Design and Development of a Computer Vision Algorithm and Tool for Currency Recognition in Indian Vernacular Languages for Visually Challenged People

Design and Development of a Computer Vision Algorithm and Tool for Currency Recognition in Indian Vernacular Languages for Visually Challenged People

Conditional density estimation tools in python and R with applications to photometric redshifts and likelihood-free cosmological inference

It is well known in astronomy that propagating non-Gaussian prediction uncertainty in photometric redshift estimates is key to reducing bias in downstream cosmological analyses. Similarly, likelihood-free inference approaches, which are beginning to emerge as a tool for cosmological analysis, require a characterization of the full uncertainty landscape of the parameters of interest given observed data. However, most machine learning (ML) or training-based methods with open-source software target point prediction or classification, and hence fall short in quantifying uncertainty in complex regression and parameter inference settings. As an alternative to methods that focus on predicting the response (or parameters) $\mathbf{y}$ from features $\mathbf{x}$, we provide nonparametric conditional density estimation (CDE) tools for approximating and validating the entire probability density function (PDF) $\mathrm{p}(\mathbf{y}|\mathbf{x})$ of $\mathbf{y}$ given (i.e., conditional on) $\mathbf{x}$. As there is no one-size-fits-all CDE method, the goal of this work is to provide a comprehensive range of statistical tools and open-source software for nonparametric CDE and method assessment which can accommodate different types of settings and be easily fit to the problem at hand. Specifically, we introduce four CDE software packages in $\texttt{Python}$ and $\texttt{R}$ based on ML prediction methods adapted and optimized for CDE: $\texttt{NNKCDE}$, $\texttt{RFCDE}$, $\texttt{FlexCode}$, and $\texttt{DeepCDE}$. Furthermore, we present the $\texttt{cdetools}$ package, which includes functions for computing a CDE loss function for tuning and assessing the quality of individual PDFs, along with diagnostic functions. We provide sample code in $\texttt{Python}$ and $\texttt{R}$ as well as examples of applications to photometric redshift estimation and likelihood-free cosmological inference via CDE.

Migrating INFN-T1 from CREAM-CE/LSF to HTCondor-CE/HTCondor

The INFN Tier-1 datacentre provides computing resources to several HEP and Astrophysics experiments. These are organized in Virtual Organizations submitting jobs to our computing facilities through Computing Elements, acting as Grid interfaces to the Local Resource Manager. We are phasing-out our current LRMS (IBM/Platform LSF 9.1.3) and CEs (CREAM) set to adopt HTCondor as a replacement for LSF and HTCondor-CE in place of CREAM. A small instance has been set up to practice with the cluster management and evaluate the feasibility of our migration plans to a new LRMS and CE set. A second cluster instance has been setup to work on production. A number of management tools have been adapted or rewritten in order to integrate the new system with the existing infrastructure. Two different accounting solution for the HTCondor-CE have been implemented, and the more reliable one have been adopted. A python tool has been written to disentangle the management of HTCondor machines from our puppet instance, and to enable a quicker configuration of the cluster nodes. The monitoring tools tied to the old system are being adapted to also work on the new one. Finally, the most relevant setup steps have been documented in a public wiki page and a support mailing has been created to help other INFN sites willing to migrate their LRMS and CE to HTCondor. This document reports about our experience with HTCondor-CE on top of HTCondor and the integration of this system into our infrastructure.

MRMD2.0: A Python Tool for Machine Learning with Feature Ranking and Reduction

MRMD2.0: A Python Tool for Machine Learning with Feature Ranking and Reduction

Research on the Effectiveness of Network Negotiation Based on Evolutionary Game Model

This paper explores the effectiveness of network negotiation, which can not only provide better suggestions for government decision-making, but also regulate the network order and create a more harmonious network ecological environment. Based on this, this study constructs an evolutionary game model between government and netizens from the perspective of Internet. This paper uses Python tools to analyze the evolution of the game model. It explores the impact of initial strategy, the additional cost of efficient negotiation and the conversion rate of netizens on the evolution results. Furthermore, it introduces the motivate mechanism to study the impact of reward mechanism and punishment mechanism on system evolution.

HMST-Seq-Analyzer: A new python tool for differential methylation and hydroxymethylation analysis in various DNA methylation sequencing data

DNA methylation (5mC) and hydroxymethylation (5hmC) are chemical modifications of cytosine bases which play a crucial role in epigenetic gene regulation. However, cost, data complexity and unavailability of comprehensive analytical tools is one of the major challenges in exploring these epigenetic marks. Hydroxymethylation-and Methylation-Sensitive Tag sequencing (HMST-seq) is one of the most cost-effective techniques that enables simultaneous detection of 5mC and 5hmC at single base pair resolution. We present HMST-Seq-Analyzer as a comprehensive and robust method for performing simultaneous differential methylation analysis on 5mC and 5hmC data sets. HMST-Seq-Analyzer can detect Differentially Methylated Regions (DMRs), annotate them, give a visual overview of methylation status and also perform preliminary quality check on the data. In addition to HMST-Seq, our tool can be used on whole-genome bisulfite sequencing (WGBS) and reduced representation bisulfite sequencing (RRBS) data sets as well. The tool is written in Python with capacity to process data in parallel and is available at (https://hmst-seq.github.io/hmst/).

An Open Framework for Remote-PPG Methods and Their Assessment

This paper presents a comprehensive framework for studying methods of pulse rate estimation relying on remote photoplethysmography (rPPG). There has been a remarkable development of rPPG techniques in recent years, and the publication of several surveys too, yet a sound assessment of their performance has been overlooked at best, whether not undeveloped. The methodological rationale behind the framework we propose is that in order to study, develop and compare new rPPG methods in a principled and reproducible way, the following conditions should be met: 1) a structured pipeline to monitor rPPG algorithms’ input, output, and main control parameters; 2) the availability and the use of multiple datasets; and 3) a sound statistical assessment of methods’ performance. The proposed framework is instantiated in the form of a Python package named pyVHR (short for Python tool for Virtual Heart Rate), which is made freely available on GitHub ( github.com/phuselab/pyVHR ). Here, to substantiate our approach, we evaluate eight well-known rPPG methods, through extensive experiments across five public video datasets, and subsequent nonparametric statistical analysis. Surprisingly, performances achieved by the four best methods, namely POS, CHROM, PCA and SSR, are not significantly different from a statistical standpoint higighting the importance of evaluate the different approaches with a statistical assessment.

A content analysis of Arabic YouTube videos for cancer treatment

Purpose The purpose of this paper is to describe and assess Arabic videos related to cancer treatment to gain insights about the nature of health information as it is shared on YouTube. Accordingly, future strategies for different bodies are suggested to promote effective communication. Design/methodology/approach The approach is to select a representative sample of YouTube videos for certain search terms related to cancer treatment in the Arabic language. In order to identify the search terms, Google Trends is utilized. To retrieve the most relevant videos, a simple python tool is developed using YouTube API V3. For this study, the first 150 relevant videos are quantitatively and qualitatively analyzed. Objective data and subjective data are collected for each video and analyzed. Objective data include video title, URL, length, view count, like count, dislike count, comment count and the associated tags. For content analysis, coding themes are defined for the subjective data as follows: video format, video authorship and video content. Video content includes three categories: types of treatments, targeted part and evidence-based indicators. Findings The study included 150 videos, from which 30 videos were not content related; therefore, 120 videos remain in the analysis. Using rounding values, it can be observed that the average video lasted 10 min, had 184,966 views, was commented on 263 times, was liked by 2,295 users and disliked by 148 users. Non-professional individuals (46 percent) posted less than half of the videos, whereas public institutions posted only 18 percent of videos. More than half of videos (56 percent) promoted using herbal, botanical, and other natural products for cancer treatment. The majority of YouTube video formats were videos (52 percent), followed by audio with captions (30 percent). News and stories were the dominant videos, with (16 percent), and other types of videos were mostly testimonials and private centers promotions. Only 6 and 9 percent of videos targeted the genetic and immune systems, respectively. Out of the 120 analyzed videos, 86 percent did not mention any risk factor for the recommended treatment, and 73 percent did not offer the details of their usage direction. Research limitations/implications Researchers need to understand the information that is currently available on social media platforms related to the high-risk diseases in order to design initiatives, tools, and actions to allow an easy effective transfer of knowledge. Practical implications Recounting in-depth knowledge of YouTube cancer treatment contents will allow policy makers, YouTube management, medical organizations, and government agencies to understand the viewers’ behavior of YouTube and their needs to provide accurate and trustworthy information to adopt evidence-based resources. Social implications Creating the suitable content, in terms of health promotion strategies, associated with the appropriate format and understandable language that people need will be one of the major responsibilities of YouTube management, government and professional bodies. The well-designed health messages will enhance users’ engagement and attention to health issues from trusted sources. Originality/value There is very less information about Arabic messages in social media, YouTube in particular, specifically regarding cancer treatment. Thus, this study is one of the first studies to explore how Arabic messages are presented on YouTube. The aim of the assessment is to extract the current status and suggest future strategies for different bodies to have effective communication toward the Arabic communities.

EarthPy: A Python package that makes it easier to explore and plot raster and vector data using open source Python tools.

EarthPy: A Python package that makes it easier to explore and plot raster and vector data using open source Python tools.

Regression and classifier of real-time fault data using non-linear autoregressive exogenous model (NARX analysis) and comparing with Python tool

Regression and classifier of real-time fault data using non-linear autoregressive exogenous model (NARX analysis) and comparing with Python tool

Clustering of business structures by the level of their digital maturity using two approaches: iterative and hierarchical

Purpose. The aim of the article is experimental finding of the optimal number of clusters and their characteristic features for interpreted (understandable) segmentation of business structures by the level of digital maturity by several methods; comparing the results obtained by different methods and determining the most effective for a particular data analysis task. Methodology of research. Two clustering methods are used in the study: 1) using the Data Mining add-in for MS Excel spread sheets. Clustering capabilities in MS Excel are represented by iterative algorithms: k-means and Expectation-Maximization. For the reference, it was determined EM-algorithm; 2) using the functions of libraries for machine learning Python programming language. Findings. An experimental comparison of the use of two approaches to the clustering of respondents according to the results of online questionnaire using Google Forms service – hard and soft clustering, is conducted. Hard clustering was implemented using Python tools using the hierarchical agglomerative method, soft using the Data Mining add-in MS Excel and using the iterative EM method. A comparative analysis of the results obtained by the two methods showed that the agglomerative hierarchical clustering method is an effective method for solving the problem of clustering of mixed-type data obtained from the survey of respondents. Originality. An algorithm for solving the problem of respondents' clustering according to the results of online survey is proposed, including the stages of collection, preparation of data, obtaining the main results and development of future goals, which will help to solve the problem of processing and clustering of data of mixed type and to provide higher productivity of analytical data and analytical data. Practical value. This approach can be used to develop a Digital Transformation Index for domestic business structures and to measure their digital maturity, which will enhance their economic potential and therefore the country's economy. Key words: digital maturity; digital transformation; business structures; clustering methods; mixed-type data; surveys.

Enabling Immediate Access to Earth Science Models through Cloud Computing: Application to the GEOS-Chem Model

AbstractCloud computing platforms can provide fast and easy access to complex Earth science models and large datasets. This article presents a mature capability for running the GEOS-Chem global 3D model of atmospheric chemistry on the Amazon Web Services (AWS) cloud. GEOS-Chem users at any experience level can get immediate access to the latest, standard version of the model in a preconfigured software environment with all needed meteorological and other input data, and they can analyze model output data easily within the cloud using Python tools in Jupyter notebooks. Users with no prior knowledge of cloud computing are provided with easy-to-follow, step-by-step instructions. They can learn how to complete a demo project in less than one hour, and from there they can configure and submit their own simulations. The cloud is particularly attractive for beginning and occasional users who otherwise may need to spend substantial time configuring a local computing environment. Heavy users with their own local clusters can also benefit from the cloud to access the latest standard model and datasets, share simulation configurations and results, benchmark local simulations, and respond to surges in computing demand. Software containers allow GEOS-Chem and its software environment to be moved smoothly between cloud platforms and local clusters, so that the exact same simulation can be reproduced everywhere. Because the software requirements and workflows tend to be similar across Earth science models, the work presented here provides general guidance for porting models to cloud computing platforms in a user-accessible way.

FORECASTING SHARE PRICES USING SOFT COMPUTING TECHNIQUES

Background: For a long time, there has been a trend of trading of shares. Brokerage firms and dealers buy/sell stocks for clients and companies. Their work is based on knowing how the share price of the company will react in the market. Market/ share price predictions are useful as the investor/broker can attempt to predict the output in order to maximize his dividends or minimize his losses.
 Methodology: R and Python tools are used to sort, segregate and process the data, and techniques/algorithms such as Genetic Algorithm, ARIMA, Artificial Neural Networks, and Linear Regression are used to forecast results of data. Along with the model data, external factors affecting share prices also be taken into account.
 Findings: For each of the applied algorithms, their results are compared and the difference in output with the real-time values has been observed and recorded.
 Implications: Using data mining techniques, an attempt is made to estimate a prediction model to help forecast share prices.

A Python Tool for Implementations on Bipolar Neutrosophic Matrices

Bipolar neutrosophic matrices (BNM) are obtained by bipolar neutrosophic sets. Eachbipolar neutrosophic number represents an element of the matrix. The matrices are representablemulti-dimensional arrays (3D arrays). The arrays have nested list data type. Some operations,especially the composition is a challenging algorithm in terms of coding because there are so manynested lists to manipulate. This paper presents a Python tool for bipolar neutrosophic matrices. Theadvantage of this work, is that the proposed Python tool can be used also for fuzzy matrices, bipolarfuzzy matrices, intuitionistic fuzzy matrices, bipolar intuitionistic fuzzy matrices and single valuedneutrosophic matrices.

Unification of miRNA and isomiR research: the mirGFF3 format and the mirtop API.

MicroRNAs (miRNAs) are small RNA molecules (∼22 nucleotide long) involved in post-transcriptional gene regulation. Advances in high-throughput sequencing technologies led to the discovery of isomiRs, which are miRNA sequence variants. While many miRNA-seq analysis tools exist, the diversity of output formats hinders accurate comparisons between tools and precludes data sharing and the development of common downstream analysis methods. To overcome this situation, we present here a community-based project, miRNA Transcriptomic Open Project (miRTOP) working towards the optimization of miRNA analyses. The aim of miRTOP is to promote the development of downstream isomiR analysis tools that are compatible with existing detection and quantification tools. Based on the existing GFF3 format, we first created a new standard format, mirGFF3, for the output of miRNA/isomiR detection and quantification results from small RNA-seq data. Additionally, we developed a command line Python tool, mirtop, to create and manage the mirGFF3 format. Currently, mirtop can convert into mirGFF3 the outputs of commonly used pipelines, such as seqbuster, isomiR-SEA, sRNAbench, Prost! as well as BAM files. Some tools have also incorporated the mirGFF3 format directly into their code, such as, miRge2.0, IsoMIRmap and OptimiR. Its open architecture enables any tool or pipeline to output or convert results into mirGFF3. Collectively, this isomiR categorization system, along with the accompanying mirGFF3 and mirtop API, provide a comprehensive solution for the standardization of miRNA and isomiR annotation, enabling data sharing, reporting, comparative analyses and benchmarking, while promoting the development of common miRNA methods focusing on downstream steps of miRNA detection, annotation and quantification. https://github.com/miRTop/mirGFF3/ and https://github.com/miRTop/mirtop. Supplementary data are available at Bioinformatics online.