Portable Batch System Research Articles

NGSPERL: a semi-automated framework for large scale next generation sequencing data analysis

High-throughput sequencing technologies have been widely used in medical and biological research, especially in cancer biology. With the huge amounts of sequencing data being generated, data analysis has become the bottle-neck of the research procedure. We have designed and implemented NGSPERL, a semi-automated module-based framework, for high-throughput sequencing data analysis. Three major analysis pipelines with multiple tasks have been developed for RNA sequencing, exome sequencing, and small RNA sequencing data. Each task was developed as module. The module uses the output from the previous task as the input parameter to generate the corresponding portable batch system (PBS) script. The PBS scripts can be either submitted to cluster or run directly based on user choice. Multiple tasks can also be combined together as a single task to simplify the data analysis. Such a flexible framework will significantly automate and simplify the process of large scale sequencing data analysis.

WBSA: Web Service for Bisulfite Sequencing Data Analysis

Whole-Genome Bisulfite Sequencing (WGBS) and genome-wide Reduced Representation Bisulfite Sequencing (RRBS) are widely used to study DNA methylation. However, data analysis is complicated, lengthy, and hampered by a lack of seamless analytical pipelines. To address these issues, we developed a convenient, stable, and efficient web service called Web Service for Bisulfite Sequencing Data Analysis (WBSA) to analyze bisulfate sequencing data. WBSA focuses on not only CpG methylation, which is the most common biochemical modification in eukaryotic DNA, but also non-CG methylation, which have been observed in plants, iPS cells, oocytes, neurons and stem cells of human. WBSA comprises three main modules as follows: WGBS data analysis, RRBS data analysis, and differentially methylated region (DMR) identification. The WGBS and RRBS modules execute read mapping, methylation site identification, annotation, and advanced analysis, whereas the DMR module identifies actual DMRs and annotates their correlations to genes. WBSA can be accessed and used without charge either online or local version. WBSA also includes the executables of the Portable Batch System (PBS) and standalone versions that can be downloaded from the website together with the installation instructions. WBSA is available at no charge for academic users at http://wbsa.big.ac.cn.

Energy-saving control of parallel computer cluster

In order to save the energy in using the parallel computer cluster,a scheme was presented in which Open Portable Batch System(OpenPBS) was used to control the parallel computers' boot and shutdown automatically.Homeostasis can be achieved between task computing and energy saving.

HPC-EPIC for high resolution simulations of environmental and sustainability assessment

Multiple concerns over the impact of wide scale changes in land management have motivated comprehensive analyses of environmental sustainability of food and biofuel production. These call for high-resolution land management tools that enable comprehensive analyses of natural resources for decision-making. The agroecosystem simulation models with the most biophysical detail are point models, which often have a user interface that allows users to provide inputs and examine results for agricultural field scale analyses. These are not able to meet the needs of high-resolution regional or national simulations. We describe an efficient computational approach for deployment of the Environmental Policy Integrated Climate (EPIC) model at high-resolution spatial scales using high performance computing (HPC) techniques. We developed an integrated procedure for executing the millions of simulations required for high-resolution, regional studies, and also address building databases for model initialization, model forcing data, and model outputs. We first ported EPIC from Windows to an HPC platform and validated output from both platforms. We then developed methods of packaging simulations for efficient, unattended parallel execution on the HPC cluster. The job queuing system, Portable Batch System (PBS) is employed to control job submission. Simulation outputs are extracted to PostgreSQL database for analysis. In a case study covering four counties in central Wisconsin using HPC-EPIC, we finished over 140 K simulations in a total of 10 h on an HPC cluster using 20 nodes. This is a speedup of 40 times. More nodes could be used to achieve larger speedups. The HPC-EPIC model developed in this study is anticipated to provide information useful for high-resolution land use management and decision making. The framework for high-performance computing can be extended to other traditional, point-based biophysical simulation models.

XtalOpt version r7: An open-source evolutionary algorithm for crystal structure prediction

A new version of XtalOpt, a user-friendly GPL-licensed evolutionary algorithm for crystal structure prediction, is available for download from the CPC library or the XtalOpt website, http://xtalopt.openmolecules.net. The new version now supports four external geometry optimization codes (VASP, GULP, PWSCF, and CASTEP), as well as three queuing systems: PBS, SGE, SLURM, and “Local”. The local queuing system allows the geometry optimizations to be performed on the userʼs workstation if an external computational cluster is unavailable. Support for the Windows operating system has been added, and a Windows installer is provided. Numerous bugfixes and feature enhancements have been made in the new release as well. New version program summaryProgram title:XtalOptCatalogue identifier: AEGX_v2_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEGX_v2_0.htmlProgram obtainable from: CPC Program Library, Queenʼs University, Belfast, N. IrelandLicensing provisions: GPL v2.1 or later [1]No. of lines in distributed program, including test data, etc.: 125 383No. of bytes in distributed program, including test data, etc.: 11 607 415Distribution format: tar.gzProgramming language: C++Computer: PCs, workstations, or clustersOperating system: Linux, MS WindowsClassification: 7.7External routines: Qt [2], Open Babel [3], Avogadro [4], and one of: VASP [5], PWSCF [6], GULP [7], CASTEP [8]Catalogue identifier of previous version: AEGX_v1_0Journal reference of previous version: Comput. Phys. Comm. 182 (2011) 372Does the new version supersede the previous version?: YesNature of problem: Predicting the crystal structure of a system from its stoichiometry alone remains a grand challenge in computational materials science, chemistry, and physics.Solution method: Evolutionary algorithms are stochastic search techniques which use concepts from biological evolution in order to locate the global minimum of a crystalline structure on its potential energy surface. Our evolutionary algorithm, XtalOpt, is freely available for use and collaboration under the GNU Public License. See the original publication on XtalOptʼs implementation [11] for more information on the method.Reasons for new version: Since XtalOptʼs initial release in June 2010, support for additional optimizers, queuing systems, and an operating system has been added. XtalOpt can now use VASP, GULP, PWSCF, or CASTEP to perform local geometry optimizations. The queue submission code has been rewritten, and now supports running any of the above codes on ssh-accessible computer clusters that use the Portable Batch System (PBS), Sun Grid Engine (SGE), or SLURM queuing systems for managing the optimization jobs. Alternatively, geometry optimizations may be performed on the userʼs workstation using the new internal “Local” queuing system if high performance computing resources are unavailable. XtalOpt has been built and tested on the Microsoft Windows operating system (XP or later) in addition to Linux, and a Windows installer is provided. The installer includes a development version of Avogadro that contains expanded crystallography support [12] that is not available in the mainline Avogadro releases. Other notable new developments include:•LIBSSH [10] is distributed with the XtalOpt sources and used for communication with the remote clusters, eliminating the previous requirement to set up public-key authentication;•Plotting enthalpy (or energy) vs. structure number in the plot tab will trace out the history of the most stable structure as the search progresses A read-only mode has been added to allow inspection of previous searches through the user interface without connecting to a cluster or submitting new jobs;•The tutorial [13] has been rewritten to reflect the changes to the interface and the newly supported codes. Expanded sections on optimizations schemes and save/resume have been added;•The included version of SPGLIB has been updated. An option has been added to set the Cartesian tolerance of the space group detection. A new option has been added to the Progress tableʼs right-click menu that copies the selected structureʼs POSCAR formatted representation to the clipboard;•Numerous other small bugfixes/enhancements.Summary of revisions: See “Reasons for new version” above.Running time: User dependent. The program runs until stopped by the user.

Integrated Web-Based ab initio Modeling Platform for G-protein Coupled Receptors

A web-based G-protein coupled receptor (GPCR) ab initio modeling platform has been developed to support scientists involved in internal in silico drug discovery projects. Starting from a target GPCR sequence, the ab initio modeling workflow predicts an ensemble of 3-dimensional (3D) receptor structures which can subsequently be used for structure- based drug design activities. The workflow used to develop the web-based tools integrates MembStruk and Modeler application packages with significant improvements and enhancements made to the former. The workflow enabled through the web-based user interface encompasses the following steps: a) prediction of the transmembrane (TM) regions using the target GPCR sequence and other similar sequences; b) initial packing of the TM-helices; c) optimization of the TM-helical bundles by translation, rotation, and relaxation of the helices; d) addition of intra- and extra-cellular loops; e) final refinement of the 3D receptor structures including side chains and loops. A wizard-like workflow approach implemented in web-based GPCR modeling tools hides the complexity involved in various modeling steps and guides the user through the calculation one step at a time starting from a target sequence to a set of predicted 3D structures. The successful implementation of the workflow is presented by modeling the human adenosine A2A receptor (AA2AR) structure without the bound ligand and comparing the predicted structure of the TM-helices with the recently reported X-ray crystal structure. Keywords: A2a receptor, ab initio modeling, GPCR modeling, Structure-based drug design, Web-based tools, Workflow automation, G-protein coupled receptor, 3-dimensional, transmembrane, adenosine A2A receptor, X-ray crystal structure, crystallography, molecular dynamics, Eisenberg hydrophobicty, lipid bilayers, cell multiple method, 3D macromolecular structures, Portable Batch System, Linux cluster, rhodopsin like, metabotropic glutamate, pheromone, Root Mean Square Deviation, C atoms, canonical helix structure, dilauroylphosphatidylcholine

Experiments with Storage and Preservation of NASA's Planetary Data via the Cloud

The computing and storage demands force to optimize and manage complex and often conflicting software engineering challenges. Several domain-specific, independent software solutions have been developed to manage large amounts of data, including grid computing platforms- specifically, data-grid software packages such as the Globus Toolkit, DSpace, and OODT (ObjectOriented Data Technology). In addition, several computationally focused software products are geared toward executing large numbers of jobs, including workflow technologies such as Condor and Pegasus, and batch submission systems like the Portable Batch System (PBS) and Torque. The use of cloud computing in NASA's Planetary Data System for large-volume data storage and preservation illustrates how clouds can help researchers meet modern data backup demands, which are approaching the petabyte scale.

GIMSAN: a Gibbs motif finder with significance analysis

We present GIMSAN (GIbbsMarkov with Significance ANalysis): a novel tool for de novo motif finding. GIMSAN combines GibbsMarkov, our variant of the Gibbs Sampler, described here for the first time, with our recently introduced significance analysis. GIMSAN is currently available as a web application and a stand-alone application on Unix and PBS (Portable Batch System) cluster through links from http://www.cs.cornell.edu/~keich.

BaBar MC production on the Canadian grid using a web services approach

The present paper highlights the approach used to design and implement a web services based BaBar Monte Carlo (MC) production grid using Globus Toolkit version 4. The grid integrates the resources of two clusters at the University of Victoria, using the ClassAd mechanism provided by the Condor-G metascheduler. Each cluster uses the Portable Batch System (PBS) as its local resource management system (LRMS). Resource brokering is provided by the Condor matchmaking process, whereby the job and resource attributes are expressed as ClassAds. The important features of the grid are automatic registering of resource ClassAds to the central registry, ClassAds extraction from the registry to the metascheduler for matchmaking, and the incorporation of input/output file staging. Web-based monitoring is employed to track the status of grid resources and the jobs for an efficient operation of the grid. The performance of this new grid for BaBar jobs, and the existing Canadian computational grid (GridX1) based on Globus Toolkit version 2 is found to be consistent.

CGKB: an annotation knowledge base for cowpea (Vigna unguiculata L.) methylation filtered genomic genespace sequences.

BackgroundCowpea [Vigna unguiculata (L.) Walp.] is one of the most important food and forage legumes in the semi-arid tropics because of its ability to tolerate drought and grow on poor soils. It is cultivated mostly by poor farmers in developing countries, with 80% of production taking place in the dry savannah of tropical West and Central Africa. Cowpea is largely an underexploited crop with relatively little genomic information available for use in applied plant breeding. The goal of the Cowpea Genomics Initiative (CGI), funded by the Kirkhouse Trust, a UK-based charitable organization, is to leverage modern molecular genetic tools for gene discovery and cowpea improvement. One aspect of the initiative is the sequencing of the gene-rich region of the cowpea genome (termed the genespace) recovered using methylation filtration technology and providing annotation and analysis of the sequence data.DescriptionCGKB, Cowpea Genespace/Genomics Knowledge Base, is an annotation knowledge base developed under the CGI. The database is based on information derived from 298,848 cowpea genespace sequences (GSS) isolated by methylation filtering of genomic DNA. The CGKB consists of three knowledge bases: GSS annotation and comparative genomics knowledge base, GSS enzyme and metabolic pathway knowledge base, and GSS simple sequence repeats (SSRs) knowledge base for molecular marker discovery. A homology-based approach was applied for annotations of the GSS, mainly using BLASTX against four public FASTA formatted protein databases (NCBI GenBank Proteins, UniProtKB-Swiss-Prot, UniprotKB-PIR (Protein Information Resource), and UniProtKB-TrEMBL). Comparative genome analysis was done by BLASTX searches of the cowpea GSS against four plant proteomes from Arabidopsis thaliana, Oryza sativa, Medicago truncatula, and Populus trichocarpa. The possible exons and introns on each cowpea GSS were predicted using the HMM-based Genscan gene predication program and the potential domains on annotated GSS were analyzed using the HMMER package against the Pfam database. The annotated GSS were also assigned with Gene Ontology annotation terms and integrated with 228 curated plant metabolic pathways from the Arabidopsis Information Resource (TAIR) knowledge base. The UniProtKB-Swiss-Prot ENZYME database was used to assign putative enzymatic function to each GSS. Each GSS was also analyzed with the Tandem Repeat Finder (TRF) program in order to identify potential SSRs for molecular marker discovery. The raw sequence data, processed annotation, and SSR results were stored in relational tables designed in key-value pair fashion using a PostgreSQL relational database management system. The biological knowledge derived from the sequence data and processed results are represented as views or materialized views in the relational database management system. All materialized views are indexed for quick data access and retrieval. Data processing and analysis pipelines were implemented using the Perl programming language. The web interface was implemented in JavaScript and Perl CGI running on an Apache web server. The CPU intensive data processing and analysis pipelines were run on a computer cluster of more than 30 dual-processor Apple XServes. A job management system called Vela was created as a robust way to submit large numbers of jobs to the Portable Batch System (PBS).ConclusionCGKB is an integrated and annotated resource for cowpea GSS with features of homology-based and HMM-based annotations, enzyme and pathway annotations, GO term annotation, toolkits, and a large number of other facilities to perform complex queries. The cowpea GSS, chloroplast sequences, mitochondrial sequences, retroelements, and SSR sequences are available as FASTA formatted files and downloadable at CGKB. This database and web interface are publicly accessible at .

Libra: a computational economy‐based job scheduling system for clusters

AbstractClusters of computers have emerged as mainstream parallel and distributed platforms for high‐performance, high‐throughput and high‐availability computing. To enable effective resource management on clusters, numerous cluster management systems and schedulers have been designed. However, their focus has essentially been on maximizing CPU performance, but not on improving the value of utility delivered to the user and quality of services. This paper presents a new computational economy driven scheduling system called Libra, which has been designed to support allocation of resources based on the users' quality of service requirements. It is intended to work as an add‐on to the existing queuing and resource management system. The first version has been implemented as a plugin scheduler to the Portable Batch System. The scheduler offers market‐based economy driven service for managing batch jobs on clusters by scheduling CPU time according to user‐perceived value (utility), determined by their budget and deadline rather than system performance considerations. The Libra scheduler has been simulated using the GridSim toolkit to carry out a detailed performance analysis. Results show that the deadline and budget based proportional resource allocation strategy improves the utility of the system and user satisfaction as compared with system‐centric scheduling strategies. Copyright © 2004 John Wiley & Sons, Ltd.

What is the GRID

What is the GRID