Iterative Workflow Research Articles

Spherical: an iterative workflow for assembling metagenomic datasets

BackgroundThe consensus emerging from the study of microbiomes is that they are far more complex than previously thought, requiring better assemblies and increasingly deeper sequencing. However, current metagenomic assembly techniques regularly fail to incorporate all, or even the majority in some cases, of the sequence information generated for many microbiomes, negating this effort. This can especially bias the information gathered and the perceived importance of the minor taxa in a microbiome.ResultsWe propose a simple but effective approach, implemented in Python, to address this problem. Based on an iterative methodology, our workflow (called Spherical) carries out successive rounds of assemblies with the sequencing reads not yet utilised. This approach also allows the user to reduce the resources required for very large datasets, by assembling random subsets of the whole in a “divide and conquer” manner.ConclusionsWe demonstrate the accuracy of Spherical using simulated data based on completely sequenced genomes and the effectiveness of the workflow at retrieving lost information for taxa in three published metagenomics studies of varying sizes. Our results show that Spherical increased the amount of reads utilized in the assembly by up to 109% compared to the base assembly. The additional contigs assembled by the Spherical workflow resulted in a significant (P < 0.05) changes in the predicted taxonomic profile of all datasets analysed. Spherical is implemented in Python 2.7 and freely available for use under the MIT license. Source code and documentation is hosted publically at: https://github.com/thh32/Spherical.

Drifting in Timeless Polysemy: Problems of Chronology in Sanskrit Lexicography

Sanskrit lexicography has a long history, yet it does not satisfactorily address diachronic development of word-senses. Traditional Sanskrit lexicographers show little interest in chronological matters, partly because, for a variety of linguistic and textual reasons, Sanskrit sources cannot be dated reliably. I argue here that the creation of a "natively digital" Sanskrit dictionary would improve the situation. An electronic dictionary that draws on a database of semantically categorized examples associated with chronological metadata would serve a double purpose. First, it would enable lexicographers to revise sense-ordering within entries on the basis of the latest discoveries about the relative chronology of the sources. Second, the dataset of semantically categorized examples of word in context could serve as the basis for further quantitative analysis, findings of which could, in turn, feed into the lexicographic database in an iterative loop. This iterative workflow holds the potential to generate new knowledge as to the chronology of the sources. Rather than passively relying on the dating proposed by philologists and historians, lexicographers could thus play an active role in dating their own sources, while reaping the benefits of new chronological knowledge in their dictionaries.

Workflow for Target-Oriented Parametrization of an Enhanced Mechanistic Cell Culture Model.

The goal of this study is to develop a macroscopic mechanistic model describing growth and production within fed-batch cultivations of CHO cells. The model should be used for process characterization as well as for process monitoring including real-time parameter adaptations. The model proved to be able to describe a data-set of 40 processes differing in clones, scales, and process conditions with a normalized root mean square error of approximately 10%. However, due to limited parameter identifiability and limited knowledge about physiologically meaningful parameter values, a broad range of parameters could describe the data with similar quality. This hampered comparison of the model parameters as well as their real-time estimation. Therefore an iterative workflow combining techniques like sensitivity and identifiability analysis, analysis of the specific rates as well as structural adaptations of the parameter space is developed. By applying it the parameter variability could be reduced by 80% with similar predictive power as the original parameters. Summing up, based on a mechanistic CHO model, a generic and transferrable workflow is created for target-oriented parameter estimation in case of limited parameter identifiability. Finally, we suggest a methodology, which fits ideally into the frame of Process Analytical Technology aiming to increase process understanding.

Real-time global illumination by precomputed local reconstruction from sparse radiance probes

We present a direct-to-indirect transport technique that enables accurate real-time rendering of indirect illumination in mostly static scenes of complexity on par with modern games while supporting fully dynamic lights, cameras and diffuse surface materials. Our key contribution is an algorithm for reconstructing the incident radiance field from a sparse set of local samples --- radiance probes --- by incorporating mutual visibility into the reconstruction filter. To compute global illumination, we factorize the direct-to-indirect transport operator into global and local parts, sample the global transport with sparse radiance probes at real-time, and use the sampled radiance field as input to our precomputed local reconstruction operator to obtain indirect radiance. In contrast to previous methods aiming to encode the global direct-to-indirect transport operator, our precomputed data is local in the sense that it needs no long-range interactions between probes and receivers, and every receiver depends only on a small, constant number of nearby radiance probes, aiding compression, storage, and iterative workflows. While not as accurate, we demonstrate that our method can also be used for rendering indirect illumination on glossy surfaces, and approximating global illumination in scenes with large-scale dynamic geometry.

Improving virtual screening of G protein-coupled receptors via ligand-directed modeling.

G protein-coupled receptors (GPCRs) play crucial roles in cell physiology and pathophysiology. There is increasing interest in using structural information for virtual screening (VS) of libraries and for structure-based drug design to identify novel agonist or antagonist leads. However, the sparse availability of experimentally determined GPCR/ligand complex structures with diverse ligands impedes the application of structure-based drug design (SBDD) programs directed to identifying new molecules with a select pharmacology. In this study, we apply ligand-directed modeling (LDM) to available GPCR X-ray structures to improve VS performance and selectivity towards molecules of specific pharmacological profile. The described method refines a GPCR binding pocket conformation using a single known ligand for that GPCR. The LDM method is a computationally efficient, iterative workflow consisting of protein sampling and ligand docking. We developed an extensive benchmark comparing LDM-refined binding pockets to GPCR X-ray crystal structures across seven different GPCRs bound to a range of ligands of different chemotypes and pharmacological profiles. LDM-refined models showed improvement in VS performance over origin X-ray crystal structures in 21 out of 24 cases. In all cases, the LDM-refined models had superior performance in enriching for the chemotype of the refinement ligand. This likely contributes to the LDM success in all cases of inhibitor-bound to agonist-bound binding pocket refinement, a key task for GPCR SBDD programs. Indeed, agonist ligands are required for a plethora of GPCRs for therapeutic intervention, however GPCR X-ray structures are mostly restricted to their inactive inhibitor-bound state.

The value of constrained conjugate-gradient least-squares migration in seismic inversion: Application to a fractured-basement play, Texas Panhandle

Seismic inversion has become almost routine in quantitative 3D seismic interpretation. To ensure the quality of the seismic inversion, the input seismic data need to have a high signal-to-noise ratio. With the current low oil price environment, seismic reprocessing is often preferred over reacquisition to improve data quality. Common filter pairs include forward and inverse [Formula: see text]-[Formula: see text] and Radon transforms. Forward and inverse migrations (i.e., migration and demigration) are a more recently introduced transform pair that, when used together in an iterative workflow, results in a least-squares migration algorithm. Least-squares migration compensates for surface variation in data density and, when combined with a filter applied to prestack migrated images, suppresses the operator and data aliasing. We apply a least-squares migration workflow to a fractured-basement data set from the Texas Panhandle to demonstrate the enhancement in signal-to-noise ratio, the reduction in acquisition footprint and migration artifacts, and the improvement in the P-impedance inversion result.

Better living through operational semantics: an optimizing compiler for radio protocols

Software-defined radio (SDR) promises to bring the flexibility and rapid iterative workflow of software to radio protocol design. Many factors make achieving this promise challenging, not least of which are the high data rates and timing requirements of real-world radio protocols. The Ziria language and accompanying compiler demonstrated that a high-level language can compete in this demanding space, but extracting reusable lessons from this success is difficult due to Ziria's lack of a formal semantics. Our first contribution is a core language, operational semantics, and type system for Ziria. The insight we gained through developing this operational semantics led to our second contribution, consisting of two program transformations. The first is fusion, which can eliminate intermediate queues in Ziria programs. Fusion subsumes many one-off optimizations performed by the original Ziria compiler. The second transformation is pipeline coalescing, which reduces execution overhead by batching IO. Pipeline coalescing relies critically on fusion and provides a much simpler story for the original Ziria compiler's "vectorization" transformation. These developments serve as the basis of our third contribution, a new compiler for Ziria that produces significantly faster code than the original compiler. The new compiler leverages our intermediate language to help eliminate unnecessary memory traffic. As well as providing a firm foundation for the Ziria language, our work on an operational semantics resulted in very real software engineering benefits. These benefits need not be limited to SDR---the core language and accompanying transformations we present are applicable to other domains that require processing streaming data at high speed.

Complementing coal seam gas facies modelling workflows with decompaction based processes

Differential compaction plays a key role in influencing the palaeogeographic organisation of many depositional systems. In the Jurassic Walloon Subgroup, Surat Basin, Eastern Australia, the process of compensational stacking contributes significantly to the complex coal layer architecture and is documented in mine exposure, borehole and seismic datasets. Despite this understanding, current best-practices do not formally consider the mechanics of compensational stacking when populating palaeogeography facies in coal seam gas (CSG) reservoir models. To address this limitation, a hybrid modelling workflow was developed in which numerical rules representing the process of differential compaction are used explicitly to condition an iterative workflow containing traditional geostatistical facies modelling algorithms. The workflow is facilitated by a newly developed open source plugin which allows grid decompaction in Schlumberger PETREL™ 2015. Application of the workflow was tested in a CSG production area containing closely spaced wellbores and a 3D seismic survey. In this area, facies models were constructed using both traditional geostatistical approaches and the newly developed hybrid methodology. Comparison of these models suggests that facies models constructed via unconstrained geostatistical approaches often result in unrepresentative realisations, inconsistent with coal seam architectures as observed in seismic and outcrop. The hybrid geostatistical-forward modelling approach developed during this study was better able to reproduce complex alluvial stacking patterns, particularly with respect to coal seam amalgamation, bifurcation and washout.

Using 3D direct manipulation for real-time structural design exploration

Before a new structure can be built, it must be designed. This design phase is a very important step in the building process. The total cost of the structure and its structural performance are largely dependent on the structural design process. The impact of decisions on the design process is initially high and declines as the design matures. However, few computational tools are available for the conceptual design phase; thus, an opportunity exists to create such tools. In the conventional workflow, the architect uses geometric modeling tools and the engineer uses structural analysis tools in sequential steps. Parametric modeling tools represent an improvement to this workflow, as structural analysis plug-ins are available. This allows the architect or engineer to receive structural feedback at an earlier stage, but still as a sequential step to the geometric modeling. The present work aims to improve this workflow by integrating structural feedback with geometric modeling.The user interfaces of conceptual design tools should be interactive and agile enough to follow the designer’s iterative workflow. Direct manipulation involves human-computer interaction, which enables an interactive user interface. In this user interface style, users can directly manipulate on-screen objects using real-world metaphors, which engages the users with their task and encourages further explorations. This is achieved by reducing the perceptual and cognitive resources required to understand and use the interface. New technologies have opened up the possibility of creating new design tools that make use of very direct manipulation. This possibility is further explored in this thesis through the development of two such applications. The first application makes use of multi-touch tablets. The multi-touch interface has literally closed the gap between humans and computers, enabling very direct manipulation interactions with two-dimensional user interfaces. The developed application is an interactive conceptual design tool with real-time structural feedback that allows the user to quickly input and modify structural models through the use of gestures. The second application extends these concepts and ideas into a three-dimensional user interface using an input device named the Leap Motion Controller.

An Iterative Work Flow for History Matching by Use of Design of Experiment, Response-Surface Methodology, and Markov Chain Monte Carlo Algorithm Applied to Tight Oil Reservoirs

Summary Although a large amount of wells has been extensively drilled to develop unconventional reservoirs, uncertainty associated with reservoir and fracture properties can significantly affect the evaluation of an individual well's performance with simulations. From the history-matching point of view, this uncertainty causes nonuniqueness of solutions that should be obtained with a robust probabilistic method without overly exhausting simulation resources. Because of the challenge in this work, a proxy-based history-matching approach can deliver unique advantages of reducing the computational requirement and providing probabilistic interpretation. The work flow presented in this paper is designed to exploit full ranges of proxy-modeling benefits from prioritizing significant reservoir properties, estimating the variation of model responses, and searching multiple history-matching solutions for reliable probabilistic forecasts. A screening process is introduced at the initial stage of the work flow with design of experiment (DOE) and response-surface methodology (RSM), so the dimensions of proxy models can be reduced. In addition, the proxy models progress through the work flow by iterations that aim to improve their accuracy. This iterative work flow efficiently uses simulation resources by use of all the completed runs to provide more information about the model responses for the subsequent iterations. While the work flow is being iterated, multiple proxy models are explored for history-matching solutions by a Markov chain Monte Carlo (MCMC) algorithm. Then, history-matching solutions are used to evaluate the probability distribution of the long-term estimated ultimate recovery (EUR). Finally, an application of the work flow to a horizontal well in the Middle Bakken is presented in this paper.

Application of a complete workflow for 2D elastic full-waveform inversion to recorded shallow-seismic Rayleigh waves

The S-wave velocity of the shallow subsurface can be inferred from shallow-seismic Rayleigh waves. Traditionally, the dispersion curves of the Rayleigh waves are inverted to obtain the (local) S-wave velocity as a function of depth. Two-dimensional elastic full-waveform inversion (FWI) has the potential to also infer lateral variations. We have developed a novel workflow for the application of 2D elastic FWI to recorded surface waves. During the preprocessing, we apply a line-source simulation (spreading correction) and perform an a priori estimation of the attenuation of waves. The iterative multiscale 2D elastic FWI workflow consists of the preconditioning of the gradients in the vicinity of the sources and a source-wavelet correction. The misfit is defined by the least-squares norm of normalized wavefields. We apply our workflow to a field data set that has been acquired on a predominantly depth-dependent velocity structure, and we compare the reconstructed S-wave velocity model with the result obtained by a 1D inversion based on wavefield spectra (Fourier-Bessel expansion coefficients). The 2D S-wave velocity model obtained by FWI shows an overall depth dependency that agrees well with the 1D inversion result. Both models can explain the main characteristics of the recorded seismograms. The small lateral variations in S-wave velocity introduced by FWI additionally explain the lateral changes of the recorded Rayleigh waves. The comparison thus verifies the applicability of our 2D FWI workflow and confirms the potential of FWI to reconstruct shallow small-scale lateral changes of S-wave velocity.

Improved decision making with new efficient workflows for well placement optimization

Determining optimum infill well placement has been one of most challenging events in overall field development strategy of any types of field. Because there could be a large number of possible candidates for new infill well locations, it is not practically feasible to evaluate all candidate locations, particularly at high-resolution geological models including millions of geological cells. Conventional static measure maps generally used in the industry has limited applicability as this does not address dynamic fluid flow and interference between existing wells. In contrast, direct application of stochastic search optimization methods such as genetic algorithms to large-scale field models may better account for dynamically changing reservoir conditions but can be complex to apply or computationally expensive.We propose a novel method for infill well placement optimization designed as a two-stage optimization based on both dynamic flow diagnostic characteristics and genetic algorithm optimization methods. The main advantage of the proposed method over the conventional approach of using static reservoir property maps is its ability to integrate static reservoir properties with dynamic variables such as sweep efficiencies. By incorporating sweep efficiencies, new infill well potential map is generated to guide the optimization. The other benefits of this method are its maneuverability and capability to search in a narrower and tight space making the evolutionary algorithm an appropriate choice to identify rewarding locations which was not practically applicable due to the significant computational burden.The proposed automated workflow is seamlessly integrated from building a simulation model to performing reservoir simulation and conducting the optimization process. The workflow is combined with novel engineering methods to identify best rewarding well location areas, and to identify strategies to maximize sweep efficiency through optimizing well placement and completion strategies resulting in substantial engineering time-savings. The power and utility of the proposed workflow are first demonstrated by a synthetic example and then applied to a full field model. Strategies to improve incremental oil recoveries were identified with a proposed novel, iterative and robust workflow.

Refinement of arrival-time picks using a cross-correlation based workflow

We propose a new iterative workflow based on cross-correlation for improved arrival-time picking on microseismic data. In this workflow, signal-to-noise ratio (S/N) and polarity weighted stacking are used to minimize the effect of S/N and polarity fluctuations on the pilot waveform computation. We use an exhaustive search technique for polarity estimation through stack power maximization. We use pseudo-synthetic and real microseismic data from western Canada in order to demonstrate the effectiveness of proposed workflow relative to Akaike information criterion (AIC) and a previously published cross-correlation based method. The pseudo-synthetic microseismic waveforms are obtained by introducing Gaussian noise and polarity fluctuations into waveforms from a high S/N microseismic event. We find that the cross-correlation based approaches yield more accurate arrival-time picks as compared to AIC for low S/N waveforms. AIC is not affected by waveform polarities as it works on individual receiver levels whereas the accuracy of existing cross-correlation method decreases in spite of using envelope correlation. We show that our proposed workflow yields better and consistent arrival-time picks regardless of waveform amplitude and polarity variations within the receiver array. After refinement, the initial arrival-time picks are located closer to the best estimated manual picks.

Protein Inference from the Integration of Tandem MS Data and Interactome Networks.

Since proteins are digested into a mixture of peptides in the preprocessing step of tandem mass spectrometry (MS), it is difficult to determine which specific protein a shared peptide belongs to. In recent studies, besides tandem MS data and peptide identification information, some other information is exploited to infer proteins. Different from the methods which first use only tandem MS data to infer proteins and then use network information to refine them, this study proposes a protein inference method named TMSIN, which uses interactome networks directly. As two interacting proteins should co-exist, it is reasonable to assume that if one of the interacting proteins is confidently inferred in a sample, its interacting partners should have a high probability in the same sample, too. Therefore, we can use the neighborhood information of a protein in an interactome network to adjust the probability that the shared peptide belongs to the protein. In TMSIN, a multi-weighted graph is constructed by incorporating the bipartite graph with interactome network information, where the bipartite graph is built with the peptide identification information. Based on multi-weighted graphs, TMSIN adopts an iterative workflow to infer proteins. At each iterative step, the probability that a shared peptide belongs to a specific protein is calculated by using the Bayes' law based on the neighbor protein support scores of each protein which are mapped by the shared peptides. We carried out experiments on yeast data and human data to evaluate the performance of TMSIN in terms of ROC, q-value, and accuracy. The experimental results show that AUC scores yielded by TMSIN are 0.742 and 0.874 in yeast dataset and human dataset, respectively, and TMSIN yields the maximum number of true positives when q-value less than or equal to 0.05. The overlap analysis shows that TMSIN is an effective complementary approach for protein inference.

Towards programmable plant genetic circuits.

Synthetic biology enables the construction of genetic circuits with predictable gene functions in plants. Detailed quantitative descriptions of the transfer function or input-output function for genetic parts (promoters, 5' and 3' untranslated regions, etc.) are collected. These data are then used in computational simulations to determine their robustness and desired properties, thereby enabling the best components to be selected for experimental testing in plants. In addition, the process forms an iterative workflow which allows vast improvement to validated elements with sub-optimal function. These processes enable computational functions such as digital logic in living plants and follow the pathway of technological advances which took us from vacuum tubes to cell phones.

A NEW IDL IMPLEMENTATION OF THE JUPP METHOD FOR BATHYMETRY EXTRACTION IN SHALLOW WATERS

Abstract. There is an ongoing effort in using imagery from remote sensing platforms to obtain information about the sea depth; this allows to monitor the dynamics of coastal erosion without the need for costly and repeated local surveys. We worked on a new implementation of the Jupp method to extract depth information from satellite images. Our software is based on previous implementations of the algorithm in the IDL language, but we made our current implementation more modular in order to make possible experimentations with different approaches. We used this implementation on a series of six images (three from the Landsat TM sensor and three from the Landsat OLI sensor) in order to improve the available tools. We established an iterative workflow for working on the Landat-8 images widely exposed in this paper.

A NEW IDL IMPLEMENTATION OF THE JUPP METHOD FOR BATHYMETRY EXTRACTION IN SHALLOW WATERS

There is an ongoing effort in using imagery from remote sensing platforms to obtain information about the sea depth; this allows to monitor the dynamics of coastal erosion without the need for costly and repeated local surveys. We worked on a new implementation of the Jupp method to extract depth information from satellite images. Our software is based on previous implementations of the algorithm in the IDL language, but we made our current implementation more modular in order to make possible experimentations with different approaches. We used this implementation on a series of six images (three from the Landsat TM sensor and three from the Landsat OLI sensor) in order to improve the available tools. We established an iterative workflow for working on the Landat-8 images widely exposed in this paper.

Optimization of tomographic reconstruction workflows on geographically distributed resources.

New technological advancements in synchrotron light sources enable data acquisitions at unprecedented levels. This emergent trend affects not only the size of the generated data but also the need for larger computational resources. Although beamline scientists and users have access to local computational resources, these are typically limited and can result in extended execution times. Applications that are based on iterative processing as in tomographic reconstruction methods require high-performance compute clusters for timely analysis of data. Here, time-sensitive analysis and processing of Advanced Photon Source data on geographically distributed resources are focused on. Two main challenges are considered: (i) modeling of the performance of tomographic reconstruction workflows and (ii) transparent execution of these workflows on distributed resources. For the former, three main stages are considered: (i) data transfer between storage and computational resources, (i) wait/queue time of reconstruction jobs at compute resources, and (iii) computation of reconstruction tasks. These performance models allow evaluation and estimation of the execution time of any given iterative tomographic reconstruction workflow that runs on geographically distributed resources. For the latter challenge, a workflow management system is built, which can automate the execution of workflows and minimize the user interaction with the underlying infrastructure. The system utilizes Globus to perform secure and efficient data transfer operations. The proposed models and the workflow management system are evaluated by using three high-performance computing and two storage resources, all of which are geographically distributed. Workflows were created with different computational requirements using two compute-intensive tomographic reconstruction algorithms. Experimental evaluation shows that the proposed models and system can be used for selecting the optimum resources, which in turn can provide up to 3.13× speedup (on experimented resources). Moreover, the error rates of the models range between 2.1 and 23.3% (considering workflow execution times), where the accuracy of the model estimations increases with higher computational demands in reconstruction tasks.

Latency-Based Anycast Geolocation: Algorithms, Software, and Data Sets

Use of IP-layer anycast has increased in the last few years beyond the DNS realm. Existing measurement techniques to identify and enumerate anycast replicas exploit specifics of the DNS protocol, which limits their applicability to this particular service. In this paper, we propose and thoroughly validate a protocol-agnostic technique for anycast replicas discovery and geolocation. Furthermore, we also provide the community with open-source software and data sets allowing others to replicate our experimental results, potentially facilitating the development of new techniques such as ours. In particular, our proposed method achieves thorough enumeration and city-level geolocalization of anycast instances from a set of known vantage points. The algorithm features an iterative workflow, pipelining enumeration (an optimization problem using latency as an input), and geolocalization (a classification problem using side channel information, such as city population) of anycast replicas. Results of a thorough validation campaign show our algorithm to be robust to measurement noise, and very lightweight as it requires only a handful of latency measurements.

LATUX: an Iterative Workflow for Designing, Validating and Deploying Learning Analytics Visualisations

Designing, validating and deploying learning analytics tools for instructors or students is a challenge that requires techniques and methods from different disciplines, such as software engineering, human-computer interaction, computer graphics, educational design and psychology. Whilst each of these disciplines has established its own design methodologies, there is a need for methodological frameworks that meet the specific demands of the cross-disciplinary space defined by learning analytics. In particular there is no systematic workflow for producing learning analytics tools that are both technologically feasible and truly underpin the learning experience. In this paper, we present a set of guiding principles and recommendations derived from the LATUX workflow. LATUX is a five-stage workflow to design, validate and deploy awareness interfaces in technology-enabled learning environments. LATUX is grounded on a well-established design process for creating, testing and re-designing user interfaces. We extend this process by integrating the pedagogical requirements, to guide the design of learning analytics visualisations that can inform instructors’ pedagogical decisions or intervention strategies. The workflow is illustrated with a case study in which collaborative activities were deployed in a real classroom. Finally, the paper proposes a research agenda to support designers and implementers of learning analytics interfaces.