Flat Files Research Articles

Explainable Machine learning on New Zealand strong motion for PGV and PGA

Estimating ground motion characteristics at various locations as a function of fault characteristics is useful for the proper damage assessment and risk mitigation strategies. This paper explores the application of machine learning approaches to predict peak ground acceleration (PGA) and peak ground velocity (PGV) using New Zealand’s strong motion data. Five machine learning algorithms, namely linear regression, kNN, SVM, Random Forest, and XGBoost, are used in this study. Using the New Zealand flat-file database, the geometric mean of the peak ground motion parameters is used as predictor variables in training the machine learning algorithms. The performance of the chosen algorithms and how they work on PGV and PGA are discussed. The best prediction for PGA is obtained using random forest but for PGV XGboost worked best. The relative importance of various features in the flat file is also presented for the best-performing machine learning algorithm. Although the magnitude of an earthquake is found to be most influential for PGV, rupture distance showed the highest impact for PGA. Finally, the predictions are also explained using SHApley Additive exPlanations (SHAP) for the overall dataset as well as on a sample by sample basis, for a few samples. Pairwise dependency of some features with the highest feature importance is also presented using SHAP.

Bakta: rapid and standardized annotation of bacterial genomes via alignment-free sequence identification.

Command-line annotation software tools have continuously gained popularity compared to centralized online services due to the worldwide increase of sequenced bacterial genomes. However, results of existing command-line software pipelines heavily depend on taxon-specific databases or sufficiently well annotated reference genomes. Here, we introduce Bakta, a new command-line software tool for the robust, taxon-independent, thorough and, nonetheless, fast annotation of bacterial genomes. Bakta conducts a comprehensive annotation workflow including the detection of small proteins taking into account replicon metadata. The annotation of coding sequences is accelerated via an alignment-free sequence identification approach that in addition facilitates the precise assignment of public database cross-references. Annotation results are exported in GFF3 and International Nucleotide Sequence Database Collaboration (INSDC)-compliant flat files, as well as comprehensive JSON files, facilitating automated downstream analysis. We compared Bakta to other rapid contemporary command-line annotation software tools in both targeted and taxonomically broad benchmarks including isolates and metagenomic-assembled genomes. We demonstrated that Bakta outperforms other tools in terms of functional annotations, the assignment of functional categories and database cross-references, whilst providing comparable wall-clock runtimes. Bakta is implemented in Python 3 and runs on MacOS and Linux systems. It is freely available under a GPLv3 license at https://github.com/oschwengers/bakta. An accompanying web version is available at https://bakta.computational.bio.

ClusterSheep: A Graphics Processing Unit-Accelerated Software Tool for Large-Scale Clustering of Tandem Mass Spectra from Shotgun Proteomics.

Modern shotgun proteomics experiments generate gigabytes of spectra every hour, only a fraction of which were utilized to form biological conclusions. Instead of being stored as flat files in public data repositories, this large amount of data can be better organized to facilitate data reuse. Clustering these spectra by similarity can be helpful in building high-quality spectral libraries, correcting identification errors, and highlighting frequently observed but unidentified spectra. However, large-scale clustering is time-consuming. Here, we present ClusterSheep, a method utilizing Graphics Processing Units (GPUs) to accelerate the process. Unlike previously proposed algorithms for this purpose, our method performs true pairwise comparison of all spectra within a precursor mass-to-charge ratio tolerance, thereby preserving the full cluster structures. ClusterSheep was benchmarked against previously reported clustering tools, MS-Cluster, MaRaCluster, and msCRUSH. The software tool also functions as an interactive visualization tool with a persistent state, enabling the user to explore the resulting clusters visually and retrieve the clustering results as desired.

The GA4GH Variation Representation Specification: A computational framework for variation representation and federated identification

SUMMARYMaximizing the personal, public, research, and clinical value of genomic information will require the reliable exchange of genetic variation data. We report here the Variation Representation Specification (VRS, pronounced “verse”), an extensible framework for the computable representation of variation that complements contemporary human-readable and flat file standards for genomic variation representation. VRS provides semantically precise representations of variation and leverages this design to enable federated identification of biomolecular variation with globally consistent and unique computed identifiers. The VRS framework includes a terminology and information model, machine-readable schema, data sharing conventions, and a reference implementation, each of which is intended to be broadly useful and freely available for community use. VRS was developed by a partnership among national information resource providers, public initiatives, and diagnostic testing laboratories under the auspices of the Global Alliance for Genomics and Health (GA4GH).

Enhancing Biotech Data Management through SAP: A Comprehensive Review of Data Ingestion in SAP BW/4HANA

The importance of data-centric decision-making has amplified the role of enterprise data warehousing solutions, leading to an increased demand for the ingestion of data from diverse sources in recent times. SAP BW/4HANA, an on-premises packaged data warehousing solutionfrom SAP offers advanced and flexible data ingestion capabilities to integrate heterogeneous data sources into a unified repository. This paper explores the various data ingestion mechanisms available in SAP BW/4HANA. It reviews the native SAP HANA-based Operational Data Provisioning framework that enhances extraction efficiency. The paper also examines the different source system types supported, including Operational Data Provisioning, SAP HANA, Big Data, and flat files. Additionally, it analyses the batch, real-time and hybrid ingestion patterns facilitated by SAP BW/4HANA. Data provisioning directly with SAP HANA database using tools like SAP Data Services is also discussed. Recommendations are provided for efficient data ingestion based on real-world implementation experience. The insights presented will equip technical teams with comprehensive knowledge regarding ingestion architectures in SAP BW/4HANA aligned to their integration and analytics requirements. By leveraging appropriate ingestion strategies, organizations can optimize the performance of their SAP BW/4HANA implementation and accelerate data-driven business innovation.

CovPDB: a high-resolution coverage of the covalent protein-ligand interactome.

In recent years, the drug discovery paradigm has shifted toward compounds that covalently modify disease-associated target proteins, because they tend to possess high potency, selectivity, and duration of action. The rational design of novel targeted covalent inhibitors (TCIs) typically starts from resolved macromolecular structures of target proteins in their apo or holo forms. However, the existing TCI databases contain only a paucity of covalent protein–ligand (cP–L) complexes. Herein, we report CovPDB, the first database solely dedicated to high-resolution cocrystal structures of biologically relevant cP–L complexes, curated from the Protein Data Bank. For these curated complexes, the chemical structures and warheads of pre-reactive electrophilic ligands as well as the covalent bonding mechanisms to their target proteins were expertly manually annotated. Totally, CovPDB contains 733 proteins and 1,501 ligands, relating to 2,294 cP–L complexes, 93 reactive warheads, 14 targetable residues, and 21 covalent mechanisms. Users are provided with an intuitive and interactive web interface that allows multiple search and browsing options to explore the covalent interactome at a molecular level in order to develop novel TCIs. CovPDB is freely accessible at http://www.pharmbioinf.uni-freiburg.de/covpdb/ and its contents are available for download as flat files of various formats.

Implementation of Data Flow, Predictive Model, and Data Visualization in Corbion Process Analysis System

Corbion Group Netherlands B.V. is a company specialized in food ingredients such as lactic acid which involves fermentation process. Thus, a system was created to monitor on real-time fermentation process in order to gain insight and control of the process. This system is called Corbion Process Analysis System, or CorPAS. CorPAS covers many projects in Corbion, and in general, it extracts data from flat files, then transforms and loads the data into Power BI for data visualization. The data extraction still needs to be manually done by the users by opening the databases and copy all the data into flat files. CorPAS 2.0. was initiated to improve the previous system and add a new feature. Automated data extraction was successfully implemented with data transformation and visualization, alongside with implementation of a new feature, such as data science predictive modeling. The solution was built with R language and data visualization tool such as Power BI.

From a genome assembly to full regulatory network prediction: the case study of Rhodotorula toruloides putative Haa1-regulon

Numerous genomes are sequenced and made available to the community through the NCBI portal. However, and, unlike what happens for gene function annotation, annotation of promoter sequences and the underlying prediction of regulatory associations is mostly unavailable, severely limiting the ability to interpret genome sequences in a functional genomics perspective. Here we present an approach where one can download a genome of interest from NCBI in the GenBank Flat File (.gbff) format and, with a minimum set of commands, have all the information parsed, organized and made available through the platform web interface. Also, the new genomes are compared with a given genome of reference in search of homologous genes, shared regulatory elements and predicted transcription associations. We present this approach within the context of Community YEASTRACT of the YEASTRACT + portal, thus benefiting from immediate access to all the comparative genomics queries offered in the YEASTRACT + portal. Besides the yeast community, other communities can install the platform independently, without any constraints. In this work, we exemplify the usefulness of the presented tool, within Community YEASTRACT, in constructing a dedicated database and analysing the genome of the highly promising oleaginous red yeast species Rhodotorula toruloides currently poorly studied at the genome and transcriptome levels and with limited genome editing tools. Regulatory prediction is based on the conservation of promoter sequences and available regulatory networks. The case-study examined is focused on the Haa1 transcription factor—a key regulator of yeast resistance to acetic acid, an important inhibitor of industrial bioconversion of lignocellulosic hydrolysates. The new tool described here led to the prediction of a RtHaa1 regulon with expected impact in the optimization of R. toruloides robustness for lignocellulosic and pectin-rich residue biorefinery processes.

REDCap on FHIR: Clinical Data Interoperability Services

BackgroundDespite widespread use of electronic data capture (EDC) systems for research and electronic health records (EHR), most transfer of data between EHR and EDC systems is manual and error prone. Increased adoption of Health Level Seven Fast Healthcare Interoperability Resource (FHIR) application programming interfaces (APIs) in recent years by EHR systems has increased the availability of patient data for external applications such as REDCap. ObjectiveDescribe the development of the REDCap Clinical Data Interoperability Services (CDIS) module that provides seamless data exchange between the REDCap research EDC and any EHR system with a FHIR API. CDIS enables end users to independently set up their data collection projects, map EHR data to fields, and adjudicate data transfer without project-by-project involvement from Health Information Technology staff. MethodsWe identified two use cases for EHR data transfer into REDCap. Clinical Data Pull (CDP) automatically pulls EHR data into user-defined REDCap fields and replaces the workflow of having to transcribe or copy and paste data from the EHR. Clinical Data Mart (CDM) collects all specified data for a patient over a given time period and replaces the process of importing EHR data for registries from research databases. With an iterative process, we designed our access control, authentication, variable selection, and mapping interfaces in such a way that end users could easily set up and use CDIS. ResultsSince its release, the REDCap CDIS has been used to pull over 19.5 million data points for 82 projects at Vanderbilt University Medical Center. Software and documentation are available through the REDCap Consortium. ConclusionsThe new REDCap Clinical Data and Interoperability Services (CDIS) module leverages the FHIR standard to enable real-time and direct data extraction from the EHR. Researchers can self-service the mapping and adjudication of EHR data into REDCap. The uptake of CDIS at VUMC and other REDCap consortium sites is improving the accuracy and efficiency of EHR data collection by reducing the need for manual transcription and flat file uploads.

BB-SPEEDset: A Validated Dataset of Broadband Near-Source Earthquake Ground Motions from 3D Physics-Based Numerical Simulations

ABSTRACTThis article introduces a strong-motion dataset of near-source broadband earthquake ground motions from 3D physics-based numerical simulations—named BB-SPEEDset—obtained by the code SPEED (SPectral Elements in Elastodynamics with Discontinuous Galerkin)—developed at Politecnico di Milano, Italy. Taking advantage of the earthquake ground-motion scenarios produced so far by SPEED, in most cases validated against earthquake recordings, the main objective of this work is to construct and validate a dataset of simulated broadband waveforms to be used as a support for characterization and modeling of near-source earthquake ground motions. To pursue this objective, the following steps were necessary, namely: (1) the implementation of an effective workflow suitable to process in an homogeneous format various SPEED simulations; (2) the generation of broadband time histories using a technique based on artificial neural networks, trained on strong-motion records; (3) the creation of a flat file collecting, for each simulated scenario, the most relevant metadata (fault rupture scenario, site response proxies, source-to-site distances) as well as a comprehensive set of ground-motion intensity measures of the processed broadband waveforms (peak ground acceleration, velocity and displacement, spectral ordinates, duration, pulse period, etc.). Finally, a comprehensive set of consistency checks is made to verify the absence of any systematic bias in the trend of the BB-SPEEDset results with respect to the NEar-Source Strong-motion (NESS) version 2.0 near-source recorded ground-motion dataset. Indeed, the main features of near-source ground motion in BB-SPEEDset, ranging from the statistical distributions of peak and integral measures both at short and long periods, the ground-motion attenuation with distance, to the features of impulsive ground motions and directionality effects, are in substantial agreement with those from NESS.

Beyond Taxonomic Identification: Integration of Ecological Responses to a Soil Bacterial 16S rRNA Gene Database.

High-throughput sequencing 16S rRNA gene surveys have enabled new insights into the diversity of soil bacteria, and furthered understanding of the ecological drivers of abundances across landscapes. However, current analytical approaches are of limited use in formalizing syntheses of the ecological attributes of taxa discovered, because derived taxonomic units are typically unique to individual studies and sequence identification databases only characterize taxonomy. To address this, we used sequences obtained from a large nationwide soil survey (GB Countryside Survey, henceforth CS) to create a comprehensive soil specific 16S reference database, with coupled ecological information derived from survey metadata. Specifically, we modeled taxon responses to soil pH at the OTU level using hierarchical logistic regression (HOF) models, to provide information on both the shape of landscape scale pH-abundance responses, and pH optima (pH at which OTU abundance is maximal). We identify that most of the soil OTUs examined exhibited a non-flat relationship with soil pH. Further, the pH optima could not be generalized by broad taxonomy, highlighting the need for tools and databases synthesizing ecological traits at finer taxonomic resolution. We further demonstrate the utility of the database by testing against geographically dispersed query 16S datasets; evaluating efficacy by quantifying matches, and accuracy in predicting pH responses of query sequences from a separate large soil survey. We found that the CS database provided good coverage of dominant taxa; and that the taxa indicating soil pH in a query dataset corresponded with the pH classifications of top matches in the CS database. Furthermore we were able to predict query dataset community structure, using predicted abundances of dominant taxa based on query soil pH data and the HOF models of matched CS database taxa. The database with associated HOF model outputs is released as an online portal for querying single sequences of interest (https://shiny-apps.ceh.ac.uk/ID-TaxER/), and flat files are made available for use in bioinformatic pipelines. The further development of advanced informatics infrastructures incorporating modeled ecological attributes along with new functional genomic information will likely facilitate large scale exploration and prediction of soil microbial functional biodiversity under current and future environmental change scenarios.

NESS2.0: An Updated Version of the Worldwide Dataset for Calibrating and Adjusting Ground-Motion Models in Near Source

ABSTRACT We present an extended and updated version of the worldwide NEar-Source Strong-motion (NESS) flat file, which includes an increased number of moderate-to-strong earthquakes recorded in epicentral area, new source metadata and intensity measures, comprising spectral displacements and fling-step amplitudes retrieved from the extended baseline correction processing of velocity time series. The resulting dataset consists of 81 events with moment magnitude≥5.5 and hypocentral depth shallower than 40 km, corresponding to 1189 three-component waveforms, which are selected to have a maximum source-to-site distance within one fault length. Details on the flat files, metadata, and ground-motion parameters, processing scheme, and statistical findings are presented and discussed. The analysis of these data allows recognizing the presence of distinctive features (such as pulse-like waveforms, large vertical components, and hanging-wall effects) that can be exploited to assess their impact on near-source seismic motion. As an example, we use the NESS2.0 dataset for calibrating an empirical correction factor of a regional ground-motion model (GMM) mainly based on far-field records. In this way, we can adjust the median predictions to account for near-source effects not fully captured by the reference model. The final goal of this work is to promote the use of the NESS2 flat file as a tool to disseminate qualified and referenced near-source data and metadata in the light of improving the constraints of GMMs (both empirical and physics-based) close to the source.

A Beginner's Guide to Conducting Reproducible Research

A Beginner's Guide to Conducting Reproducible Research

Pre-Occupied

Pre-Occupied Jeanne Heifetz (bio) In the Pre-Occupied series, I challenged myself to confront something that terrifies me. I have had death panics since I was eight years old. Ironically, the only real estate I am ever likely to own is a parcel of eight cemetery plots I inherited from my grandfather. The deed to the plots came with a map of the cemetery, which seemed like the logical place to begin to address my fear. Each drawing in this series is based on the map of a different Jewish cemetery, including the ones where my own relatives are buried. (I am not religious, but the historical and familial connection was important: these are all places I could be buried, even though I remain completely unreconciled to the idea of my own death.) I can’t claim that drawing the maps allays my panic. Death remains entirely unknowable terrain: the map can never be the territory. And yet, stripped of identifying text, the cemeteries’ abstract forms are mysteriously compelling, grounding me in the universal human drive to create beauty, order, and ritual in the face of our own mortality. ________ Pre-Occupied is an ongoing series begun in 2016, currently comprising 138 individual works. The drawings are burnished graphite on handmade paper. Portions of the series have been included in group shows at Drive-By Projects, Watertown, MA; Five Points Gallery, Torrington, CT; Odetta Gallery, Brooklyn, NY; Lehman College Gallery, Bronx, NY; Patchogue Arts Center, Patchogue, NY; The Tobacco Factory, Long Island City, NY; 404 Gallery, Brooklyn, NY; BergenPAC Gallery, Englewood, NJ; and in a solo show at the Lobby Gallery of the 14th Street Y in New York City. The full series can be viewed at www.jeanneheifetz.com/pre-occupied. The following is a selection of four images from the series. [End Page 45] Click for larger view View full resolution Pre-Occupied 59. [End Page 46] Click for larger view View full resolution Pre-Occupied 64. [End Page 47] Click for larger view View full resolution Pre-Occupied 90. [End Page 48] Click for larger view View full resolution Pre-Occupied 103. [End Page 50] Jeanne Heifetz Jeanne Heifetz came to visual art by a circuitous route. She has two degrees in English (Harvard AB; NYU MA), spent a year as the writer-in-residence at Exeter, worked at Esquire and American Heritage magazines, and wrote a book on organic food (HarperCollins) and one on the origin of color names (Henry Holt). Heifetz also had a parallel career as a weaver, exhibiting at venues like the Philadelphia Museum of Art Craft Show and the Smithsonian Craft Show. In 2007, she started applying fiber techniques to nontraditional materials; by 2011, her work had shifted entirely from fine craft to fine art. Since then, she has shown in galleries and museums in seventeen states, the United Kingdom, France, Italy, Germany, Australia, and Israel. Her work is in the Drawing Center’s curated registry and the flat files of the Kentler International Drawing Space. She has curated shows at Schema Projects and the Workshop Gallery in Brooklyn and at the Philadelphia Museum of Jewish Art. A fellow at LABA: A Laboratory for Jewish Culture in 2018–19, Heifetz lives and works in Brooklyn. Copyright © 2021 Purdue University

SAMA: a real-time Web search architecture

ABSTRACT With the increasing number of information resources in the World Wide Web, how to retrieve real-time information is becoming very important. The resources include traditional databases, flat files, knowledge bases, topical content and programs. In this paper, we propose a new model for retrieving high-quality data from distributed Web resources over the Internet. Unlike traditional models which are web crawler-based algorithms and collect data prior to indexing, supposedly making the process going offline, the proposed algorithm embeds the crawling process with the searching process to make the process more realistic since data are collected and indexed in real-time fashion. We aim to get high-quality and refreshed results for every user’s query, as well as, for better query expansion and formulation. In our experiments, we found that the proposed approach was highly relevant for most recently searched queries, consisting in its performance and resilient to the drawbacks faced by other techniques.

MetaNetX/MNXref: unified namespace for metabolites and biochemical reactions in the context of metabolic models.

MetaNetX/MNXref is a reconciliation of metabolites and biochemical reactions providing cross-links between major public biochemistry and Genome-Scale Metabolic Network (GSMN) databases. The new release brings several improvements with respect to the quality of the reconciliation, with particular attention dedicated to preserving the intrinsic properties of GSMN models. The MetaNetX website (https://www.metanetx.org/) provides access to the full database and online services. A major improvement is for mapping of user-provided GSMNs to MXNref, which now provides diagnostic messages about model content. In addition to the website and flat files, the resource can now be accessed through a SPARQL endpoint (https://rdf.metanetx.org).

From flat file to interface: Synthesis of personas and analytics for enhanced user understanding

AbstractThe persona approach is a communicative design technique for understanding users. With increasing access to analytics data, personas are more and more generated from online user statistics using big data and algorithmic approaches. This integration offers new opportunities to shift personas from flat files of data presentation to that of interactive interfaces for analytics systems. We illustrate this transition towards the concept of “persona as interface” with a persona analytics system, Automatic Persona Generation (APG). In pushing advancements of both persona and analytics conceptualization, development, and use, the APG system presents a multi‐layered “full‐stack” integration. APG affords three levels of user data presentation, which are (a) the conceptual persona, (b) the analytical user metrics, and (c) the foundational user data. Moving to a “personas as interfaces” approach offers the benefits of both personas and analytics systems and addresses some of the shortcomings of each. We provide results from user experiments of and use cases for APG. The result is a better tool than either persona or analytics along for user understanding.

Extraction of CPRD additional clinical data using R

The Clinical Practice Research Datalink is a nation-wide database of primary healthcare data records in England (UK) linked to several health services. A visit to a health practitioner can result in the digital storing of diagnostic and prescription therapeutic information. Access to patient primary care and linked service data depends on the research in mind; however, typically several flat files that describe patient interactions with a health practitioner are delivered. Some of these files will describe additional data such as the result of medical tests and patient lifestyles, denoted collectively into entity values. This data is used to supplement the medical notes recorded by a general practitioner. We have made available a set of R scripts that reads the clinical flat files, additional clinical flat files and entity values, and returns patient clinical data linked with the requested additional data. We have also included medcode descriptions associated with several entities along with instruction of how to extend the code for additional entities. The code is free to download under the MIT license: https://github.com/acnash/CPRD_Additional_Clinical

A Relational Database of Roman Law Based on Justinian’s <i>Digest</i>

The Digest is the definitive historical sourcebook of Roman law compiled under the Byzantine emperor Justinian I (533 CE). This relational database includes the text of Theodor Mommsen’s authoritative edition of the Digest with accompanying information about its compositional structure and its featured jurists. The data was collected from raw text files and transformed to structured machine-readable form in a Python coding environment. Preprocessed data stored in flat files were loaded to a single super lightweight SQLite database (<7Mb) which chains six tables to each other in many-to-one relationships. The database can be easily browsed, queried and expanded in a variety of free-to-use interfaces. Apart from reliable, efficient and structured information retrieval, the database assists large scale quantitative analyses and it can be used as the starting point for further computer-assisted Roman law projects.

A Link between Machine Learning and Optimization in Ground-Motion Model Development: Weighted Mixed-Effects Regression with Data-Driven Probabilistic Earthquake Classification

ABSTRACTThe steady increase of ground-motion data not only allows new possibilities but also comes with new challenges in the development of ground-motion models (GMMs). Data classification techniques (e.g., cluster analysis) do not only produce deterministic classifications but also probabilistic classifications (e.g., probabilities for each datum to belong to a given class or cluster). One challenge is the integration of such continuous classification in regressions for GMM development such as the widely used mixed-effects model. We address this issue by introducing an extension of the mixed-effects model to incorporate data weighting. The parameter estimation of the mixed-effects model, that is, fixed-effects coefficients of the GMMs and the random-effects variances, are based on the weighted likelihood function, which also provides analytic uncertainty estimates. The data weighting permits for earthquake classification beyond the classical, expert-driven, binary classification based, for example, on event depth, distance to trench, style of faulting, and fault dip angle. We apply Angular Classification with Expectation–maximization, an algorithm to identify clusters of nodal planes from focal mechanisms to differentiate between, for example, interface- and intraslab-type events. Classification is continuous, that is, no event belongs completely to one class, which is taken into account in the ground-motion modeling. The theoretical framework described in this article allows for a fully automatic calibration of ground-motion models using large databases with automated classification and processing of earthquake and ground-motion data. As an example, we developed a GMM on the basis of the GMM by Montalva et al. (2017) with data from the strong-motion flat file of Bastías and Montalva (2016) with ∼2400 records from 319 events in the Chilean subduction zone. Our GMM with the data-driven classification is comparable to the expert-classification-based model. Furthermore, the model shows temporal variations of the between-event residuals before and after large earthquakes in the region.