Large-scale Prediction Research Articles

Comparing recurrent convolutional neural networks for large scale bird species classification

We present a deep learning approach towards the large-scale prediction and analysis of bird acoustics from 100 different bird species. We use spectrograms constructed on bird audio recordings from the Cornell Bird Challenge (CBC)2020 dataset, which includes recordings of multiple and potentially overlapping bird vocalizations with background noise. Our experiments show that a hybrid modeling approach that involves a Convolutional Neural Network (CNN) for learning the representation for a slice of the spectrogram, and a Recurrent Neural Network (RNN) for the temporal component to combine across time-points leads to the most accurate model on this dataset. We show results on a spectrum of models ranging from stand-alone CNNs to hybrid models of various types obtained by combining CNNs with other CNNs or RNNs of the following types: Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRU), and Legendre Memory Units (LMU). The best performing model achieves an average accuracy of 67% over the 100 different bird species, with the highest accuracy of 90% for the bird species, Red crossbill. We further analyze the learned representations visually and find them to be intuitive, where we find that related bird species are clustered close together. We present a novel way to empirically interpret the representations learned by the LMU-based hybrid model which shows how memory channel patterns change over time with the changes seen in the spectrograms.

Reducing computational cost of large-scale genomic evaluation by using indirect genomic prediction

Over half a million Holsteins are being genotyped annually in the United States. The computational cost of including all genotypes in single-step genomic (ssG)BLUP is high, although it is feasible to conduct large-scale genomic prediction using an efficient algorithm such as APY (algorithm for proven and young). An effective method to further reduce the computing cost could be the use of indirect genomic predictions (IGP) for genotyped animals when they have neither progeny nor phenotypes. These young genotyped animals have no effect on the other genotyped animals and could have their genomic prediction done indirectly. The main objective of this study was to calculate IGP for various groups of genotyped animals and investigate the reduction in computing time as well as bias and accuracy of the IGP. We compared IGP with genomic (G)EBV for 18 linear type traits in US Holsteins, including 2.3 million (M) genotyped animals. The full data set consisted of 10.9M records for 18 linear type traits up to 2018 calving, 13.6M animals in the pedigree, and 2.3M animals genotyped for 79K SNP. For IGP, ssGBLUP included all genotyped animals except those with neither progeny nor phenotypes by year from 2014 to 2018 (i.e., the target animals). The SNP marker effects were computed based on GEBV for genotyped animals that had progeny, or phenotypes, or both. Further, IGP were calculated for target genotyped animals in each year group. For all genotyped animal groups from 2014 to 2018, the coefficients of determination (R2) of a linear regression of GEBV on IGP were 0.960 for males and 0.954 for females for 18 traits on average. To reduce computing costs, the SNP marker effects were calculated based on GEBV from randomly selected genotyped animals from 15K to 60K. By randomly selecting a small number of genotyped animals, the computing time was dramatically reduced. As more genotyped animals were randomly selected to calculate SNP effects, R2 was higher (more accurate) and the regression coefficient was lower (more inflated IGP). In a practical genomic evaluation in US Holsteins, to get sufficient contributions from GEBV, 25K to 35K is a rational number of genotyped animals that can be randomly selected to compute SNP effects and obtain accurate and unbiased IGP. Considering the computing time and both unbiasedness and accuracy of IGP, genomic evaluation can be conducted separately in GEBV for genotyped animals with phenotypes or progeny and in IGP for young genotyped animals. This can be a practical solution when conducting a large-scale genomic evaluation and would enable more frequent evaluation at lower cost, especially when many genotyped animals have neither phenotypes nor progeny.

Functional materials exploration through evolutionary searching and large-scale crystal structure prediction

Functional materials exploration through evolutionary searching and large-scale crystal structure prediction

Research on Actual Road Emission Prediction Model of Heavy-Duty Diesel Vehicles Based on OBD Remote Method and Artificial Neural Network

In order to establish a prediction model of PM and NOx emission factors for heavy-duty diesel vehicles under actual road conditions based on OBD remote monitoring and big data, this paper carried out actual road tests on two China V heavy-duty diesel vehicles to obtain transient OBD and emission data by a Portable Emission Measurement System (PEMS) and self-developed On-board Remote Emission Measurement System (OREMS). According to the degree of influence of different parameters in the engine OBD on PM and NOx emissions, the principal component analysis method is used to extract the principal component parameters used to predict the model input, and the construction of a "Heavy-duty Diesel Vehicle Predictive Model based on Remote Monitoring Data and Neural Network Technology". Finally, the predictive model is trained and verified by PEMS test data. The prediction model provides new means and methods for the future development of large-scale heavy-duty diesel vehicle NOx and PM emission predictions under actual road operation.

Highly accurate protein structure prediction for the human proteome

Protein structures can provide invaluable information, both for reasoning about biological processes and for enabling interventions such as structure-based drug development or targeted mutagenesis. After decades of effort, 17% of the total residues in human protein sequences are covered by an experimentally determined structure1. Here we markedly expand the structural coverage of the proteome by applying the state-of-the-art machine learning method, AlphaFold2, at a scale that covers almost the entire human proteome (98.5% of human proteins). The resulting dataset covers 58% of residues with a confident prediction, of which a subset (36% of all residues) have very high confidence. We introduce several metrics developed by building on the AlphaFold model and use them to interpret the dataset, identifying strong multi-domain predictions as well as regions that are likely to be disordered. Finally, we provide some case studies to illustrate how high-quality predictions could be used to generate biological hypotheses. We are making our predictions freely available to the community and anticipate that routine large-scale and high-accuracy structure prediction will become an important tool that will allow new questions to be addressed from a structural perspective.

A co-fractionation mass spectrometry-based prediction of protein complex assemblies in the developing rice aleurone-subaleurone.

Multiprotein complexes execute and coordinate diverse cellular processes such as organelle biogenesis, vesicle trafficking, cell signaling, and metabolism. Knowledge about their composition and localization provides useful clues about the mechanisms of cellular homeostasis and system-level control. This is of great biological importance and practical significance in heterotrophic rice (Oryza sativa) endosperm and aleurone-subaleurone tissues, which are a primary source of seed vitamins and stored energy. Dozens of protein complexes have been implicated in the synthesis, transport, and storage of seed proteins, lipids, vitamins, and minerals. Mutations in protein complexes that control RNA transport result in aberrant endosperm with shrunken and floury phenotypes, significantly reducing seed yield and quality. The purpose of this study was to broadly predict protein complex composition in the aleurone-subaleurone layers of developing rice seeds using co-fractionation mass spectrometry. Following orthogonal chromatographic separations of biological replicates, thousands of protein elution profiles were subjected to distance-based clustering to enable large-scale multimerization state measurements and protein complex predictions. The predicted complexes had predicted functions across diverse functional categories, including novel heteromeric RNA binding protein complexes that may influence seed quality. This effective and open-ended proteomics pipeline provides useful clues about system-level posttranslational control during the early stages of rice seed development.

Protein Interaction Network-based Deep Learning Framework for Identifying Disease-Associated Human Proteins

Infectious diseases in humans appear to be one of the most primary public health issues. Identification of novel disease-associated proteins will furnish an efficient recognition of the novel therapeutic targets. Here, we develop a Graph Convolutional Network (GCN)-based model called PINDeL to identify the disease-associated host proteins by integrating the human Protein Locality Graph and its corresponding topological features. Because of the amalgamation of GCN with the protein interaction network, PINDeL achieves the highest accuracy of 83.45% while AUROC and AUPRC values are 0.90 and 0.88, respectively. With high accuracy, recall, F1-score, specificity, AUROC, and AUPRC, PINDeL outperforms other existing machine-learning and deep-learning techniques for disease gene/protein identification in humans. Application of PINDeL on an independent dataset of 24320 proteins, which are not used for training, validation, or testing purposes, predicts 6448 new disease-protein associations of which we verify 3196 disease-proteins through experimental evidence like disease ontology, Gene Ontology, and KEGG pathway enrichment analyses. Our investigation informs that experimentally-verified 748 proteins are indeed responsible for pathogen-host protein interactions of which 22 disease-proteins share their association with multiple diseases such as cancer, aging, chem-dependency, pharmacogenomics, normal variation, infection, and immune-related diseases. This unique Graph Convolution Network-based prediction model is of utmost use in large-scale disease-protein association prediction and hence, will provide crucial insights on disease pathogenesis and will further aid in developing novel therapeutics.

Large-Scale Membrane Permeability Prediction of Cyclic Peptides Crossing a Lipid Bilayer Based on Enhanced Sampling Molecular Dynamics Simulations.

Membrane permeability is a significant obstacle facing the development of cyclic peptide drugs. However, membrane permeation mechanisms are poorly understood. To investigate common features of permeable (and nonpermeable) designs, it is necessary to reproduce the membrane permeation process of cyclic peptides through the lipid bilayer. We simulated the membrane permeation process of 100 six-residue cyclic peptides across the lipid bilayer based on steered molecular dynamics (MD) and replica-exchange umbrella sampling simulations and predicted membrane permeability using the inhomogeneous solubility-diffusion model and a modified version of it. Furthermore, we confirmed the effectiveness of this protocol by predicting the membrane permeability of 56 eight-residue cyclic peptides with diverse chemical structures, including some confidential designs from a pharmaceutical company. As a result, a reasonable correlation between experimentally assessed and calculated membrane permeability of cyclic peptides was observed for the peptide libraries, except for strongly hydrophobic peptides. Our analysis of the MD trajectory demonstrated that most peptides were stabilized in the boundary region between bulk water and membrane and that for most peptides, the process of crossing the center of the membrane is the main obstacle to membrane permeation. The height of this barrier is well correlated with the electrostatic interaction between the peptide and the surrounding media. The structural and energetic features of the representative peptide at each vertical position within the membrane were also analyzed, revealing that peptides permeate the membrane by changing their orientation and conformation according to the surrounding environment.

A Novel Feature Extraction Model for Large-Scale Workload Prediction in Cloud Environment

A Novel Feature Extraction Model for Large-Scale Workload Prediction in Cloud Environment

The efficacy of ethnic specific blood groups genotyping for routine donor investigation and rare donor identification in Taiwan.

Large-scale single nucleotide variation (SNV)-based blood group genotyping assays have been made available for over a decade. Due to differences in ethnic groups, there is much diversity in clinically important blood group antigens and genetic variants. Here, we developed a robust matrix-assisted laser desorption/ionization time-of-flight (MALDI-TOF)-based blood group genotyping method on MassARRAY system. A total of 1428 donors were enrolled into three groups: (a) reagent red cell donors; (b) rare donor or common antigen-negative donors; and (c) group O, R1 R1 /R2 R2 donors. Forty-two SNVs were designed for determining nine blood groups, with X/Y chromosome in two multiplex reactions, on MassARRAY 96-well format system. Further targeted sequence analyses were performed by Sanger sequencing. WHO reference reagent (NIBSC code: 11/214) was tested for concordance with the provided genotype results. Among the donors, concordance rate was over 99%. Alleles of important phenotypes such as Mi(a+), Di(a+), and Asian-type DEL and alleles of rare blood groups such as Fy(a-), Jk(a-b-) and s- were screened. Three types of discrepancies were found. Serologically, the 'N' antigen was expressed on genetically MM with GYP*Mur red blood cells and caused genuine discrepancies (9.5%). Genetically, allele dropout (ADO) was caused by rare SNV in the primer for Ss genotype (2.1%) and partial insertion of RHD genes (0.9%) led to difficulties in predicting phenotypes. Hemo panel module and MassARRAY System in 96-well format showed good performance in terms of large-scale blood group genotyping and phenotype predictions. Implementation of this method is effective for routine blood group genotype screening of donors.

E-Pedigrees: a large-scale automatic family pedigree prediction application.

The use and functionality of Electronic Health Records (EHR) have increased rapidly in the past few decades. EHRs are becoming an important depository of patient health information and can capture family data. Pedigree analysis is a longstanding and powerful approach that can gain insight into the underlying genetic and environmental factors in human health, but traditional approaches to identifying and recruiting families are low-throughput and labor-intensive. Therefore, high-throughput methods to automatically construct family pedigrees are needed. We developed a stand-alone application: Electronic Pedigrees, or E-Pedigrees, which combines two validated family prediction algorithms into a single software package for high throughput pedigrees construction. The convenient platform considers patients' basic demographic information and/or emergency contact data to infer high-accuracy parent-child relationship. Importantly, E-Pedigrees allows users to layer in additional pedigree data when available and provides options for applying different logical rules to improve accuracy of inferred family relationships. This software is fast and easy to use, is compatible with different EHR data sources, and its output is a standard PED file appropriate for multiple downstream analyses. The Python 3.3+ version E-Pedigrees application is freely available on: https://github.com/xiayuan-huang/E-pedigrees.

Large-scale radio propagation path loss measurements and predictions in the VHF and UHF bands

For decades now, a lot of radio wave path loss propagation models have been developed for predictions across different environmental terrains. Amongst these models, empirical models are practically the most popular due to their ease of application. However, their prediction accuracies are not as high as required. Therefore, extensive path loss measurement data are needed to develop novel measurement-oriented path loss models with suitable correction factors for varied frequency, capturing both local terrain and clutter information, this have been found to be relatively expensive. In this paper, a large-scale radio propagation path loss measurement campaign was conducted across the VHF and UHF frequencies. A multi-transmitter propagation set-up was employed to measure the strengths of radio signals from seven broadcasting transmitters (operating at 89.30, 103.5, 203.25, 479.25, 615.25, 559.25 and 695.25 MHz respectively) at various locations covering a distance of 145.5 km within Nigerian urban environments. The measurement procedure deployed ensured that the data obtained strictly reflect the shadowing effects on radio signal propagation by filtering out the small-scale fading components. The paper also, examines the feasibilities of applying Kriging method to predict distanced-based path losses in the VHF and UHF bands. This method was introduced to minimize the cost of measurements, analysis and predictions of path losses in built-up propagation environments.

NetGO 2.0: improving large-scale protein function prediction with massive sequence, text, domain, family and network information.

With the explosive growth of protein sequences, large-scale automated protein function prediction (AFP) is becoming challenging. A protein is usually associated with dozens of gene ontology (GO) terms. Therefore, AFP is regarded as a problem of large-scale multi-label classification. Under the learning to rank (LTR) framework, our previous NetGO tool integrated massive networks and multi-type information about protein sequences to achieve good performance by dealing with all possible GO terms (>44 000). In this work, we propose the updated version as NetGO 2.0, which further improves the performance of large-scale AFP. NetGO 2.0 also incorporates literature information by logistic regression and deep sequence information by recurrent neural network (RNN) into the framework. We generate datasets following the critical assessment of functional annotation (CAFA) protocol. Experiment results show that NetGO 2.0 outperformed NetGO significantly in biological process ontology (BPO) and cellular component ontology (CCO). In particular, NetGO 2.0 achieved a 12.6% improvement over NetGO in terms of area under precision-recall curve (AUPR) in BPO and around 2.6% in terms of n}{}mathbf {F_{max}} in CCO. These results demonstrate the benefits of incorporating text and deep sequence information for the functional annotation of BPO and CCO. The NetGO 2.0 web server is freely available at http://issubmission.sjtu.edu.cn/ng2/.

Structure determination of an amorphous drug through large-scale NMR predictions

Knowledge of the structure of amorphous solids can direct, for example, the optimization of pharmaceutical formulations, but atomic-level structure determination in amorphous molecular solids has so far not been possible. Solid-state nuclear magnetic resonance (NMR) is among the most popular methods to characterize amorphous materials, and molecular dynamics (MD) simulations can help describe the structure of disordered materials. However, directly relating MD to NMR experiments in molecular solids has been out of reach until now because of the large size of these simulations. Here, using a machine learning model of chemical shifts, we determine the atomic-level structure of the hydrated amorphous drug AZD5718 by combining dynamic nuclear polarization-enhanced solid-state NMR experiments with predicted chemical shifts for MD simulations of large systems. From these amorphous structures we then identify H-bonding motifs and relate them to local intermolecular complex formation energies.

ConsRM: collection and large-scale prediction of the evolutionarily conserved RNA methylation sites, with implications for the functional epitranscriptome.

Motivation N6-methyladenosine (m6A) is the most prevalent RNA modification on mRNAs and lncRNAs. Evidence increasingly demonstrates its crucial importance in essential molecular mechanisms and various diseases. With recent advances in sequencing techniques, tens of thousands of m6A sites are identified in a typical high-throughput experiment, posing a key challenge to distinguish the functional m6A sites from the remaining 'passenger' (or 'silent') sites. Results: We performed a comparative conservation analysis of the human and mouse m6A epitranscriptomes at single site resolution. A novel scoring framework, ConsRM, was devised to quantitatively measure the degree of conservation of individual m6A sites. ConsRM integrates multiple information sources and a positive-unlabeled learning framework, which integrated genomic and sequence features to trace subtle hints of epitranscriptome layer conservation. With a series validation experiments in mouse, fly and zebrafish, we showed that ConsRM outperformed well-adopted conservation scores (phastCons and phyloP) in distinguishing the conserved and unconserved m6A sites. Additionally, the m6A sites with a higher ConsRM score are more likely to be functionally important. An online database was developed containing the conservation metrics of 177998 distinct human m6A sites to support conservation analysis and functional prioritization of individual m6A sites. And it is freely accessible at: https://www.xjtlu.edu.cn/biologicalsciences/con.

CrepHAN: cross-species prediction of enhancers by using hierarchical attention networks.

Enhancers are important functional elements in genome sequences. The identification of enhancers is a very challenging task due to the great diversity of enhancer sequences and the flexible localization on genomes. Till now, the interactions between enhancers and genes have not been fully understood yet. To speed up the studies of the regulatory roles of enhancers, computational tools for the prediction of enhancers have emerged in recent years. Especially, thanks to the ENCODE project and the advances of high-throughput experimental techniques, a large amount of experimentally verified enhancers have been annotated on the human genome, which allows large-scale predictions of unknown enhancers using data-driven methods. However, except for human and some model organisms, the validated enhancer annotations are scarce for most species, leading to more difficulties in the computational identification of enhancers for their genomes. In this study, we propose a deep learning-based predictor for enhancers, named CrepHAN, which is featured by a hierarchical attention neural network and word embedding-based representations for DNA sequences. We use the experimentally supported data of the human genome to train the model, and perform experiments on human and other mammals, including mouse, cow and dog. The experimental results show that CrepHAN has more advantages on cross-species predictions, and outperforms the existing models by a large margin. Especially, for human-mouse cross-predictions, the area under the receiver operating characteristic (ROC) curve (AUC) score of ROC curve is increased by 0.033∼0.145 on the combined tissue dataset and 0.032∼0.109 on tissue-specific datasets. bcmi.sjtu.edu.cn/∼yangyang/CrepHAN.html. Supplementary data are available at Bioinformatics online.

Linking functional traits and demography to model species-rich communities

It has long been anticipated that relating functional traits to species demography would be a cornerstone for achieving large-scale predictability of ecological systems. If such a relationship existed, species demography could be modeled only by measuring functional traits, transforming our ability to predict states and dynamics of species-rich communities with process-based community models. Here, we introduce a new method that links empirical functional traits with the demographic parameters of a process-based model by calibrating a transfer function through inverse modeling. As a case study, we parameterize a modified Lotka–Volterra model of a high-diversity mountain grassland with static plant community and functional trait data only. The calibrated trait–demography relationships are amenable to ecological interpretation, and lead to species abundances that fit well to the observed community structure. We conclude that our new method offers a general solution to bridge the divide between trait data and process-based models in species-rich ecosystems.

What do we need to predict groundwater nitrate recovery trajectories?

Nitrate contamination affects many of the Earth's aquifers and surface waters. Large-scale predictions of groundwater nitrate trends normally require the characterization of multiple anthropic and natural factors. To assess different approaches for upscaling estimates of nitrate recovery, we tested the influence of hydrological, historical, and biological factors on predictions of future nitrate concentration in aquifers. We tested the factors with a rich hydrogeological dataset from a fractured bedrock catchment in western France (Brittany). A sensitivity analysis performed on a calibrated model of groundwater flow, denitrification, and nitrogen inputs revealed that trends in nitrate concentration can effectively be approximated with a limited number of key parameters. The total mass of nitrate that entered the aquifer since the beginning of the industrial period needs to be characterized, but the shape of the historical nitrogen input time series can be largely simplified without substantially altering the predictions. Aquifer flow and transport processes can be represented by the mean and standard deviation of the residence time distribution, offering a tractable tool to make reasonable predictions at watershed to regional scales. Apparent sensitivity to denitrification rate was primarily attributable to time lags in oxygen depletion, meaning that denitrification can be simplified to an on/off process, defined only by the time needed for nitrate to reach the hypoxic reactive layer. Obtaining these key parameters at large scales is still challenging with currently available information, but the results are promising regarding our future ability to predict nitrate concentration with integrated monitoring and modeling approaches.

Large-Scale Road Network Congestion Pattern Analysis and Prediction Using Deep Convolutional Autoencoder

The transportation system, especially the road network, is the backbone of any modern economy. However, with rapid urbanization, the congestion level has surged drastically, causing a direct effect on the quality of urban life, the environment, and the economy. In this paper, we propose (i) an inexpensive and efficient Traffic Congestion Pattern Analysis algorithm based on Image Processing, which identifies the group of roads in a network that suffers from reoccurring congestion; (ii) deep neural network architecture, formed from Convolutional Autoencoder, which learns both spatial and temporal relationships from the sequence of image data to predict the city-wide grid congestion index. Our experiment shows that both algorithms are efficient because the pattern analysis is based on the basic operations of arithmetic, whereas the prediction algorithm outperforms two other deep neural networks (Convolutional Recurrent Autoencoder and ConvLSTM) in terms of large-scale traffic network prediction performance. A case study was conducted on the dataset from Seoul city.

Super Sites for Advancing Understanding of the Oceanic and Atmospheric Boundary Layers

Abstract Air‐sea interactions are critical to large-scale weather and climate predictions because of the ocean's ability to absorb excess atmospheric heat and carbon and regulate exchanges of momentum, water vapor, and other greenhouse gases. These exchanges are controlled by molecular, turbulent, and wave-driven processes in the atmospheric and oceanic boundary layers. Improved understanding and representation of these processes in models are key for increasing Earth system prediction skill, particularly for subseasonal to decadal time scales. Our understanding and ability to model these processes within this coupled system is presently inadequate due in large part to a lack of data: contemporaneous long-term observations from the top of the marine atmospheric boundary layer (MABL) to the base of the oceanic mixing layer.We propose the concept of “Super Sites” to provide multi-year suites of measurements at specific locations to simultaneously characterize physical and biogeochemical processes within the coupled boundary layers at high spatial and temporal resolution. Measurements will be made from floating platforms, buoys, towers, and autonomous vehicles, utilizing both in-situ and remote sensors. The engineering challenges and level of coordination, integration, and interoperability required to develop these coupled ocean‐atmosphere Super Sites place them in an “Ocean Shot” class.