Large-scale Prediction Research Articles

Study of crystal properties based on attention mechanism and crystal graph convolutional neural network

The prediction of crystal properties has always been limited by huge computational costs. In recent years, the rise of machine learning methods has gradually made it possible to study crystal properties on a large scale. We propose an attention mechanism-based crystal graph convolutional neural network, which builds a machine learning model by inputting crystallographic information files and target properties. In our research, the attention mechanism is introduced in the crystal graph convolutional neural network (CGCNN) to learn the local chemical environment, and node normalization is added to reduce the risk of overfitting. We collect structural information and calculation data of about 36 000 crystals and examine the prediction performance of the models for the formation energy, total energy, bandgap, and Fermi energy of crystals in our research. Compared with the CGCNN, it is found that the accuracy (ACCU) of the predicted properties can be further improved to varying degrees by the introduction of the attention mechanism. Moreover, the total magnetization and bandgap can be classified under the same neural network framework. The classification ACCU of wide bandgap semiconductor crystals with a bandgap threshold of 2.3 eV reaches 93.2%, and the classification ACCU of crystals with a total magnetization threshold of 0.5 μ B reaches 88.8%. The work is helpful to realize large-scale prediction and classification of crystal properties, accelerating the discovery of new functional crystal materials.

Building a machine learning surrogate model for wildfire activities within a global Earth system model

Abstract. Wildfire is an important ecosystem process, influencing land biogeophysical and biogeochemical dynamics and atmospheric composition. Fire-driven loss of vegetation cover, for example, directly modifies the surface energy budget as a consequence of changing albedo, surface roughness, and partitioning of sensible and latent heat fluxes. Carbon dioxide and methane emitted by fires contribute to a positive atmospheric forcing, whereas emissions of carbonaceous aerosols may contribute to surface cooling. Process-based modeling of wildfires in Earth system land models is challenging due to limited understanding of human, climate, and ecosystem controls on fire counts, fire size, and burned area. Integration of mechanistic wildfire models within Earth system models requires careful parameter calibration, which is computationally expensive and subject to equifinality. To explore alternative approaches, we present a deep neural network (DNN) scheme that surrogates the process-based wildfire model with the Energy Exascale Earth System Model (E3SM) interface. The DNN wildfire model accurately simulates observed burned area with over 90 % higher accuracy with a large reduction in parameterization time compared with the current process-based wildfire model. The surrogate wildfire model successfully captured the observed monthly regional burned area during validation period 2011 to 2015 (coefficient of determination, R2=0.93). Since the DNN wildfire model has the same input and output requirements as the E3SM process-based wildfire model, our results demonstrate the applicability of machine learning for high accuracy and efficient large-scale land model development and predictions.

Traditional and Computational Screening of Non-Toxic Peptides and Approaches to Improving Selectivity.

Peptides have positively impacted the pharmaceutical industry as drugs, biomarkers, or diagnostic tools of high therapeutic value. However, only a handful have progressed to the market. Toxicity is one of the main obstacles to translating peptides into clinics. Hemolysis or hemotoxicity, the principal source of toxicity, is a natural or disease-induced event leading to the death of vital red blood cells. Initial screenings for toxicity have been widely evaluated using erythrocytes as the gold standard. More recently, many online databases filled with peptide sequences and their biological meta-data have paved the way toward hemolysis prediction using user-friendly, fast-access machine learning-driven programs. This review details the growing contributions of in silico approaches developed in the last decade for the large-scale prediction of erythrocyte lysis induced by peptides. After an overview of the pharmaceutical landscape of peptide therapeutics, we highlighted the relevance of early hemolysis studies in drug development. We emphasized the computational models and algorithms used to this end in light of historical and recent findings in this promising field. We benchmarked seven predictors using peptides from different data sets, having 7–35 amino acids in length. According to our predictions, the models have scored an accuracy over 50.42% and a minimal Matthew’s correlation coefficient over 0.11. The maximum values for these statistical parameters achieved 100.0% and 1.00, respectively. Finally, strategies for optimizing peptide selectivity were described, as well as prospects for future investigations. The development of in silico predictive approaches to peptide toxicity has just started, but their important contributions clearly demonstrate their potential for peptide science and computer-aided drug design. Methodology refinement and increasing use will motivate the timely and accurate in silico identification of selective, non-toxic peptide therapeutics.

Software defect prediction based on nested-stacking and heterogeneous feature selection

Software testing guarantees the delivery of high-quality software products, and software defect prediction (SDP) has become an important part of software testing. Software defect prediction is divided into traditional software defect prediction and just-in-time software defect prediction (JIT-SDP). However, most of the existing software defect prediction frameworks are relatively simplified, which makes it extremely difficult to provide developers with more detailed reference information. To improve the effectiveness of software defect prediction and realize effective software testing resource allocation, this paper proposes a software defect prediction framework based on Nested-Stacking and heterogeneous feature selection. The framework includes three stages: data set preprocessing and feature selection, Nested-Stacking classifier, and model classification performance evaluation. The novel heterogeneous feature selection and nested custom classifiers in the framework can effectively improve the accuracy of software defect prediction. This paper conducts experiments on two software defect data sets (Kamei, PROMISE), and demonstrates the classification performance of the model through two comprehensive evaluation indicators, AUC, and F1-score. The experiment carried out large-scale within-project defect prediction (WPDP) and cross-project defect prediction (CPDP). The results show that the framework proposed in this paper has an excellent classification performance on the two types of software defect data sets, and has been greatly improved compared with the baseline models.

High Precision Sea Surface Temperature Prediction of Long Period and Large Area in the Indian Ocean Based on the Temporal Convolutional Network and Internet of Things.

Impacted by global warming, the global sea surface temperature (SST) has increased, exerting profound effects on local climate and marine ecosystems. So far, investigators have focused on the short-term forecast of a small or medium-sized area of the ocean. It is still an important challenge to obtain accurate large-scale and long-term SST predictions. In this study, we used the reanalysis data sets provided by the National Centers for Environmental Prediction based on the Internet of Things technology and temporal convolutional network (TCN) to predict the monthly SSTs of the Indian Ocean from 2014 to 2018. The results yielded two points: Firstly, the TCN model can accurately predict long-term SSTs. In this paper, we used the Pearson correlation coefficient (hereafter this will be abbreviated as “correlation”) to measure TCN model performance. The correlation coefficient between the predicted and true values was 88.23%. Secondly, compared with the CFSv2 model of the American National Oceanic and Atmospheric Administration (NOAA), the TCN model had a longer prediction time and produced better results. In short, TCN can accurately predict the long-term SST and provide a basis for studying large oceanic physical phenomena.

Quantum computer: The state of the problem in the world and in Ukraine

The report analyzes the current state of the problem of creating a quantum computer as a real working device, which the world scientific community identifies as one of the most important fundamental tasks of physics in the XXI century. It is emphasized that the work of the quantum computer is based on the results of basic research in the field of quantum physics. Today, the world is actively developing quantum technologies, which are expected to be implemented in the near future and provide breakthrough results in many areas of public life: from cryptography to modeling large-scale systems, description and prediction of complex processes and more. The contribution of Ukrainian scientists to the study of quantum systems, the development of quantum technologies and the solution of physical problems that arise in the implementation of elements of the quantum computer is considered.

SPOT-Contact-LM: improving single-sequence-based prediction of protein contact map using a transformer language model.

MotivationAccurate prediction of protein contact-map is essential for accurate protein structure and function prediction. As a result, many methods have been developed for protein contact map prediction. However, most methods rely on protein-sequence-evolutionary information, which may not exist for many proteins due to lack of naturally occurring homologous sequences. Moreover, generating evolutionary profiles is computationally intensive. Here, we developed a contact-map predictor utilizing the output of a pre-trained language model ESM-1b as an input along with a large training set and an ensemble of residual neural networks.ResultsWe showed that the proposed method makes a significant improvement over a single-sequence-based predictor SSCpred with 15% improvement in the F1-score for the independent CASP14-FM test set. It also outperforms evolutionary-profile-based methods trRosetta and SPOT-Contact with 48.7% and 48.5% respective improvement in the F1-score on the proteins without homologs (Neff = 1) in the independent SPOT-2018 set. The new method provides a much faster and reasonably accurate alternative to evolution-based methods, useful for large-scale prediction.Availability and implementationStand-alone-version of SPOT-Contact-LM is available at https://github.com/jas-preet/SPOT-Contact-Single. Direct prediction can also be made at https://sparks-lab.org/server/spot-contact-single. The datasets used in this research can also be downloaded from the GitHub.Supplementary information Supplementary data are available at Bioinformatics online.

Large-scale Prediction of Drug-Protein Interactions Based on Network Information.

The prediction of drug-protein interaction (DPI) plays an important role in drug discovery and repositioning. Unfortunately, traditional experimental validation of DPIs is expensive and time-consuming. Therefore, it is necessary to develop in silico methods for the identification of potential DPIs. In this work, the identification of DPIs was performed by the generated recommendation of the unexplored interaction of the drug-protein bipartite graph. Three kinds of recommenders were proposed to predict the potential DPIs. The simulation results showed that the proposed models obtained good performance in crossvalidation and independent test. Our recommendation strategy based on collaborative filtering can effectively improve the DPI identification performance, especially for certain DPIs lacking chemical structure similarity or genomic sequence similarity.

Regional soil organic matter mapping models based on the optimal time window, feature selection algorithm and Google Earth Engine

The spatial distribution of soil organic matter (SOM) is highly significant to the assessment of the regional carbon balance, food security and cultivated land quality. Due to climate change and the increasing food demand, the intensity of cultivated land development in the Northeast China black soil region is increasing, and it is urgent to accurately map the SOM content in this region. Remote sensing technology has been widely applied in the field of soil mapping, but large-scale and high-precision soil mapping remains a significant challenge. In this study, the Google Earth Engine (GEE) platform is adopted to generate synthetic soil images based on Landsat-8 and Sentinel-2 images capturing bare soil periods at 20-d intervals. Then, the spectral index and band are adopted as input variables to evaluate the prediction accuracy of these synthetic images depicting different periods using random forest (RF) regression. Finally, two feature selection methods (Boruta and recursive feature elimination (RFE)) are employed to evaluate the performance of these two methods. The results indicate that 1) the optimal time window for SOM prediction is day of year (DOY) 120–140 for the Songnen Plain; 2) the performance of SOM prediction based on Landsat-8 synthetic images is better than that based on Sentinel-2 synthetic images; and 3) both feature selection methods improve the SOM prediction accuracy, but RFE has the highest accuracy(Landsat-8 with Coefficient of Determination (R2) of 0.702, Root Mean Square Error (RMSE) of 0.681%; Sentinel-2 with R2 of 0.5963, RMSE of 0.793%). This study provides a new model for large-scale and high-spatial resolution SOM prediction and verifies the importance of the time window to the SOM prediction accuracy.

Large-scale cellular traffic prediction based on graph convolutional networks with transfer learning

Large-scale cellular traffic prediction based on graph convolutional networks with transfer learning

Long-term prediction of sea surface chlorophyll-a concentration based on the combination of spatio-temporal features

Harmful algal blooms (HABs) events have a serious impact on marine fisheries and marine management. They occur globally with high frequency and are characterized by a long duration and difficult governance. HABs incidents have occurred in the South China Sea (SCS), and the frequency of occurrence has been on the rise in recent decades. Predicting the long-term chlorophyll-a (Chl-a) concentration has the potential to facilitate long-term monitoring and early warning of HABs events. Currently, long-term predictions of ocean circulation and temperature are common, while long-term predictions of marine biochemistry are still in their infancy. Traditional Chl-a prediction methods have problems, such as low accuracy and the inability to carry out long-term predictions. This research improved the CNN-LSTM model by combining spatio-temporal features to predict Chl-a concentrations. This model can extract both the temporal and spatial features of Chl-a, expand the dataset, and improve the prediction accuracy and training speed. The predictions were made using a Chl-a dataset for the Reed Tablemount in the SCS. The time series of Chl-a used was the satellite data of NASA's official website from January 2002 to June 2020. The results indicate that the predictions of the CNN-LSTM model are better than those of the LSTM and SARIMA models. The five-year long-term rolling prediction of Chl-a was carried out, and the three-year Pearson correlation coefficient reached 0.5. The novelty of this study is the realization of a three-year long-term prediction of Chl-a concentrations. The Mann-Kendall trend test method and the least square method were used to fit the straight line to detect the trend of the five-year predicted value and the true value, respectively. The results indicated that the prediction value and true value of the sea surface Chl-a from 2015 to 2020 both exhibited an overall upward trend. In addition, the prediction performance of the model in large-scale prediction is better than that in small-scale prediction.

A Distributed Framework for Large-scale Protein-protein Interaction Data Analysis and Prediction Using MapReduce

Protein-protein interactions are of great significance for human to understand the functional mechanisms of proteins. With the rapid development of high-throughput genomic technologies, massive protein-protein interaction (PPI) data have been generated, making it very difficult to analyze them efficiently. To address this problem, this paper presents a distributed framework by reimplementing one of state-of-the-art algorithms, i.e., CoFex, using MapReduce. To do so, an in-depth analysis of its limitations is conducted from the perspectives of efficiency and memory consumption when applying it for large-scale PPI data analysis and prediction. Respective solutions are then devised to overcome these limitations. In particular, we adopt a novel tree-based data structure to reduce the heavy memory consumption caused by the huge sequence information of proteins. After that, its procedure is modified by following the MapReduce framework to take the prediction task distributively. A series of extensive experiments have been conducted to evaluate the performance of our framework in terms of both efficiency and accuracy. Experimental results well demonstrate that the proposed framework can considerably improve its computational efficiency by more than two orders of magnitude while retaining the same high accuracy.

A large-scale prediction of protein-protein interactions based on random forest and matrix of sequence

Protein-protein interaction (PPIs) is an important part of many life activities in organisms, and the prediction of protein-protein interactions is closely related to protein function, disease occurrence, and disease treatment. In order to optimize the prediction performance of protein interactions, here a RT-MOS model was constructed based on Random Forest (RF) and Matrix of Sequence (MOS) to predict protein-protein interactions. Firstly, MOS is used to encode the protein sequences into a 29-dimensional feature vector; Then, a prediction model RT-MOS is build based on random forest, and the RT-MOS model is optimized and evaluated using the test set; Finally, the optimized model RT-MOS is used for prediction. The experimental results show that the accuracy rates of the RT-MOS model on the benchmark dataset and the non-redundant dataset are 97.18% and 91.34%, respectively, and the accuracies on four external datasets of C.elegans, Drosophila, E.coli and H.sapiens are 96.21%, 97.86%, 97.54% and 97.75%, respectively. Compared with the existing methods, it is found that it is superior to the existing methods. The experimental results show that the model RT-MOS has the advantages of saving time, preventing overfitting and high accuracy, and is suitable for large-scale PPIs prediction.

A multi-task learning-based framework for global maritime trajectory and destination prediction with AIS data

Automatic Identification System (AIS) was initially developed for tracking ships and avoiding collisions. As numerous small satellites were launched in recent years, millions of AIS messages are captured by these satellite AIS providers every day. The massive amount of data allow shipping firms and port operators to better predict vessels’ movement. In this study, a computational framework is developed to predict future trajectories and destinations of vessels by applying deep learning to the satellite AIS messages containing self-reporting positioning data. We employed deep learning approaches for large-scale maritime prediction and identifies the most suitable deep neural networks for this specific application. Three prediction models, Convolutional Neural Networks (CNN), Dense Neural Networks (DNN), and Long Short-Term Memory (LSTM) have been tested in the framework. A multi-task learning architecture is implemented based on test results. Through sharing parameters while training, this architecture enables long-term destination predictions by balancing short-term trajectory predictions. The framework is validated globally with the experiments covering five large-scale sea areas. The experimental results demonstrate that the LSTM model with an average accuracy percentage of 85.1% outperforms the CNN and DNN with accuracy percentages of 68.3% and 78.2% respectively. The LSTM model with multi-task architecture can achieve an accuracy of 87.0%. With the proposed computational framework, AIS data can be utilized for maritime predictions, supporting more complex marine applications and covering more expansive geographic areas.

Predictable Condition Analysis and Prediction Method of SBAS-InSAR Coal Mining Subsidence

The forward prediction of mining subsidence in coal mining areas is key to evaluating mining risk and improving mine management plans. At present, the conditions that ensure successful application of the time function method to predict future subsidence in coal mining areas remain poorly known, and a prediction method suitable for large-scale subsidence prediction during coal mining has not been established. Based on the characteristics of logistic model and simulation experiments, we determined its predictable conditions for coal mining subsidence prediction. We propose a small baseline subset-synthetic aperture radar interferometric (SBAS-InSAR) coal mining subsidence prediction method with a predictable dynamic range based on the predictable condition obtained. The method uses time-series subsidence data obtained by SBAS-InSAR as the fitting data. The logistic model parameters are obtained pixel-by-pixel via the Levenberg–Marquardt (LM) algorithm. The predictable range is subsequently determined based on the predictable condition. Finally, future subsidence in the predictable range is predicted. The methodology was tested in two coal mining areas in Inner Mongolia–one a single working face mine and one a parallel double working face mine. The predicted results agree well with the InSAR monitoring results. The average root mean square error (RMSE) of the predicted results was 0.0119 m. In addition, we used the Knothe model to conduct comparative experiments without considering predictable conditions. The results reflect the advantages of our proposed method and the necessity of predictable conditions. The new prediction method is beneficial for risk assessment and coal mining planning.

Deep neural learning based protein function prediction.

It is vital for the annotation of uncharacterized proteins by protein function prediction. At present, Deep Neural Network based protein function prediction is mainly carried out for dataset of small scale proteins or Gene Ontology, and usually explore the relationships between single protein feature and function tags. The practical methods for large-scale multi-features protein prediction still need to be studied in depth. This paper proposes a DNN based protein function prediction approach IGP-DNN. This method uses Grasshopper Optimization Algorithm (GOA) and Intuitionistic Fuzzy c-Means clustering (IFCM) based protein function modules extracting algorithm to extract the features of protein modules, utilizing Kernel Principal Component Analysis (KPCA) method to reduce the dimensionality of the protein attribute information, and integrating module features and attribute features. Inputting integrated data into DNN through multiple hidden layers to classify proteins and predict protein functions. In the experiments, the F-measure value of IGP-DNN on the DIP dataset reaches 0.4436, which shows better performance.

A Knowledge Graph-Gcn-Community Detection Integrated Model for Large-Scale Stock Price Prediction

A Knowledge Graph-Gcn-Community Detection Integrated Model for Large-Scale Stock Price Prediction

On the Use of Deep Neural Networks for Large-Scale Spatial Prediction

For spatial kriging (prediction), the Gaussian process (GP) has been the go-to tool of spatial statisticians for decades. However, the GP is plagued by computational intractability, rendering it infeasible for use on large spatial data sets. Neural networks (NNs), on the other hand, have arisen as a flexible and computationally feasible approach for capturing nonlinear relationships. To date, however, NNs have only been scarcely used for problems in spatial statistics but their use is beginning to take root. In this work, we argue for equivalence between a NN and a GP and demonstrate how to implement NNs for kriging from large spatial data. We compare the computational efficacy and predictive power of NNs with that of GP approximations across a variety of big spatial Gaussian, non-Gaussian and binary data applications of up to size $n={10^{6}}$. Our results suggest that fully-connected NNs perform similarly to state-of-the-art, GP-approximated models for short-range predictions but can suffer for longer range predictions.

Large-Scale Prediction of Stream Water Quality Using an Interpretable Deep Learning Approach

Large-Scale Prediction of Stream Water Quality Using an Interpretable Deep Learning Approach

The role of the hippocampus in statistical learning and language recovery in persons with post stroke aphasia

Although several studies have aimed for accurate predictions of language recovery in post stroke aphasia, individual language outcomes remain hard to predict. Large-scale prediction models are built using data from patients mainly in the chronic phase after stroke, although it is clinically more relevant to consider data from the acute phase. Previous research has mainly focused on deficits, i.e., behavioral deficits or specific brain damage, rather than compensatory mechanisms, i.e., intact cognitive skills or undamaged brain regions. One such unexplored brain region that might support language (re)learning in aphasia is the hippocampus, a region that has commonly been associated with an individual’s learning potential, including statistical learning. This refers to a set of mechanisms upon which we rely heavily in daily life to learn a range of regularities across cognitive domains. Against this background, thirty-three patients with aphasia (22 males and 11 females, M = 69.76 years, SD = 10.57 years) were followed for 1 year in the acute (1–2 weeks), subacute (3–6 months) and chronic phase (9–12 months) post stroke. We evaluated the unique predictive value of early structural hippocampal measures for short-term and long-term language outcomes (measured by the ANELT). In addition, we investigated whether statistical learning abilities were intact in patients with aphasia using three different tasks: an auditory-linguistic and visual task based on the computation of transitional probabilities and a visuomotor serial reaction time task. Finally, we examined the association of individuals’ statistical learning potential with acute measures of hippocampal gray and white matter. Using Bayesian statistics, we found moderate evidence for the contribution of left hippocampal gray matter in the acute phase to the prediction of long-term language outcomes, over and above information on the lesion and the initial language deficit (measured by the ScreeLing). Non-linguistic statistical learning in patients with aphasia, measured in the subacute phase, was intact at the group level compared to 23 healthy older controls (8 males and 15 females, M = 74.09 years, SD = 6.76 years). Visuomotor statistical learning correlated with acute hippocampal gray and white matter. These findings reveal that particularly left hippocampal gray matter in the acute phase is a potential marker of language recovery after stroke, possibly through its statistical learning ability.