Developed Machine Learning Methods Research Articles

SATS: simplification aware text summarization of scientific documents.

Simplifying summaries of scholarly publications has been a popular method for conveying scientific discoveries to a broader audience. While text summarization aims to shorten long documents, simplification seeks to reduce the complexity of a document. To accomplish these tasks collectively, there is a need to develop machine learning methods to shorten and simplify longer texts. This study presents a new Simplification Aware Text Summarization model (SATS) based on future n-gram prediction. The proposed SATS model extends ProphetNet, a text summarization model, by enhancing the objective function using a word frequency lexicon for simplification tasks. We have evaluated the performance of SATS on a recently published text summarization and simplification corpus consisting of 5,400 scientific article pairs. Our results in terms of automatic evaluation demonstrate that SATS outperforms state-of-the-art models for simplification, summarization, and joint simplification-summarization across two datasets on ROUGE, SARI, and CSS1 . We also provide human evaluation of summaries generated by the SATS model. We evaluated 100 summaries from eight annotators for grammar, coherence, consistency, fluency, and simplicity. The average human judgment for all evaluated dimensions lies between 4.0 and 4.5 on a scale from 1 to 5 where 1 means low and 5 means high.

Learning unbounded-domain spatiotemporal differential equations using adaptive spectral methods

AbstractRapidly developing machine learning methods have stimulated research interest in computationally reconstructing differential equations (DEs) from observational data, providing insight into the underlying mechanistic models. In this paper, we propose a new neural-ODE-based method that spectrally expands the spatial dependence of solutions to learn the spatiotemporal DEs they obey. Our spectral spatiotemporal DE learning method has the advantage of not explicitly relying on spatial discretization (e.g., meshes or grids), thus allowing reconstruction of DEs that may be defined on unbounded spatial domains and that may contain long-ranged, nonlocal spatial interactions. By combining spectral methods with the neural ODE framework, our proposed spectral DE method addresses the inverse-type problem of reconstructing spatiotemporal equations in unbounded domains. Even for bounded domain problems, our spectral approach is as accurate as some of the latest machine learning approaches for learning or numerically solving partial differential equations (PDEs). By developing a spectral framework for reconstructing both PDEs and partial integro-differential equations (PIDEs), we extend dynamical reconstruction approaches to a wider range of problems, including those in unbounded domains.

Image Filtering to Improve Maize Tassel Detection Accuracy Using Machine Learning Algorithms.

Unmanned aerial vehicle (UAV)-based imagery has become widely used to collect time-series agronomic data, which are then incorporated into plant breeding programs to enhance crop improvements. To make efficient analysis possible, in this study, by leveraging an aerial photography dataset for a field trial of 233 different inbred lines from the maize diversity panel, we developed machine learning methods for obtaining automated tassel counts at the plot level. We employed both an object-based counting-by-detection (CBD) approach and a density-based counting-by-regression (CBR) approach. Using an image segmentation method that removes most of the pixels not associated with the plant tassels, the results showed a dramatic improvement in the accuracy of object-based (CBD) detection, with the cross-validation prediction accuracy (r2) peaking at 0.7033 on a detector trained with images with a filter threshold of 90. The CBR approach showed the greatest accuracy when using unfiltered images, with a mean absolute error (MAE) of 7.99. However, when using bootstrapping, images filtered at a threshold of 90 showed a slightly better MAE (8.65) than the unfiltered images (8.90). These methods will allow for accurate estimates of flowering-related traits and help to make breeding decisions for crop improvement.

Linking Entities across Relations and Graphs

This article proposes a notion of parametric simulation to link entities across a relational database 𝒟 and a graph G . Taking functions and thresholds for measuring vertex closeness, path associations, and important properties as parameters, parametric simulation identifies tuples t in 𝒟 and vertices v in G that refer to the same real-world entity, based on both topological and semantic matching. We develop machine learning methods to learn the parameter functions and thresholds. We show that parametric simulation is in quadratic-time by providing such an algorithm. Moreover, we develop an incremental algorithm for parametric simulation; we show that the incremental algorithm is bounded relative to its batch counterpart, i.e., it incurs the minimum cost for incrementalizing the batch algorithm. Putting these together, we develop HER , a parallel system to check whether ( t, v ) makes a match, find all vertex matches of t in G , and compute all matches across 𝒟 and G , all in quadratic-time; moreover, HER supports incremental computation of these in response to updates to 𝒟 and G . Using real-life and synthetic data, we empirically verify that HER is accurate with F-measure of 0.94 on average, and is able to scale with database 𝒟 and graph G for both batch and incremental computations.

Comparison of multiple linear regression and machine learning methods in predicting cognitive function in older Chinese type 2 diabetes patients.

The prevalence of type 2 diabetes (T2D) has increased dramatically in recent decades, and there are increasing indications that dementia is related to T2D. Previous attempts to analyze such relationships principally relied on traditional multiple linear regression (MLR). However, recently developed machine learning methods (Mach-L) outperform MLR in capturing non-linear relationships. The present study applied four different Mach-L methods to analyze the relationships between risk factors and cognitive function in older T2D patients, seeking to compare the accuracy between MLR and Mach-L in predicting cognitive function and to rank the importance of risks factors for impaired cognitive function in T2D. We recruited older T2D between 60-95years old without other major comorbidities. Demographic factors and biochemistry data were used as independent variables and cognitive function assessment (CFA) was conducted using the Montreal Cognitive Assessment as an independent variable. In addition to traditional MLR, we applied random forest (RF), stochastic gradient boosting (SGB), Naïve Byer's classifier (NB) and eXtreme gradient boosting (XGBoost). Totally, the test cohort consisted of 197 T2D (98 men and 99 women). Results showed that all ML methods outperformed MLR, with symmetric mean absolute percentage errors for MLR, RF, SGB, NB and XGBoost respectively of 0.61, 0.599, 0.606, 0.599 and 0.2139. Education level, age, frailty score, fasting plasma glucose and body mass index were identified as key factors in descending order of importance. In conclusion, our study demonstrated that RF, SGB, NB and XGBoost are more accurate than MLR for predicting CFA score, and identify education level, age, frailty score, fasting plasma glucose, body fat and body mass index as important risk factors in an older Chinese T2D cohort.

How accurately can one predict drug binding modes using AlphaFold models?

Computational prediction of protein structure has been pursued intensely for decades, motivated largely by the goal of using structural models for drug discovery. Recently developed machine-learning methods such as AlphaFold 2 (AF2) have dramatically improved protein structure prediction, with reported accuracy approaching that of experimentally determined structures. To what extent do these advances translate to an ability to predict more accurately how drugs and drug candidates bind to their target proteins? Here, we carefully examine the utility of AF2 protein structure models for predicting binding poses of drug-like molecules at the largest class of drug targets, the G-protein-coupled receptors. We find that AF2 models capture binding pocket structures much more accurately than traditional homology models, with errors nearly as small as differences between structures of the same protein determined experimentally with different ligands bound. Strikingly, however, the accuracy of ligand-binding poses predicted by computational docking to AF2 models is not significantly higher than when docking to traditional homology models and is much lower than when docking to structures determined experimentally without these ligands bound. These results have important implications for all those who might use predicted protein structures for drug discovery.

Grow the pie, or have it? Using machine learning to impact heterogeneity in the Ultra-poor graduation model

ABSTRACT The ‘Ultra-poor Graduation’ model, though highly effective in poverty alleviation, costs substantially more than alternative poverty alleviation approaches. One possible way of improving the cost-effectiveness of the model is to analyse the treatment effect heterogeneity and identify the participants who do not gain much from the programme and better customise the interventions to their needs. Applying recently developed machine learning methods on a large-scale RCT dataset from Bangladesh, we identify and characterise the program participants who benefit and who do not. We find significant variation in impact on assets where the top quintile gainers experience asset growth of 337% while asset growth is only 189% for the bottom quintile. Heterogeneity in impact on household expenditures is found to be present but of lower magnitude than that of assets. Importantly, the machine learning techniques we apply reveal contrasts in characteristics of beneficiaries who made the most in assets vs. consumption. The most benefitted households in per-capita wealth outcome were relatively older, were more dependent on wage income, had less involvement in self-employment activities, and had lower participation in household decision-making at baseline. In contrast, the top quintile gainers of household expenditure are younger, earn less from wages, depend more on self-employment income, and have higher participation in household decision-making. The results identify beneficiary characteristics that can be used in targeting households either to maximise impact on the desired dimension and/or to customise interventions for balancing the asset and consumption trade-off.

Determinants and performance prediction on photocatalytic properties of hydroxyapatite by machine learning

Hydroxyapatite (HAP) have been widely used as photocatalysts in environmental remediation. However, the effects of crystallinity, crystal size, and purity of HAP on their photocatalytic performances have not been fully understood yet. In this paper, we synthesized HAP samples with different properties by controlling the calcination conditions and evaluated their photocatalytic performance in the degradation reaction of methyl orange. Combing the catalytic experiments with machine learning technique, we found that the crystallinity of HAP was the dominant parameter that determined the photocatalytic performance. And the developed machine learning method could predict the catalytic results with an error of only 2.2 %. Our work provides a route to predict material properties based its characters by experiment combined with machine learning.

Non-Invasive Blood Pressure Sensing via Machine Learning.

In this paper, a machine learning (ML) approach to estimate blood pressure (BP) using photoplethysmography (PPG) is presented. The final aim of this paper was to develop ML methods for estimating blood pressure (BP) in a non-invasive way that is suitable in a telemedicine health-care monitoring context. The training of regression models useful for estimating systolic blood pressure (SBP) and diastolic blood pressure (DBP) was conducted using new extracted features from PPG signals processed using the Maximal Overlap Discrete Wavelet Transform (MODWT). As a matter of fact, the interest was on the use of the most significant features obtained by the Minimum Redundancy Maximum Relevance (MRMR) selection algorithm to train eXtreme Gradient Boost (XGBoost) and Neural Network (NN) models. This aim was satisfactorily achieved by also comparing it with works in the literature; in fact, it was found that XGBoost models are more accurate than NN models in both systolic and diastolic blood pressure measurements, obtaining a Root Mean Square Error (RMSE) for SBP and DBP, respectively, of 5.67 mmHg and 3.95 mmHg. For SBP measurement, this result is an improvement compared to that reported in the literature. Furthermore, the trained XGBoost regression model fulfills the requirements of the Association for the Advancement of Medical Instrumentation (AAMI) as well as grade A of the British Hypertension Society (BHS) standard.

Machine learning methods for particle stress development in suspension Poiseuille flows

Machine learning methods for particle stress development in suspension Poiseuille flows

Exploring YouTube's Recommendation System in the Context of COVID-19 Vaccines: Computational and Comparative Analysis of Video Trajectories.

Throughout the COVID-19 pandemic, there has been a concern that social media may contribute to vaccine hesitancy due to the wide availability of antivaccine content on social media platforms. YouTube has stated its commitment to removing content that contains misinformation on vaccination. Nevertheless, such claims are difficult to audit. There is a need for more empirical research to evaluate the actual prevalence of antivaccine sentiment on the internet. This study examines recommendations made by YouTube's algorithms in order to investigate whether the platform may facilitate the spread of antivaccine sentiment on the internet. We assess the prevalence of antivaccine sentiment in recommended videos and evaluate how real-world users' experiences are different from the personalized recommendations obtained by using synthetic data collection methods, which are often used to study YouTube's recommendation systems. We trace trajectories from a credible seed video posted by the World Health Organization to antivaccine videos, following only video links suggested by YouTube's recommendation system. First, we gamify the process by asking real-world participants to intentionally find an antivaccine video with as few clicks as possible. Having collected crowdsourced trajectory data from respondents from (1) the World Health Organization and United Nations system (nWHO/UN=33) and (2) Amazon Mechanical Turk (nAMT=80), we next compare the recommendations seen by these users to recommended videos that are obtained from (3) the YouTube application programming interface's RelatedToVideoID parameter (nRTV=40) and (4) from clean browsers without any identifying cookies (nCB=40), which serve as reference points. We develop machine learning methods to classify antivaccine content at scale, enabling us to automatically evaluate 27,074 video recommendations made by YouTube. We found no evidence that YouTube promotes antivaccine content; the average share of antivaccine videos remained well below 6% at all steps in users' recommendation trajectories. However, the watch histories of users significantly affect video recommendations, suggesting that data from the application programming interface or from a clean browser do not offer an accurate picture of the recommendations that real users are seeing. Real users saw slightly more provaccine content as they advanced through their recommendation trajectories, whereas synthetic users were drawn toward irrelevant recommendations as they advanced. Rather than antivaccine content, videos recommended by YouTube are likely to contain health-related content that is not specifically related to vaccination. These videos are usually longer and contain more popular content. Our findings suggest that the common perception that YouTube's recommendation system acts as a "rabbit hole" may be inaccurate and that YouTube may instead be following a "blockbuster" strategy that attempts to engage users by promoting other content that has been reliably successful across the platform.

Machine learning application in modelling marine and coastal phenomena: a critical review

This study provides an extensive review of over 200 journal papers focusing on Machine Learning (ML) algorithms’ use for promoting a sustainable management of the marine and coastal environments. The research covers various facets of ML algorithms, including data preprocessing and handling, modeling algorithms for distinct phenomena, model evaluation, and use of dynamic and integrated models. Given that machine learning modeling relies on experience or trial-and-error, examining previous applications in marine and coastal modeling is proven to be beneficial. The performance of different ML methods used to predict wave heights was analyzed to ascertain which method was superior with various datasets. The analysis of these papers revealed that properly developed ML methods could successfully be applied to multiple aspects. Areas of application include data collection and analysis, pollutant and sediment transport, image processing and deep learning, and identification of potential regions for aquaculture and wave energy activities. Additionally, ML methods aid in structural design and optimization and in the prediction and classification of oceanographic parameters. However, despite their potential advantages, dynamic and integrated ML models remain underutilized in marine projects. This research provides insights into ML’s application and invites future investigations to exploit ML’s untapped potential in marine and coastal sustainability.

OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over.

In the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods for solving this problem, we created the OHO (Object Hand-Over) dataset of tools and other everyday objects being held by human hands. Our dataset consists of color, depth, and thermal images with the addition of pose and shape information about the objects in a real-world scenario. Although the focus of this paper is on instance segmentation, our dataset also enables training for different tasks such as 3D pose estimation or shape estimation of objects. For the instance segmentation task, we present a pipeline for automated label generation in point clouds, as well as image data. Through baseline experiments, we show that these labels are suitable for training an instance segmentation to distinguish hands from objects on a per-pixel basis. Moreover, we present qualitative results for applying our trained model in a real-world application.

Semantic determinants of memorability

We examine why some words are more memorable than others by using predictive machine learning models applied to word recognition and recall datasets. Our approach provides more accurate out-of-sample predictions for recognition and recall than previous psychological models, and outperforms human participants in new studies of memorability prediction. Our approach’s predictive power stems from its ability to capture the semantic determinants of memorability in a data-driven manner. We identify which semantic categories are important for memorability and show that, unlike features such as word frequency that influence recognition and recall differently, the memorability of semantic categories is consistent across recognition and recall. Our paper sheds light on the complex psychological drivers of memorability, and in doing so illustrates the power of machine learning methods for psychological theory development.

Anomaly Based Intrusion Detection through Efficient Machine Learning Model

Machine learning is commonly utilised to construct an intrusion detection system (IDS) that automatically detects and classifies network intrusions and host-level threats. Malicious assaults change and occur in high numbers, needing a scalable solution. Cyber security researchers may use public malware databases for research and related work. No research has examined machine learning algorithm performance on publicly accessible datasets. Data and physical level security and analysis for Data protection have become more important as data volumes grow. IDSs collect and analyse data to identify system or network intrusions for data prevention. The amount, diversity, and speed of network data make data analysis to identify assaults challenging. IDS uses machine learning methods for precise and efficient development of data security mechanism. This work presented intrusion detection model using machine learning, which utilised feature extraction, feature selection and feature modelling for intrusion detection classifier.

Exploring treatment effect heterogeneity of a PROMs alert intervention in knee and hip arthroplasty patients: A causal forest application

Patient reported outcome measures (PROMs) experience an uptake in use for hip (HA) and knee arthroplasty (KA) patients. As they may be used for patient monitoring interventions, it remains unclear whether their use in HA/KA patients is effective, and which patient groups benefit the most. Nonetheless, knowledge about treatment effect heterogeneity is crucial for decision makers to target interventions towards specific subgroups that benefit to a greater extend. Therefore, we evaluate the treatment effect heterogeneity of a remote PROM monitoring intervention that includes ∼8000 HA/KA patients from a randomized controlled trial conducted in nine German hospitals. The study setting gave us the unique opportunity to apply a causal forest, a recently developed machine learning method, to explore treatment effect heterogeneity of the intervention. We found that among both HA and KA patients, the intervention was especially effective for patients that were female, >65 years of age, had a blood pressure disease, were not working, reported no backpain and were adherent. When transferring the study design into standard care, policy makers should make use of the knowledge obtained in this study and allocate the treatment towards subgroups for which the treatment is especially effective.

Harnessing data science to improve molecular structure elucidation from tandem mass spectrometry

Compound structural identification for non-targeted screening of organic molecules in complex mixtures is commonly carried out using liquid chromatography coupled to tandem mass spectrometry (UHPLC-HRMS/MS and related techniques). Instrumental developments in recent years have increased the quality and quantity of data available; however, using current data analysis methods, structures can be assigned to only a small fraction of compounds present in typical mixtures. We present a new data analysis pipeline, “MSEI”, that harnesses data science methodologies to improve structural identification capabilities from tandem mass spectrometry data. In particular, feature vectors for fingerprint calculation are found directly from tandem mass spectra, strongly reducing computational costs, and fingerprint comparison uses an optimised methodology accounting for uncertainty to improve distinction between matching and non-matching compounds. MSEI builds on the identification of a small number of compounds through current state-of-the-art data analysis on UHPLC-HRMS/MS measurements and uses targeted training and tailored molecular fingerprints to focus identification to a particular molecular space of interest. Initial compound identifications are used as training data for a set of random forests which directly predict a custom 75-digit molecular fingerprint from a vectorised MS/MS spectrum. Kendrick mass defects (KMDs) for peaks as well as “lost” fragments removed during fragmentation were found to be useful information for fingerprint prediction. Fingerprints are then compared to potential matches from the PubChem structural database using Euclidean distance, with fingerprint digit weights determined using an SVM to maximise distance between matching and non-matching compounds. Potential matches are additionally filtered for hydrophobicity based on measured retention time, using a newly developed machine learning method for retention time prediction. MSEI was able to correctly assign > 50% of structures in a test dataset and showed > 10% better performance than current state-of-the-art methods, while using an order of magnitude less computational power and a fraction of the training data.

Predicting multiaxial fatigue life of FGH96 superalloy based on machine learning models by considering failure process and loading paths

Wide applications of powder metallurgy superalloys in mechanical engineering imply the great significance of studying their mechanical properties. Fatigue property of superalloys is complicated because of many influencing factors with variability. This means the difficulty to correctly predict a real-valued fatigue life due to the propagation of error. Specially, extreme dispersion of multiaxial fatigue life is an important challenge of fatigue life prediction for superalloys. In this paper, we focus on the difficulty and dispersion by developing machine learning (ML) models for predicting interval-valued fatigue life of FGH96 superalloy under multiaxial loadings. Some data extracted from fatigue failure process of FGH96 superalloy is used to develop ML methods such as back-propagation neural network (BP), support vector regression (SVR) and random forest (RF). Then considering the randomness in the data originating from multiaxial loading paths, measures of sample geometry and mechanical loading, testing system error and others, the low cycle fatigue life of FGH96 superalloy is predicted as an interval with a probability distribution. The obtained results are analyzed and compared with the fatigue experimental observations of FGH96 superalloy under six loading paths. It is found that the multiaxial fatigue failure behavior of FGH96 superalloy can be effectively described, and the developed ML models exhibit some advantages to tackle the difficulty and dispersion in predicting multiaxial fatigue life of alloys.

Machine Learning for Ionization Potentials and Photoionization Cross Sections of Volatile Organic Compounds.

Molecular ionization potentials (IP) and photoionization cross sections (σ) can affect the sensitivity of photoionization detectors (PIDs) and other sensors for gaseous species. This study employs several methods of machine learning (ML) to predict IP and σ values at 10.6 eV (117 nm) for a dataset of 1251 gaseous organic species. The explicitness of the treatment of the species electronic structure progressively increases among the methods. The study compares the ML predictions of the IP and σ values to those obtained by quantum chemical calculations. The ML predictions are comparable in performance to those of the quantum calculations when evaluated against measurements. Pretraining further reduces the mean absolute errors (ε) compared to the measurements. The graph-based attentive fingerprint model was most accurate, for which εIP = 0.23 ± 0.01 eV and εσ = 2.8 ± 0.2 Mb compared to measurements and computed cross sections, respectively. The ML predictions for IP correlate well with both the measured IPs (R 2 = 0.88) and with IPs computed at the level of M06-2X/aug-cc-pVTZ (R 2 = 0.82). The ML predictions for σ correlated reasonably well with computed cross sections (R 2 = 0.66). The developed ML methods for IP and σ values, representing the properties of a generalizable set of volatile organic compounds (VOCs) relevant to industrial applications and atmospheric chemistry, can be used to quantitatively describe the species-dependent sensitivity of chemical sensors that use ionizing radiation as part of the sensing mechanism, such as photoionization detectors.

Sparse sensor reconstruction of vortex-impinged airfoil wake with machine learning

Reconstruction of unsteady vortical flow fields from limited sensor measurements is challenging. We develop machine learning methods to reconstruct flow features from sparse sensor measurements during transient vortex–airfoil wake interaction using only a limited amount of training data. The present machine learning models accurately reconstruct the aerodynamic force coefficients, pressure distributions over airfoil surface, and two-dimensional vorticity field for a variety of untrained cases. Multi-layer perceptron is used for estimating aerodynamic forces and pressure profiles over the surface, establishing a nonlinear model between the pressure sensor measurements and the output variables. A combination of multi-layer perceptron with convolutional neural network is utilized to reconstruct the vortical wake. Furthermore, the use of transfer learning and long short-term memory algorithm combined in the training models greatly improves the reconstruction of transient wakes by embedding the dynamics. The present machine-learning methods are able to estimate the transient flow features while exhibiting robustness against noisy sensor measurements. Finally, appropriate sensor locations over different time periods are assessed for accurately estimating the wakes. The present study offers insights into the dynamics of vortex–airfoil interaction and the development of data-driven flow estimation.Graphic abstract