Supervised Machine Learning Framework Research Articles

Natural Language Understanding for Navigation of Service Robots in Low-Resource Domains and Languages: Scenarios in Spanish and Nahuatl

Human–robot interaction is becoming increasingly common to perform useful tasks in everyday life. From the human–machine communication perspective, achieving effective interaction in natural language is one challenge. To address it, natural language processing strategies have recently been used, commonly following a supervised machine learning framework. In this context, most approaches rely on the use of linguistic resources (e.g., taggers or embeddings), including training corpora. Unfortunately, such resources are scarce for some languages in specific domains, increasing the complexity of solution approaches. Motivated by these challenges, this paper explores deep learning methods for understanding natural language commands emitted to service robots that guide their movements in low-resource scenarios, defined by the use of Spanish and Nahuatl languages, for which linguistic resources are scarcely unavailable for this specific task. Particularly, we applied natural language understanding (NLU) techniques using deep neural networks and transformers-based models. As part of the research methodology, we introduced a labeled dataset of movement commands in the mentioned languages. The results show that models based on transformers work well to recognize commands (intent classification task) and their parameters (e.g., quantities and movement units) in Spanish, achieving a performance of 98.70% (accuracy) and 96.96% (F1) for the intent classification and slot-filling tasks, respectively). In Nahuatl, the best performance obtained was 93.5% (accuracy) and 88.57% (F1) in these tasks, respectively. In general, this study shows that robot movements can be guided in natural language through machine learning models using neural models and cross-lingual transfer strategies, even in low-resource scenarios.

FEM and machine learning aided modelling of compressive deformation anisotropy in pressure-less sintered Cu-15Ni-8Sn alloy

Pressure-less sintering integrated forging is an effective method for densification enhancement of powder metallurgy (PM) processed Cu-15Ni-8Sn alloy. Nevertheless, compressive deformation anisotropy associated with uniaxial forging creates heterogeneous deformation zones containing residual porosity, resulting in strength degradation of the PM alloy. This work introduces a novel approach for mitigating the anisotropy arising from forging by characterising the compressive deformation heterogeneity using a synergy of finite element method (FEM) and machine learning (ML) approaches. Results obtained from Gurson model-based FEM simulation of uniaxial forging process indicate that decreasing the aspect ratio (AR) of sintered preform leads to reduction in plastic deformation heterogeneity. Additionally, the semi supervised ML framework developed here unravels three distinct strain hardening regimes and predicts an upward paradigm shift in the quantitative distribution of instantaneous strain hardening exponent with the decrease in AR. These computational predictions align with the experimental forging results which demonstrate substantial enhancement in strain hardenability, formability and densification of lower AR preform due to more uniform deformation. Uniaxially forged compact depicted a notable increase of 14 % in relative density along with significant improvement in hardness by 66 % as compared to the sintered compact, which can be attributed to forging induced pore elimination at microstructural level. The promising results obtained in this study have practical implications in the development of sustainable green manufacturing technology to produce high strength porosity tailored Cu-15Ni-8Sn alloy.

ScCancer2: data-driven in-depth annotations of the tumor microenvironment at single-level resolution.

Single-cell RNA-seq (scRNA-seq) is a powerful technique for decoding the complex cellular compositions in the tumor microenvironment (TME). As previous studies have defined many meaningful cell subtypes in several tumor types, there is a great need to computationally transfer these labels to new datasets. Also, different studies used different approaches or criteria to define the cell subtypes for the same major cell lineages. The relationships between the cell subtypes defined in different studies should be carefully evaluated. In this updated package scCancer2, designed for integrative tumor scRNA-seq data analysis, we developed a supervised machine learning framework to annotate TME cells with annotated cell subtypes from 15 scRNA-seq datasets with 594 samples in total. Based on the trained classifiers, we quantitatively constructed the similarity maps between the cell subtypes defined in different references by testing on all the 15 datasets. Secondly, to improve the identification of malignant cells, we designed a classifier by integrating large-scale pan-cancer TCGA bulk gene expression datasets and scRNA-seq datasets (10 cancer types, 175 samples, 663857 cells). This classifier shows robust performances when no internal confidential reference cells are available. Thirdly, scCancer2 integrated a module to process the spatial transcriptomic data and analyze the spatial features of TME. The package and user documentation are available at http://lifeome.net/software/sccancer2/ and https://doi.org/10.5281/zenodo.10477296.

B2B Customer Engagement Customer Behaviour Forecast Application

The paper presents the "Casper Customer Behavior Forecast Application" a creative project focused on the creation of an advanced decision support software system. The purpose of this system is to closely observe, fully analyze, and precisely predict the behaviors of Casper Business-to-Business (B2B) clients. Our research study implements an innovative methodology by focusing on the enhancement of client interactions through the utilization of RESTful services. Our objective is to reimagine the future of customer relationship management by analyzing and forecasting client behavior in the business-to-business (B2B) setting. The project involves a comprehensive methodology that incorporates extensive study, complex software design, and careful data analysis. In more detail, it involves extensive monitoring of consecutive RESTful interactions carried out by business-to-business consumers across a period of time. By leveraging the capabilities of modern machine learning and deep learning algorithms, our objective is to develop prediction models that establish novel benchmarks within the industry. The methodology employed in this study includes the development of labeled behavioral datasets and the utilization of a supervised machine learning framework. The evaluation of model performance will be performed systematically using a range of metrics, such as F-Score and Accuracy, in order to establish the model's robustness and reliability in making accurate predictions. The expected outcomes of this project have the potential to bring about significant changes. Primarily, the integration of machine learning and deep learning algorithms will provide our company with an important amount of knowledge. Our organization will obtain an innovative software solution that possesses the capability to precisely forecast the future actions of B2B customers. This results in enhanced inventory management and a significant reduction in client waiting periods, resulting in increased levels of customer satisfaction. Additionally, this study has the potential to make significant additions to the global academic body of knowledge in the domains of machine learning and deep learning. In simple terms, the project titled "Casper Customer Behavior Forecast Application" embodies an innovative and academic effort aimed at enhancing customer engagement and predicting customer behavior within the context of business-to-business (B2B) interactions. This research project carries considerable importance, both in its potential to reinvent the field of customer relationship management and its ability to contribute to the global academic conversation on machine learning and deep learning.

A Deep Learning Filter for the Intraseasonal Variability of the Tropics

Abstract This paper presents a novel application of convolutional neural network (CNN) models for filtering the intraseasonal variability of the tropical atmosphere. In this deep learning filter, two convolutional layers are applied sequentially in a supervised machine learning framework to extract the intraseasonal signal from the total daily anomalies. The CNN-based filter can be tailored for each field similarly to fast Fourier transform filtering methods. When applied to two different fields (zonal wind stress and outgoing longwave radiation), the index of agreement between the filtered signal obtained using the CNN-based filter and a conventional weight-based filter is between 95% and 99%. The advantage of the CNN-based filter over the conventional filters is its applicability to time series with the length comparable to the period of the signal being extracted. Significance Statement This study proposes a new method for discovering hidden connections in data representative of tropical atmosphere variability. The method makes use of an artificial intelligence (AI) algorithm that combines a mathematical operation known as convolution with a mathematical model built to reflect the behavior of the human brain known as artificial neural network. Our results show that the filtered data produced by the AI-based method are consistent with the results obtained using conventional mathematical algorithms. The advantage of the AI-based method is that it can be applied to cases for which the conventional methods have limitations, such as forecast (hindcast) data or real-time monitoring of tropical variability in the 20–100-day range.

Convolutional neural network framework for the automated analysis of transition metal X-ray photoelectron spectra

X-ray photoelectron spectroscopy is an indispensable technique for the quantitative determination of sample composition and electronic structure in diverse research fields. Quantitative analysis of the phases present in XP spectra is usually conducted manually by means of empirical peak fitting performed by trained spectroscopists. However, with recent advancements in the usability and reliability of XPS instruments, ever more (inexperienced) users are creating increasingly large data sets that are harder to analyze by hand. In order to aid users with the analysis of large XPS data sets, more automated, easy-to-use analysis techniques are needed. Here, we propose a supervised machine learning framework based on artificial convolutional neural networks. By training such networks on large numbers of artificially created XP spectra with known quantifications (i.e., for each spectrum, the concentration of each chemical species is known), we created universally applicable models for auto-quantification of transition-metal XPS data that are able to predict the sample composition from spectra within seconds. Upon evaluation against more traditional peak fitting methods, we showed that these neural networks achieve competitive quantification accuracy. The proposed framework is shown to be flexible enough to accommodate spectra containing multiple chemical elements and measured with different experimental parameters. The use of dropout variational inference for the determination of quantification uncertainty is illustrated.

Developing a variable charge potential for Hf/Nb/Ta/Ti/Zr/O system via machine learning global optimization

Despite widespread attentions on the HfNbTaTiZr refractory high-entropy alloy (RHEA) owing to various exceptional properties, insights into nanostructure-property relations are severely hindered by the lack of reliable interatomic potentials in atomistic simulations. In this paper, we propose a supervised machine learning framework powered by the distributed breeder genetic algorithm (DBGA) to train a charge transfer ionic potential (CTIP) for the multi-component Hf/Nb/Ta/Ti/Zr/O system. By combining the DBGA with extensive ab initio training data and a carefully designed objective function, a robust and extrapolative potential with massive internal parameters is effectively optimized. A cross-validation of the CTIP is rigorously performed to verify its accuracy in predicting energetic, structural and mechanical properties of metals, intermetallic compounds, alloys and oxides. Using the developed potential, we further study the atomic structure and its evolution in RHEA to reveal the deformation and oxidation mechanisms in response to mechanical loadings and oxygen atmospheres. The simulation results not only provide valuable atomistic information hardly accessible through experiments and thermodynamic calculations, but also display promising prospects for large-scale reactive simulations of the Hf/Nb/Ta/Ti/Zr/O complex-component system.

Tree Segmentation and Parameter Measurement from Point Clouds Using Deep and Handcrafted Features

Accurate measurement of the geometric parameters of trees is a vital part of forest inventory in forestry management. Aerial and terrestrial Light Detection and Ranging (LiDAR) sensors are currently used in forest inventory as an effective and efficient means of forest data collection. Many recent approaches to processing and interpreting this data make use of supervised machine learning algorithms such as Deep Neural Networks (DNNs) due to their advantages in accuracy, robustness and the ability to adapt to new data and environments. In this paper, we develop new approaches to deep-learning-based forest point cloud analysis that address key issues in real applications in forests. Firstly, we develop a point cloud segmentation framework that identifies tree stem points in individual trees and is designed to improve performance when labelled training data are limited. To improve point cloud representation learning, we propose a handcrafted point cloud feature for semantic segmentation which plays a complementary role with DNNs in semantics extraction. Our handcrafted feature can be integrated with DNNs to improve segmentation performance. Additionally, we combine this feature with a semi-supervised and cross-dataset training process to effectively leverage unlabelled point cloud data during training. Secondly, we develop a supervised machine learning framework based on Recurrent Neural Networks (RNNs) that directly estimates the geometric parameters of individual tree stems (via a stacked cylinder model) from point clouds in a data-driven process, without the need for a separate procedure for model-fitting on points. The use of a one-stage deep learning algorithm for this task makes the process easily adaptable to new environments and datasets. To evaluate our methods for both the segmentation and parameter estimation tasks, we use four real-world datasets of different tree species collected using aerial and terrestrial LiDAR. For the segmentation task, we extensively evaluate our method on the three different settings of supervised, semi-supervised, and cross-dataset learning, and the experimental results indicate that both our handcrafted point cloud feature and our semi-supervised and cross-dataset learning framework can significantly improve tree segmentation performance under all three settings. For the tree parameter estimation task, our DNN-based method performs comparably to well-established traditional methods and opens up new avenues for DNN-based tree parameter estimation.

A Transferable Machine Learning Framework for Predicting Transcriptional Responses of Genes Across Species.

Leveraging existing resources in studied species to predict gene functions has the potential to rapidly expand understanding of annotated genes in other, less well-studied, species with assembled genomes. However, orthology is not a reliable predictor for the transcriptional responses of genes to stress. Machine learning methods can quantitatively estimate expression patterns and gene functions using known annotations and collections of features describing each gene. In this chapter, we describe a supervised machine learning framework to predict stress-responsive genes across species using only features derived from nucleotide sequences, using the example of cold stress-responsive genes in different Panicoid grass species.

“Let Me Get Back to You”—A Machine Learning Approach to Measuring NonAnswers

Using a supervised machine learning framework on a large training set of questions and answers, we identify 1,364 trigrams that signal nonanswers in earnings call questions and answers (Q&A). We show that this glossary has economic relevance by applying it to contemporaneous stock market reactions after earnings calls. Our findings suggest that obstructing the flow of information leads to significantly lower cumulative abnormal stock returns and higher implied volatility. As both our method and glossary are free of financial context, we believe that the measure is applicable to other fields with a Q&A setup outside the contextual domain of financial earnings conference calls. This paper was accepted by Kay Giesecke, finance. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2022.4597 .

Multiscale entropy analysis of astronomical time series

Context.The multiscale entropy assesses the complexity of a signal across different timescales. It originates from the biomedical domain and was recently successfully used to characterize light curves as part of a supervised machine learning framework to classify stellar variability.Aims.We aim to explore the behavior of the multiscale entropy in detail by studying its algorithmic properties in a stellar variability context and by linking it with traditional astronomical time series analysis methods and metrics such as the Lomb-Scargle periodogram. We subsequently use the multiscale entropy as the basis for an interpretable clustering framework that can distinguish hybrid pulsators with bothp- and g-modes from stars with onlyp-mode pulsations, such asδScuti (δSct) stars, or from stars with onlyg-mode pulsations, such asγDoradus (γDor) stars.Methods.We calculate the multiscale entropy for a set ofKeplerlight curves and simulated sine waves. We link the multiscale entropy to the type of stellar variability and to the frequency content of a signal through a correlation analysis and a set of simulations. The dimensionality of the multiscale entropy is reduced to two dimensions and is subsequently used as input to the HDBSCAN density-based clustering algorithm in order to find the hybrid pulsators within sets ofδSct andγDor stars that were observed byKepler.Results.We find that the multiscale entropy is a powerful tool for capturing variability patterns in stellar light curves. The multiscale entropy provides insights into the pulsation structure of a star and reveals how short- and long-term variability interact with each other based on time-domain information only. We also show that the multiscale entropy is correlated to the frequency content of a stellar signal and in particular to the near-core rotation rates ofg-mode pulsators. We find that our new clustering framework can successfully identify the hybrid pulsators with bothp- andg-modes in sets ofδSct andγDor stars, respectively. The benefit of our clustering framework is that it is unsupervised. It therefore does not require previously labeled data and hence is not biased by previous knowledge.

Learning finite element convergence with the Multi-fidelity Graph Neural Network

Machine learning techniques have emerged as potential alternatives to traditional physics-based modeling and partial differential equation solvers. Among these machine learning techniques, Graph Neural Networks (GNNs) simulate physics via graph models; GNNs embed relevant physical features into graph data structures, perform message passing within the graphs, and produce new attributes based on the system’s relationships. Like many machine learning frameworks, GNNs are limited by excessive data generation costs and limited generalizability outside of a narrow training domain. To address these limitations, we introduce the Multi-Fidelity Graph Neural Network (MFGNN), a supervised machine learning framework that uses low-fidelity projections to inform high-fidelity modeling across arbitrary subdomains represented by subgraphs. We implement the MFGNN for two-dimensional elastostatic problems with finite element training data. The MFGNN is trained to produce accurate analysis given low-fidelity evaluations and emulate the convergence behavior of traditional finite element analysis (FEA). Through subdomain abstraction, we also extend the MFGNN as a general model for new boundary conditions and material domains outside of the training domain.

A supervised machine learning framework with combined blocking for detecting serial crimes

A supervised machine learning framework with combined blocking for detecting serial crimes

Modeling CRISPR gene drives for suppression of invasive rodents using a supervised machine learning framework

Invasive rodent populations pose a threat to biodiversity across the globe. When confronted with these invaders, native species that evolved independently are often defenseless. CRISPR gene drive systems could provide a solution to this problem by spreading transgenes among invaders that induce population collapse, and could be deployed even where traditional control methods are impractical or prohibitively expensive. Here, we develop a high-fidelity model of an island population of invasive rodents that includes three types of suppression gene drive systems. The individual-based model is spatially explicit, allows for overlapping generations and a fluctuating population size, and includes variables for drive fitness, efficiency, resistance allele formation rate, as well as a variety of ecological parameters. The computational burden of evaluating a model with such a high number of parameters presents a substantial barrier to a comprehensive understanding of its outcome space. We therefore accompany our population model with a meta-model that utilizes supervised machine learning to approximate the outcome space of the underlying model with a high degree of accuracy. This enables us to conduct an exhaustive inquiry of the population model, including variance-based sensitivity analyses using tens of millions of evaluations. Our results suggest that sufficiently capable gene drive systems have the potential to eliminate island populations of rodents under a wide range of demographic assumptions, though only if resistance can be kept to a minimal level. This study highlights the power of supervised machine learning to identify the key parameters and processes that determine the population dynamics of a complex evolutionary system.

Vision based supervised restricted Boltzmann machine helps to actuate novel shape memory alloy accurately

Extraordinary shape recovery capabilities of shape memory alloys (SMAs) have made them a crucial building block for the development of next-generation soft robotic systems and associated cognitive robotic controllers. In this study we desired to determine whether combining video data analysis techniques with machine learning techniques could develop a computer vision based predictive system to accurately predict force generated by the movement of a SMA body that is capable of a multi-point actuation performance. We identified that rapid video capture of the bending movements of a SMA body while undergoing external electrical excitements and adapting that characterisation using computer vision approach into a machine learning model, can accurately predict the amount of actuation force generated by the body. This is a fundamental area for achieving a superior control of the actuation of SMA bodies. We demonstrate that a supervised machine learning framework trained with Restricted Boltzmann Machine (RBM) inspired features extracted from 45,000 digital thermal infrared video frames captured during excitement of various SMA shapes, is capable to estimate and predict force and stress with 93% global accuracy with very low false negatives and high level of predictive generalisation.

A novel feature engineered-CatBoost-based supervised machine learning framework for electricity theft detection

This paper presents a novel supervised machine learning-based electric theft detection approach using the feature engineered-CatBoost algorithm in conjunction with the SMOTETomek algorithm. Contrary to the previous literature, where the missing observations in data are either ignored or imputed with average values, this work utilizes k-Nearest neighbor technique for missing data imputation; thus, an accurate and realistic estimation of the missing data is achieved. To mitigate the biasness to the majority data class, the proposed model utilizes the SMOTETomek algorithm, which neutralizes the mentioned effect by managing a proper balance between over-sampling and under-sampling techniques. Feature Extraction and Scalable Hypothesis (FRESH) algorithm is utilized at the later stage of the proposed NTL detection framework to extract and select the most relevant data features from the provided dataset. Afterward, the model is trained using the CatBoost algorithm to classify the consumers into two distinct categories, i.e., genuine and theft. Finally, to interpret the model’s decision for the corresponding predictions, the tree-SHAP algorithm is utilized. To validate the efficacy of the proposed ML based theft detection approach, its performance is compared with that of the traditional gradient boosting ML algorithms such as XGBoost, lightGBM, Ensemble bagging, boosting ML models, and other conventional ML models using five of the most widely used performance metrics, i.e., precision, accuracy, F1score Kappa and MCC. The proposed technique achieved an accuracy of 93% and a detection rate of 92%, which is significantly higher than all the considered competing algorithms under identical dataset and hyperparameters.

Using Network-Based Machine Learning to Predict Transcription Factors Involved in Drought Resistance.

Gene regulatory networks underpin stress response pathways in plants. However, parsing these networks to prioritize key genes underlying a particular trait is challenging. Here, we have built the Gene Regulation and Association Network (GRAiN) of rice (Oryza sativa). GRAiN is an interactive query-based web-platform that allows users to study functional relationships between transcription factors (TFs) and genetic modules underlying abiotic-stress responses. We built GRAiN by applying a combination of different network inference algorithms to publicly available gene expression data. We propose a supervised machine learning framework that complements GRAiN in prioritizing genes that regulate stress signal transduction and modulate gene expression under drought conditions. Our framework converts intricate network connectivity patterns of 2160 TFs into a single drought score. We observed that TFs with the highest drought scores define the functional, structural, and evolutionary characteristics of drought resistance in rice. Our approach accurately predicted the function of OsbHLH148 TF, which we validated using in vitro protein-DNA binding assays and mRNA sequencing loss-of-function mutants grown under control and drought stress conditions. Our network and the complementary machine learning strategy lends itself to predicting key regulatory genes underlying other agricultural traits and will assist in the genetic engineering of desirable rice varieties.

Pseudocell Tracer-A method for inferring dynamic trajectories using scRNAseq and its application to B cells undergoing immunoglobulin class switch recombination.

Single cell RNA sequencing (scRNAseq) can be used to infer a temporal ordering of cellular states. Current methods for the inference of cellular trajectories rely on unbiased dimensionality reduction techniques. However, such biologically agnostic ordering can prove difficult for modeling complex developmental or differentiation processes. The cellular heterogeneity of dynamic biological compartments can result in sparse sampling of key intermediate cell states. To overcome these limitations, we develop a supervised machine learning framework, called Pseudocell Tracer, which infers trajectories in pseudospace rather than in pseudotime. The method uses a supervised encoder, trained with adjacent biological information, to project scRNAseq data into a low-dimensional manifold that maps the transcriptional states a cell can occupy. Then a generative adversarial network (GAN) is used to simulate pesudocells at regular intervals along a virtual cell-state axis. We demonstrate the utility of Pseudocell Tracer by modeling B cells undergoing immunoglobulin class switch recombination (CSR) during a prototypic antigen-induced antibody response. Our results revealed an ordering of key transcription factors regulating CSR to the IgG1 isotype, including the concomitant expression of Nfkb1 and Stat6 prior to the upregulation of Bach2 expression. Furthermore, the expression dynamics of genes encoding cytokine receptors suggest a poised IL-4 signaling state that preceeds CSR to the IgG1 isotype.

Yolov3 Supervised Machine Learning Framework for Real-Time Object Detection and Localization

Nowadays to detect and classify the objects from the sequence of image frames various machine learning models are used. The performance of the object recognition model is purely depending on the number of trained images. The image processing and pattern extraction focus on object recognition, localization, and classification. To classify the objects from the external events or to detect multiply objects from an image the confident level of the trained weights should be maximum. The existing object detection methods look at the specific region to detect and classify the objects in an image. The proposed object detection system takes an improved YOLOv3 model for object classification. The improved model will look at the entire image to detect and recognize the objects. The YOLOv3 model uses the neural network on the image which split the image into a region and map the confidence probability. The proposed model will detect multiple objects by exploiting the contextual information using a single CNN. The model can process 45 frames in a second and it is suitable for object detection in real-time.

DATA-DRIVEN FAILURE PREDICTION IN BRITTLE MATERIALS: A PHASE FIELD-BASED MACHINE LEARNING FRAMEWORK

Failure in brittle materials led by the evolution of micro- to macro-cracks under repetitive or increasing loads is often catastrophic with no significant plasticity to advert the onset of fracture. Early failure detection with respective location are utterly important features in any practical application, both of which can be effectively addressed using artificial intelligence. In this paper, we develop a supervised machine learning (ML) framework to predict failure in an isothermal, linear elastic and isotropic phase-field model for damage and fatigue of brittle materials. Time-series data of the phase-field model is extracted from virtual sensing nodes at different locations of the geometry. A pattern recognition scheme is introduced to represent time-series data/sensor nodes responses as a pattern with a corresponding label, integrated with ML algorithms, used for damage classification with identified patterns. We perform an uncertainty analysis by superposing random noise to the time-series data to assess the robustness of the framework with noise-polluted data. Results indicate that the proposed framework is capable of predicting failure with acceptable accuracy even in the presence of high noise levels. The findings demonstrate satisfactory performance of the supervised ML framework, and the applicability of artificial intelligence and ML to a practical engineering problem, i.,e, data-driven failure prediction in brittle materials.