Single Learning Model Research Articles

A novel model for relation prediction in knowledge graphs exploiting semantic and structural feature integration

Relation prediction is a critical task in knowledge graph completion and associated downstream tasks that rely on knowledge representation. Previous studies indicate that both structural features and semantic information are meaningful for predicting missing relations in knowledge graphs. This has led to the development of two types of methods: structure-based methods and semantics-based methods. Since these two approaches represent two distinct learning paradigms, it is difficult to fully utilize both sets of features within a single learning model, especially deep features. As a result, existing studies usually focus on only one type of feature. This leads to an insufficient representation of knowledge in current methods and makes them prone to overlooking certain patterns when predicting missing relations. In this study, we introduce a novel model, RP-ISS, which combines deep semantic and structural features for relation prediction. The RP-ISS model utilizes a two-part architecture, with the first component being a RoBERTa module that is responsible for extracting semantic features from entity nodes. The second part of the system employs an edge-based relational message-passing network designed to capture and interpret structural information within the data. To alleviate the computational burden of the message-passing network on the RoBERTa module during the sampling process, RP-ISS introduces a node embedding memory bank, which updates asynchronously to circumvent excessive computation. The model was assessed on three publicly accessible datasets (WN18RR, WN18, and FB15k-237), and the results revealed that RP-ISS surpasses all baseline methods across all evaluation metrics. Moreover, RP-ISS showcases robust performance in graph inductive learning.

CVs Classification Using Neural Network Approaches Combined with BERT and Gensim: CVs of Moroccan Engineering Students

Deep learning (DL)-oriented document processing is widely used in different fields for extraction, recognition, and classification processes from raw corpus of data. The article examines the application of deep learning approaches, based on different neural network methods, including Gated Recurrent Unit (GRU), long short-term memory (LSTM), and convolutional neural networks (CNNs). The compared models were combined with two different word embedding techniques, namely: Bidirectional Encoder Representations from Transformers (BERT) and Gensim Word2Vec. The models are designed to evaluate the performance of architectures based on neural network techniques for the classification of CVs of Moroccan engineering students at ENSAK (National School of Applied Sciences of Kenitra, Ibn Tofail University). The used dataset included CVs collected from engineering students at ENSAK in 2023 for a project on the employability of Moroccan engineers in which new approaches were applied, especially machine learning, deep learning, and big data. Accordingly, 867 resumes were collected from five specialties of study (Electrical Engineering (ELE), Networks and Systems Telecommunications (NST), Computer Engineering (CE), Automotive Mechatronics Engineering (AutoMec), Industrial Engineering (Indus)). The results showed that the proposed models based on the BERT embedding approach had more accuracy compared to models based on the Gensim Word2Vec embedding approach. Accordingly, the CNN-GRU/BERT model achieved slightly better accuracy with 0.9351 compared to other hybrid models. On the other hand, single learning models also have good metrics, especially based on BERT embedding architectures, where CNN has the best accuracy with 0.9188.

Conversational agent in HCI a review

Nowadays, AI technology is developing rapidly and slowly covering all areas of life. AI facilitates different parts of our lives and provides us with a lot of help. For example, in learning, students can acquire knowledge more conveniently through AI, and buyers can find suitable goods more conveniently through AI when shopping. At the same time, more and more technology is used in the field of voice agents, which allows humans to enjoy a lot of better services. In this article, we will study to better understand "how to understand human natural language and how the repository of knowledge is built." In the article we build with examples and deep learning models (CNN and RNN) through databases. Through repeated research and analysis, we can find that there are some limitations in this paper, such as the single learning model and the insufficient elaboration of data analysis and signal system technology. But at the same time, we also found a lot of future application prospects that voice agents can develop, it can be applied in many fields, such as finance, medical care, education and so on. For example, in children's education, parents can use voice agents to set time limits and monitor their children's progress. Existing digital interactive storytelling systems have limitations in terms of available storybooks and hand-crafted issues. Voice agents are becoming more popular in everyday scenarios, and more users are adopting devices like Siri and Google Assistant. In the future, conversational agents are expected to play an important role in oral communication with users.

Merging or ensembling: integrative analysis in multiple neuroimaging studies.

The aim of this paper is to systematically investigate merging and ensembling methods for spatially varying coefficient mixed effects models (SVCMEM) in order to carry out integrative learning of neuroimaging data obtained from multiple biomedical studies. The "merged" approach involves training a single learning model using a comprehensive dataset that encompasses information from all the studies. Conversely, the "ensemble" approach involves creating a weighted average of distinct learning models, each developed from an individual study. We systematically investigate the prediction accuracy of the merged and ensemble learners under the presence of different degrees of interstudy heterogeneity. Additionally, we establish asymptotic guidelines for making strategic decisions about when to employ either of these models in different scenarios, along with deriving optimal weights for the ensemble learner. To validate our theoretical results, we perform extensive simulation studies. The proposed methodology is also applied to 3 large-scale neuroimaging studies.

A Calibrator Fuzzy Ensemble for Highly-Accurate Robot Arm Calibration.

The absolute positioning accuracy of an industrial robot arm is vital for advancing manufacturing-related applications like automatic assembly, which can be improved via the data-driven approaches to robot arm calibration. Existing data-driven calibrators have illustrated their efficiency in addressing the issue of robot arm calibration. However, they mostly are single learning models that can be easily affected by the insufficient representation of the solution space, therefore, suffering from the calibration accuracy loss. To address this issue, this study proposes a calibrator fuzzy ensemble (CFE) with twofold ideas: 1) implementing eight data-driven calibrators relying on different sophisticated machine learning algorithms for an industrial robot arm, which guarantees the accuracy of individual base models and 2) innovatively developing a fuzzy ensemble of the obtained eight diversified calibrators to obtain impressively high calibration accuracy for an industrial robot arm. Extensive experiments on an ABB IRB120 industrial robot implemented with MATLAB demonstrate that compared with state-of-the-art calibrators, CFE decreases the maximum error at 8.59%. Hence, it has great potential for real applications.

HDSNE a new unsupervised multiple image database fusion learning algorithm with flexible and crispy production of one database: a proof case study of lung infection diagnose In chest X-ray images

Continuous release of image databases with fully or partially identical inner categories dramatically deteriorates the production of autonomous Computer-Aided Diagnostics (CAD) systems for true comprehensive medical diagnostics. The first challenge is the frequent massive bulk release of medical image databases, which often suffer from two common drawbacks: image duplication and corruption. The many subsequent releases of the same data with the same classes or categories come with no clear evidence of success in the concatenation of those identical classes among image databases. This issue stands as a stumbling block in the path of hypothesis-based experiments for the production of a single learning model that can successfully classify all of them correctly. Removing redundant data, enhancing performance, and optimizing energy resources are among the most challenging aspects. In this article, we propose a global data aggregation scale model that incorporates six image databases selected from specific global resources. The proposed valid learner is based on training all the unique patterns within any given data release, thereby creating a unique dataset hypothetically. The Hash MD5 algorithm (MD5) generates a unique hash value for each image, making it suitable for duplication removal. The T-Distributed Stochastic Neighbor Embedding (t-SNE), with a tunable perplexity parameter, can represent data dimensions. Both the Hash MD5 and t-SNE algorithms are applied recursively, producing a balanced and uniform database containing equal samples per category: normal, pneumonia, and Coronavirus Disease of 2019 (COVID-19). We evaluated the performance of all proposed data and the new automated version using the Inception V3 pre-trained model with various evaluation metrics. The performance outcome of the proposed scale model showed more respectable results than traditional data aggregation, achieving a high accuracy of 98.48%, along with high precision, recall, and F1-score. The results have been proved through a statistical t-test, yielding t-values and p-values. It’s important to emphasize that all t-values are undeniably significant, and the p-values provide irrefutable evidence against the null hypothesis. Furthermore, it’s noteworthy that the Final dataset outperformed all other datasets across all metric values when diagnosing various lung infections with the same factors.

Modeling the load carrying capacity of corroded reinforced concrete compression bending members using explainable machine learning

Corrosion of reinforcement can lead to a decrease in the load carrying capacity of reinforced concrete structures and affect their safety. Therefore, accurate evaluation of the ultimate load carrying capacity is crucial for the maintenance and reinforcement of corroded reinforced concrete structures. In this paper, based on experimental research data of 192 corroded reinforced concrete compression bending members, data-driven analysis was conducted using ANN, SVM, RF, and AdaBoost algorithms to establish the relationship between the influencing factors of the load carrying capacity and their ultimate load carrying capacity. The input variables include the section width of the member, section height of the member, length of member, yield strength of reinforcement, diameter of longitudinal reinforcement, compressive strength of concrete, thickness of concrete cover, hoop diameter, original eccentricity, corrosion rate and the ultimate load carrying capacity is the output variable. Additionally, this study innovatively utilizes the Shapley additive explanations (SHAP) method to enhance the interpretability of the ML models, overcoming the "black box" issue associated with ML methods. Furthermore, the performance of the ML models is compared with theoretical formulas. The results indicate that the ML models exhibit good predictive performance, with higher accuracy than thetheoretical calculation formulas. And the predictive performance of ensemble learning models (RF, AdaBoost) is better than that of single learning models (ANN, SVM). The newly developed hybrid ML model is likely to become a new choice for dealing with the load carrying capacity problem of corroded reinforced concrete compression bending members.

MSENet: Mean and standard deviation based ensemble network for cervical cancer detection

Cervical cancer is one of the most concerning carcinogenic diseases among women worldwide. The condition is especially bad in low- or middle-income countries due to the lack of medical facilities. In such situations, computer-aided diagnosis (CAD) systems can alleviate the need to a large extent. However, sometimes a single learning model may not be effective enough to capture relevant information for accurate prediction of diseases from complex data. To this end, we propose an ensemble of deep learning models, called Mean and Standard Deviation-based Ensemble Network (MSENet), for detecting cervical cancer from Pap smear images. Our ensemble model consists of three standard base classifiers, namely Xception, Inception V3, and VGG-16. We further improve the classification abilities of these models by implementing a novel probability enhancement scheme that takes into account the mean and standard deviation of the confidence scores. This technique enables the overall framework to capture complementary information offered by the base classifiers and is tailored to the characteristics of deep learners. Finally, we use the product rule to aggregate the obtained outcomes and get final predictions. The proposed MSENet has been evaluated on a standard public benchmark dataset, called SIPaKMeD. With a classification accuracy of 97.21% using a 5-fold cross-validation scheme, the MSENet outperforms many state-of-the-art methods. The source codes are made public in the following Github repository https://github.com/rishavpramanik/msenet.

Differential Gene Expression Prediction by Ensemble Deep Networks on Histone Modification Data.

Predicting differential gene expression (DGE) from Histone modifications (HM) signal is crucial to understand how HM controls cell functional heterogeneity through influencing differential gene regulation. Most existing prediction methods use fixed-length bins to represent HM signals and transmit these bins into a single machine learning model to predict differential expression genes of single cell type or cell type pair. However, the inappropriate bin length may cause the splitting of the important HM segment and lead to information loss. Furthermore, the bias of single learning model may limit the prediction accuracy. Considering these problems, in this paper, we proposes an Ensemble deep neural networks framework for predicting Differential Gene Expression (EnDGE). EnDGE employs different feature extractors on input HM signal data with different bin lengths and fuses the feature vectors for DGE prediction. Ensemble multiple learning models with different HM signal cutting strategies helps to keep the integrity and consistency of genetic information in each signal segment, and offset the bias of individual models. Besides the popular feature extractors, we also propose a new Residual Network based model with higher prediction accuracy to increase the diversity of feature extractors. Experiments on the real datasets from the Roadmap Epigenome Project (REMC) show that for all cell type pairs, EnDGE significantly outperforms the state-of-the-art baselines for differential gene expression prediction.

Identification of Neurodegenerative Diseases From Gait Rhythm Through Time Domain and Time-Dependent Spectral Descriptors.

The analysis of gait rhythm by pattern recognition can support the state-of-the-art clinical methods for the identification of neurodegenerative diseases (NDD). In this study, we investigated the use of time domain (TD) and time-dependent spectral features (PSDTD) for detecting NDD sub-types. Also, we proposed two classification pathways for supporting NDD diagnosis, the first one made by a two-step learning phase, whereas the second one encompasses a single learning model. We considered stride-to-stride fluctuation data of healthy controls (CN), patients affected by Parkinson's disease (PD), Huntington's disease (HD), and amyotrophic lateral sclerosis (AS). TD feature set provided good results to distinguish between CN and NDDs, while performances lowered for specific NDD identification. PSDTD features boosted the accuracy of each binary identification task. With k-nearest neighbor classifier, the first diagnosis pathway reached 98.76% accuracy to distinguish between CN and NDD and 94.56% accuracy for NDDs sub-types, whereas the second pathway offered an overall accuracy of 94.84% for a 4-class classification task. Outcomes of this study indicate that the use of TD and PSDTD features, simple to extract and with a low computational load, provides reliable results in terms of NDD identification, being also useful for the development of gait rhythm computer-aided NDD detection systems.

NIDM: Novel Intrusion Detection Model Using CNN-LSTM for Big Data Environment

Abstract: Machine learning-based (IDS) have become a critical component of safeguarding our economic and national security because of the massive quantities of data produced each day and the growing interconnection of the world's Internet infrastructures. The existing machine Learning Model technique may have difficulty comprehending the ever-increasingly complex distribution of data invasion patterns. With a small number of data points, a single deep learning algorithm may be ineffective at capturing different patterns for intrusive attacks. We presented CNN-LSTM Novel Intrusion Detection Model for Big Data to improve the efficiency of IDS-based CNN-LSTM even further (NIDM). NIDM uses behavioural traits and content functions to understand the characteristics when compared to earlier single learning model tactics, this strategy can improve the rate of intrusive attack detection. Keywords: IDS, Machine Learning, LSTM, CNN.

On implementing a powerful intrusion prevention system focused on big data

A machine-based intrusion sensing system has become an essential component in network safeguarding and information security due to the regular development of large volumes of data and the increasing interconnection among global Internet infrastructures. Previous shallow learning and deep learning techniques follow a similar model learning approach to intrusion detection. To explain the increasingly complex data delivery intrusion patterns, a single learning model methodology can face problems. Specifically, a single deep learning model cannot capture special patterns in intrusive attacks with a few experiments. We propose a hierarchical deep learning system based on big data to further boost the efficiency of IDS-based machine learning. It uses behavioral and content-functional functionality to capture both network traffic and content details. Each deep learning model in the proposed framework focuses on learning the particular data distribution in a single cluster. Compared to previous single learning models, this technique improves the classification rate of disruptive attacks. The model building time of the system is reduced considerably when several devices are implemented.

An Enhanced Ensemble Learning-Based Fault Detection and Diagnosis for Grid-Connected PV Systems

The main objective of this article is to develop an enhanced ensemble learning (EL) based intelligent fault detection and diagnosis (FDD) paradigms that aim to ensure the high-performance operation of Grid-Connected Photovoltaic (PV) systems. The developed EL based techniques consist in combining multiple learning models instead of using a single learning model. To do that, three EL-based FDD techniques are proposed. First, an EL technique that merges the benefits of Support Vector Machine (SVM), K-Nearest Neighbour (KNN), and Decision Tree (DT) is presented. The developed method contributes to the reduction of the overall diagnosis error and has the ability to combine various models. However, classical EL models ignore the time-dependence of PV measurements. In addition, the PV system data are frequently time-correlated. Therefore, kernel PCA (KPCA)-based EL and reduced KPCA (RKPCA)-based EL techniques are developed to take into consideration the dynamic and multivariate natures of the PV measurements. The two proposed KPCA -based EL and RKPCA-based EL techniques are addressed so that the features extraction and selection phases are performed using the KPCA and RKPCA models and the sensitive and significant characteristics are transmitted to the EL model for classification purposes. The presented results prove that the proposed EL based methods offer enhanced diagnosis performances when applied to PV systems.

Sparkle: User-Aware Viewport Prediction in 360-Degree Video Streaming

In 360-degree video streaming, users commonly watch a video scene within a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Field of View (FoV)</i> . Such observation provides an opportunity to save bandwidth consumption by predicting and then prefetching video tiles within the FoV. However, existing FoV prediction methods seldom consider the diversity among user behaviors and the impact of different video genres. Thus, previous one-size-fits-all models cannot make accurate prediction for users with different behavior patterns. In this paper, we propose a user-aware viewport prediction algorithm called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sparkle</i> , which is a practical whitebox approach for FoV prediction. Instead of training a single learning model to predict the behaviors for all users, our proposed algorithm is tailored to fit each individual user. In particular, unlike other learning models, our prediction model is completely explainable and all the parameters have their physical meanings. We first conduct a measurement study to analyze real user behaviors and observe that there exists sharp fluctuation of view orientation and user posture has significant impact on the viewport movement of users. Moreover, cross-user similarity is diverse across different video genres. Inspired by these insights, we further design a user-aware viewport prediction algorithm by mimicking a user's viewport movement on the tile map, and determine how a user will change the viewport angle based on his (or her) trajectory and other similar users’ behaviors in the past time window. Extensive evaluations with real datasets demonstrate that, our proposed algorithm significantly outperforms the state-of-the-art benchmark methods (e.g., LSTM-based methods) by over <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{5}\%$</tex-math></inline-formula> , and the prediction accuracy is much more stable on various types of 360-degree videos than previous methods.

Predicting defibrillation success in out-of-hospital cardiac arrested patients: Moving beyond feature design

ObjectiveOptimizing timing of defibrillation by evaluating the likelihood of a successful outcome could significantly enhance resuscitation. Previous studies employed conventional machine learning approaches and hand-crafted features to address this issue, but none have achieved superior performance to be widely accepted. This study proposes a novel approach in which predictive features are automatically learned. MethodsA raw 4s VF episode immediately prior to first defibrillation shock was feed to a 3-stage CNN feature extractor. Each stage was composed of 4 components: convolution, rectified linear unit activation, dropout and max-pooling. At the end of feature extractor, the feature map was flattened and connected to a fully connected multi-layer perceptron for classification. For model evaluation, a 10 fold cross-validation was employed. To balance classes, SMOTE oversampling method has been applied to minority class. ResultsThe obtained results show that the proposed model is highly accurate in predicting defibrillation outcome (Acc = 93.6 %). Since recommendations on classifiers suggest at least 50 % specificity and 95 % sensitivity as safe and useful predictors for defibrillation decision, the reported sensitivity of 98.8 % and specificity of 88.2 %, with the analysis speed of 3 ms/input signal, indicate that the proposed model possesses a good prospective to be implemented in automated external defibrillators. ConclusionsThe learned features demonstrate superiority over hand-crafted ones when performed on the same dataset. This approach benefits from being fully automatic by fusing feature extraction, selection and classification into a single learning model. It provides a superior strategy that can be used as a tool to guide treatment of OHCA patients in bringing optimal decision of precedence treatment. Furthermore, for encouraging replicability, the dataset has been made publicly available to the research community.

Autoencoder-based unsupervised clustering and hashing

Faced with a large amount of data and high-dimensional data information in a database, the existing exact nearest neighbor retrieval methods cannot obtain ideal retrieval results within an acceptable retrieval time. Therefore, researchers have begun to focus on approximate nearest neighbor retrieval. Recently, the hashing-based approximate nearest neighbor retrieval method has attracted increasing attention because of its small storage space and high retrieval efficiency. The development of neural networks has also promoted progress in hash learning. However, these methods are mostly supervised. In practical applications, annotating large amounts of data is a very time-consuming and laborious task. Furthermore, efficiently using a large amount of unlabeled data for hash learning is challenging. In this paper, we create a new autoencoder variant to efficiently capture the features of high-dimensional data, and propose an unsupervised deep hashing method for large-scale data retrieval, named as Autoencoder-based Unsupervised Clustering and Hashing (AUCH). By constructing a hashing layer as a hidden layer of the autoencoder, hash learning is performed together with unsupervised clustering by minimizing the overall loss. AUCH can unify unsupervised clustering and retrieval tasks into a single learning model. In addition, the method can use a deep neural network to simultaneously learn feature representations, hashing functions and cluster assignments. Experimental results on standard datasets indicate that AUCH achieves competitive results compared to state-of-the-art models for retrieval and clustering tasks.

Applying big data based deep learning system to intrusion detection

With vast amounts of data being generated daily and the ever increasing interconnectivity of the world's internet infrastructures, a machine learning based Intrusion Detection Systems (IDS) has become a vital component to protect our economic and national security. Previous shallow learning and deep learning strategies adopt the single learning model approach for intrusion detection. The single learning model approach may experience problems to understand increasingly complicated data distribution of intrusion patterns. Particularly, the single deep learning model may not be effective to capture unique patterns from intrusive attacks having a small number of samples. In order to further enhance the performance of machine learning based IDS, we propose the Big Data based Hierarchical Deep Learning System (BDHDLS). BDHDLS utilizes behavioral features and content features to understand both network traffic characteristics and information stored in the payload. Each deep learning model in the BDHDLS concentrates its efforts to learn the unique data distribution in one cluster. This strategy can increase the detection rate of intrusive attacks as compared to the previous single learning model approaches. Based on parallel training strategy and big data techniques, the model construction time of BDHDLS is reduced substantially when multiple machines are deployed.

Addressing database variability in learning from medical data: An ensemble-based approach using convolutional neural networks and a case of study applied to automatic sleep scoring

In this work we examine some of the problems associated with the development of machine learning models with the objective to achieve robust generalization capabilities on common-task multiple-database scenarios. Referred to as the “database variability problem”, we focus on a specific medical domain (sleep staging in sleep medicine) to show the non-triviality of translating the estimated model's local generalization capabilities into independent external databases. We analyze some of the scalability problems when multiple-database data are used as inputs to train a single learning model. Then, we introduce a novel approach based on an ensemble of local models, and we show its advantages in terms of inter-database generalization performance and data scalability. In addition, we analyze different model configurations and data pre-processing techniques to determine their effects on the overall generalization performance. For this purpose, we carry out experimentation that involves several sleep databases and evaluates different machine learning models based on convolutional neural networks.

Single and ensemble classifiers for defect prediction in sheet metal forming under variability

This paper presents an approach, based on machine learning techniques, to predict the occurrence of defects in sheet metal forming processes, exposed to sources of scatter in the material properties and process parameters. An empirical analysis of performance of ML techniques is presented, considering both single learning and ensemble models. These are trained using data sets populated with numerical simulation results of two sheet metal forming processes: U-Channel and Square Cup. Data sets were built for three distinct steel sheets. A total of eleven input features, related to the mechanical properties, sheet thickness and process parameters, were considered; also, two types of defects (outputs) were analysed for each process. The sampling data were generated, assuming that the variability of each input feature is described by a normal distribution. For a given type of defect, most single classifiers show similar performances, regardless of the material. When comparing single learning and ensemble models, the latter can provide an efficient alternative. The fact that ensemble predictive models present relatively high performances, combined with the possibility of reconciling model bias and variance, offer a promising direction for its application in industrial environment.

SVM with a neutral class

In many real binary classification problems, in addition to the presence of positive and negative classes, we are also given the examples of third neutral class, i.e., the examples with uncertain or intermediate state between positive and negative. Although it is a common practice to ignore the neutral class in a learning process, its appropriate use can lead to the improvement in classification accuracy. In this paper, to include neutral examples in a training stage, we adapt two variants of Tri-Class SVM (proposed by Angulo et al. in Neural Process Lett 23(1):89–101, 2006), the method designed to solve three-class problems with a use of single learning model. In analogy to classical SVM, we look for such a hyperplane, which maximizes the margin between positive and negative instances and which is localized as close to the neutral class as possible. In addition to original Angulo’s paper, we give a new interpretation of the model and show that it can be easily implemented in the primal. Our experiments demonstrate that considered methods obtain better results in binary classification problems than classical SVM and semi-supervised SVM.