Instance-based Learning Algorithm Research Articles

Genomic prediction using machine learning: a comparison of the performance of regularized regression, ensemble, instance-based and deep learning methods on synthetic and empirical data

BackgroundThe accurate prediction of genomic breeding values is central to genomic selection in both plant and animal breeding studies. Genomic prediction involves the use of thousands of molecular markers spanning the entire genome and therefore requires methods able to efficiently handle high dimensional data. Not surprisingly, machine learning methods are becoming widely advocated for and used in genomic prediction studies. These methods encompass different groups of supervised and unsupervised learning methods. Although several studies have compared the predictive performances of individual methods, studies comparing the predictive performance of different groups of methods are rare. However, such studies are crucial for identifying (i) groups of methods with superior genomic predictive performance and assessing (ii) the merits and demerits of such groups of methods relative to each other and to the established classical methods. Here, we comparatively evaluate the genomic predictive performance and informally assess the computational cost of several groups of supervised machine learning methods, specifically, regularized regression methods, deep, ensemble and instance-based learning algorithms, using one simulated animal breeding dataset and three empirical maize breeding datasets obtained from a commercial breeding program.ResultsOur results show that the relative predictive performance and computational expense of the groups of machine learning methods depend upon both the data and target traits and that for classical regularized methods, increasing model complexity can incur huge computational costs but does not necessarily always improve predictive accuracy. Thus, despite their greater complexity and computational burden, neither the adaptive nor the group regularized methods clearly improved upon the results of their simple regularized counterparts. This rules out selection of one procedure among machine learning methods for routine use in genomic prediction. The results also show that, because of their competitive predictive performance, computational efficiency, simplicity and therefore relatively few tuning parameters, the classical linear mixed model and regularized regression methods are likely to remain strong contenders for genomic prediction.ConclusionsThe dependence of predictive performance and computational burden on target datasets and traits call for increasing investments in enhancing the computational efficiency of machine learning algorithms and computing resources.

Safeguarding Connections: Machine Learning Powered Intrusion Detection

Abstract: It's crucial to have reliable intrusion detection systems. a cutting-edge method of machine learning-based intrusion detection. Our solution uses cutting-edge algorithms to detect and eliminate any threats instantly, acting as a preventative measure against a wide range of cyberattacks. Since the model has been trained on a large number of datasets, it can eventually strengthen network security by evolving and adapting to new threats. Naive Bayes (NB) classifiers and correlation-based feature selection (CFS) methods are used to reduce the amount of data. For attack classification, the Intrusion Detection System recommends using an Instance-Based Learning algorithm (IBK) in combination with a Multilayer Perceptron (MLP).

The study and application of brain tumors classification based on artificial intelligence

The application of AI in the medical field has been increasingly popular due to its ability to enhance the accuracy and efficiency of diagnoses. A plurality of individuals employs diverse techniques and refine existing methods in order to achieve greater precision. This paper provides an overview of the methodology utilized in early research papers for brain tumors classification. This includes the input of datasets, preprocessing, model building, training and testing, evaluation, and application. In addition, this paper presents four models used in early research, including the Artificial neural network (ANN) which mimics the organization and operation of neural networks in living organisms by processing information through interconnected nodes or neurons. Another model is k-Nearest Neighbor (KNN), an instance-based learning algorithm that labels new data points by comparing them to the K closest labeled data points in the training dataset. The third model is Visual Geometry Group-16 (VGG-16), a 16-layer Convolutional Neural Network (CNN) that is highly regarded for its simplicity and effectiveness and is one of the top-performing CNN models used by VGG. Lastly, GN-AlexNet is a hybrid learning mode that combines the GoogleNet structure with the AlexNet model. The remainder of the article explains the utilization of this AI in three application scenarios, namely as an assistant for doctors, patients, and insurance companies. Moreover, the paper highlights potential challenges that may arise in both the present and future, such as patient mistrust and the emergence of new models like Vision Transformer (ViT) that could potentially outperform CNNs.

Integration of sEMG-Based Learning and Adaptive Fuzzy Sliding Mode Control for an Exoskeleton Assist-as-Needed Support System

This paper presents an adaptive Fuzzy Sliding Mode Control approach for an Assist-as-Needed (AAN) strategy to achieve effective human–exoskeleton synergy. The proposed strategy employs an adaptive instance-based learning algorithm to estimate muscle effort, based on surface Electromyography (sEMG) signals. To determine and control the inverse dynamics of a highly nonlinear 4-degrees-of-freedom exoskeleton designed for upper-limb therapeutic exercises, a modified Recursive Newton-Euler Algorithm (RNEA) with Sliding Mode Control (SMC) was used. The exoskeleton position error and raw sEMG signal from the bicep’s brachii muscle were used as inputs for a fuzzy inference system to produce an output to adjust the sliding mode control law parameters. The proposed robust control law was simulated using MATLAB-Simulink, and the results showed that it could instantly adjust the necessary support, based on the combined motion of the human–exoskeleton system’s muscle engagement, while keeping the state trajectory errors and input torque bounded within ±5×10−2 rads and ±5 N.m, respectively.

Integration of instance-based learning and computed torque control for an effective assist-as-needed support in human-exoskeleton interaction

The physiological responses that arise from human-robot interaction may vary across subjects in magnitude and rate. Such individual variations may require instance-based learning over model-based learning algorithms. For instance, a wearable assist-as-needed exoskeleton may require real-time progress data to provide the appropriate level of support to a specific user. In this study, an instance-based learning algorithm was developed and integrated with a computed torque control law. Real-time bio-signals, in the form of electromyography (EMG), were tracked during a predetermined time window to quantify an adaptive threshold value and to control the torque at the exoskeleton joints. These signals were fed to the algorithm, which instantly learned and determined the support needed to accomplish a desired task. The algorithm was tested on a 5-degree-of-freedom wearable exoskeleton used in the automation of upper-limb therapeutic exercises. Results indicated that the algorithm offered the ability to adjust assist-as-needed support instantly based on the amount of muscle engagement present in the combined motion of the human-exoskeleton system.

An Insight into the Prediction of Scale Precipitation in Harsh Conditions Using Different Machine Learning Algorithms

Summary Scale precipitation in petroleum equipment is known as an important problem that causes damages in injection and production wells. Scale precipitation causes equipment corrosion and flow restriction and consequently a reduction in oil production. Due to this fact, the prediction of scale precipitation has vital importance among petroleum engineers. In the current work, different intelligent models, including the decision tree, random forest (RF), artificial neural network (ANN), K-nearest neighbors (KNN), convolutional neural network (CNN), support vector machine (SVM), ensemble learning, logistic regression, Naïve Bayes, and adaptive boosting (AdaBoost), are used to estimate scale formation as a function of pH and ionic compositions. Also, a sensitivity analysis is done to determine the most influential parameters on scale formation. The novelty of this work is to compare the performance of 10 different machine learning algorithms at modeling an extremely non-linear relationship between the inputs and the outputs in scale precipitation prediction. After determining the best models, they can be used to determine scale formation by manipulating the concentration of a variable in accordance with the result of the sensitivity analysis. Different classification metrics, including the accuracy, precision, F1-score, and recall, were used to compare the performance of the mentioned models. Results in the testing phase showed that the KNN and ensemble learning were the most accurate tools based on all performance metrics of solving the classification of scale/no-scale problem. As the output had an extremely non-linear behavior in terms of the inputs, an instance-based learning algorithm such as the KNN best suited the classification task in this study. This argumentation was backed by the classification results. Furthermore, the SVM, Naïve Bayes, and logistic regression performance metrics were not satisfactory in the prediction of scale formation. Note that the hyperparameters of the models were found by grid search and random search approaches. Finally, the sensitivity analysis showed that the variations in the concentration of Ca had the highest impact on scale precipitation.

Instance-based learning using the half-space proximal graph

• We propose a parameter-free instance-based classifier , the HSP classifier. • Our accuracy outperforms kNN while keeping its simplicity and complexity. • We also present a probabilistic and asymptotic versions of the HSP improving both the time and the original proposal’s accuracy. • The same weigthing methods that improves kNN, also improves the HSP and its asymptotic variants The primary example of instance-based learning is the k -nearest neighbor rule (kNN), praised for its simplicity and the capacity to adapt to new unseen data and toss away old data. The main disadvantages often mentioned are the classification complexity , which is O ( n ) , and the estimation of the parameter k , the number of nearest neighbors to be used. The use of indexes at classification time lifts the former disadvantage, while there is no conclusive method for the latter. This paper presents a parameter-free instance-based learning algorithm using the Half-Space Proximal (HSP) graph. The HSP neighbors simultaneously possess proximity and variety concerning the center node. To classify a given query, we compute its HSP neighbors and apply a simple majority rule over them. In our experiments, the resulting classifier bettered K N N for any k in a battery of datasets. This improvement sticks even when applying weighted majority rules to both kNN and HSP classifiers. Surprisingly, when using a probabilistic index to approximate the HSP graph and consequently speeding-up the classification task , our method could improve its accuracy in stark contrast with the kNN classifier, which worsens with a probabilistic index.

Learning GUI Completions with User-defined Constraints

A key objective in the design of graphical user interfaces (GUIs) is to ensure consistency across screens of the same product. However, designing a compliant layout is time-consuming and can distract designers from creative thinking. This paper studies layout recommendation methods that fulfill such consistency requirements using machine learning. Given a desired element type and size, the methods suggest element placements following real-world GUI design processes. Consistency requirements are given implicitly through previous layouts from which patterns are to be learned, comparable to existing screens of a software product. We adopt two recently proposed methods for this task, a Graph Neural Network (GNN) and a Transformer model, and compare them with a custom approach based on sequence alignment and nearest neighbor search (kNN) . The methods were tested on handcrafted datasets with explicit layout patterns, as well as large-scale public datasets of diverse mobile design layouts. Our results show that our instance-based learning algorithm outperforms both neural network approaches. Ultimately, this work contributes to establishing smarter design tools for professional designers with explainable algorithms that increase their efficacy.

Exploring Internet Meme Activity during COVID-19 Lockdown Using Artificial Intelligence Techniques

ABSTRACT The sudden outbreak of the novel coronavirus (nCoV-19, COVID-19) and its rampant spread led to a significant number of people being infected worldwide and disrupted several businesses. With most of the countries imposing serious lockdowns due to the increasing number of fatalities, the social lives of millions of people were affected. Although the lockdown led to an increase in network activities, online shopping, and social network usage, it also raised questions On the mental wellness of society. Interestingly, excessive usage of social networks also witnessed humor traveling across the Internet in the form of Internet Memes during the lockdown period. Humor is known to affect our well-being, decision-making, and psychological systems. In this paper, we have analyzed the Internet Meme activity in Social Networks during the COVID-19 Lockdown period. As humor is known to relieve individuals from psychological stress, it is necessary to understand how human beings adopted Internet Memes for coping up with the lockdown stress and stress-relieving mechanism during the lockdown period. In this paper, we have considered thirty popular memes and the increase in the number of their captions within the period (September 2017 to August 2020). An increase in Internet Meme activity since the lockdown period (March 2020) depicts an increase in online social behavior. We analyze the internet meme activity in social networks during the COVID-19 lockdown period using random forest, multi-layer perceptron, and instance-based learning algorithms followed by data visualization using line graph and Heat Map (8 & 15 clustered). We also compared the performance of the models using evaluation parameters like mean absolute error, root-mean-squared error & Kappa statistics and observed that random forest and instance-based learning algorithms perform better than multi-layer perceptrons. The result indicates that random forest and instance-based learning classifiers are having near perfect classification tendencies whereas multi-layer perceptrons showed around 97% classification accuracy.

Improved Salp Swarm Optimization Algorithm: Application in Feature Weighting for Blind Modulation Identification

In modulation identification issues, like in any other classification problem, the performance of the classification task is significantly impacted by the feature characteristics. Feature weighting boosts the performance of machine learning algorithms, particularly the class of instance-based learning algorithms such as the Minimum Distance (MD) classifier, in which the distance measure is highly sensitive to the magnitude of features. In this paper, we propose an improved version of the Salp Swarm optimization Algorithm (SSA), called ISSA, that will be applied to optimize feature weights for an MD classifier. The aim is to improve the performance of a blind digital modulation detection approach in the context of multiple-antenna systems. The improvements introduced to SSA mainly rely on the opposition-based learning technique. Computer simulations show that the ISSA outperforms the SSA as well as the algorithms that derive from it. The ISSA also exhibits the best performance once it is applied for feature weighting in the above context.

Optimal 1-NN prototypes for pathological geometries.

Using prototype methods to reduce the size of training datasets can drastically reduce the computational cost of classification with instance-based learning algorithms like the k-Nearest Neighbour classifier. The number and distribution of prototypes required for the classifier to match its original performance is intimately related to the geometry of the training data. As a result, it is often difficult to find the optimal prototypes for a given dataset, and heuristic algorithms are used instead. However, we consider a particularly challenging setting where commonly used heuristic algorithms fail to find suitable prototypes and show that the optimal number of prototypes can instead be found analytically. We also propose an algorithm for finding nearly-optimal prototypes in this setting, and use it to empirically validate the theoretical results. Finally, we show that a parametric prototype generation method that normally cannot solve this pathological setting can actually find optimal prototypes when combined with the results of our theoretical analysis.

Prediction of Compressive Strength of Concrete and Rock Using an Elementary Instance-Based Learning Algorithm

The use of machine learning techniques to predict material strength is becoming popular. However, not much attention has been paid to instance-based learning (IBL) algorithms. Therefore, in order to predict material strength, as the direct method by conducting tests is time-consuming and expensive and experimental errors are inevitable, an indirect method based on elementary instance-based learning algorithm was proposed. The standard k-nearest neighbors (k-NN) with cross-validation were utilized to develop compressive strength prediction models for some concretes and rocks by considering indirect parameters such as physical and mechanical parameters. Results on applying this method to datasets from literature studies show that the values of RMSE for k-NN are modest, indicating adequacy to predict compressive strength with comprehensive range values of predictors. Additionally, the R2-values of the k-NN models were high. In other words, the models were able to explain the variance in compressive strength for data with a wide range of input values.

Fast and Accurate Terrain Image Classification for ASTER Remote Sensing by Data Stream Mining and Evolutionary-EAC Instance-Learning-Based Algorithm

Remote sensing streams continuous data feed from the satellite to ground station for data analysis. Often the data analytics involves analyzing data in real-time, such as emergency control, surveillance of military operations or scenarios that change rapidly. Traditional data mining requires all the data to be available prior to inducing a model by supervised learning, for automatic image recognition or classification. Any new update on the data prompts the model to be built again by loading in all the previous and new data. Therefore, the training time will increase indefinitely making it unsuitable for real-time application in remote sensing. As a contribution to solving this problem, a new approach of data analytics for remote sensing for data stream mining is formulated and reported in this paper. Fresh data feed collected from afar is used to approximate an image recognition model without reloading the history, which helps eliminate the latency in building the model again and again. In the past, data stream mining has a drawback in approximating a classification model with a sufficiently high level of accuracy. This is due to the one-pass incremental learning mechanism inherently exists in the design of the data stream mining algorithm. In order to solve this problem, a novel streamlined sensor data processing method is proposed called evolutionary expand-and-contract instance-based learning algorithm (EEAC-IBL). The multivariate data stream is first expanded into many subspaces, and then the subspaces, which are corresponding to the characteristics of the features are selected and condensed into a significant feature subset. The selection operates stochastically instead of deterministically by evolutionary optimization, which approximates the best subgroup. Followed by data stream mining, the model learning for image recognition is done on the fly. This stochastic approximation method is fast and accurate, offering an alternative to the traditional machine learning method for image recognition application in remote sensing. Our experimental results show computing advantages over other classical approaches, with a mean accuracy improvement at 16.62%.

Novel evolutionary-EAC instance-learning-based algorithm for fast data stream mining in assisted living with extreme connectivity

In modern healthcare, sensing technologies such as IoT empower the quality of assisted living service by knowing what a resident is doing in real-time. Using extreme connectivity and cloud computing in a smart home, where a collection of sensors is installed, the sensors sample continuously from the movements of the resident as well as ambient data from the surrounding inside the house. Automatic human activity recognition of the resident's activities is one of the key components of assisted living in smart home. For monitoring in-home safety, the ability in recognizing abnormal activities such as accident, falling, acute disease attack (e.g. asthma, stroke, etc.), fainting, wobbling, is particularly important. The detection and machine learning process must be both accurate and fast, to cope with the real-time activity recognition. To this end, a novel streamlined sensor data processing method is proposed called Evolutionary Expand-and-Contract Instance-based Learning algorithm (EEAC-IBL). The multivariate data stream is first expanded into many subspaces, then the subspaces which are corresponding to the characteristics of the features are selected and condensed into a significant feature subset. The selection operates scholastically instead of deterministically by evolutionary optimization which approximates the best subgroup. Followed by data stream mining, the machine learning for activity recognition is done on the fly. This approach is unique and suitable for such extreme connectivity scenario where precise feature selection is not required, and the relative importance of each feature among the sensor data changes over time. This stochastic approximation method is fast and accurate, offering an alternative to traditional machine learning method for smart home activity recognition application. Our experimental results show computing advantages over other classical approaches.

Evaluation of Instance-Based Learning and Q-Learning Algorithms in Dynamic Environments

Reinforcement learning is an unsupervised learning algorithm, where learning is based upon feedback from the environment. Prior research has proposed cognitive (e.g., Instance-based Learning or IBL) and statistical (Q-learning) reinforcement learning algorithms. However, an evaluation of these algorithms in a single dynamic environment has not been explored. In this paper, a comparison between the statistical Q-learning algorithm and the cognitive IBL algorithm is presented. A well-known environment, “Frozen Lake,” is used to train, generalize, and scale Q-learning and IBL algorithms. For generalizing, the Q-learning and IBL agents were trained on one version of the Frozen Lake and tested on a permuted version of the same environment. For scaling, the two algorithms were tested on a larger version of the Frozen Lake environment. Results revealed that the IBL algorithm used less training time and generalized better to different environment variants. The IBL algorithm was also able to show scalability by retaining its superior performance in the larger environment. These results indicate that the IBL algorithm could be proposed as an alternative to the standard reinforcement learning algorithms based on dynamic programming such as Q-learning. The inclusion of human factors (such as memory) in the IBL algorithm makes it suitable for robust learning in complex and dynamic environments.

A Machine Learning-Based Computational System Proposal Aiming at Higher Education Dropout Prediction

In the literature related to higher education, the concept of dropout has been approached from several perspectives and, over the years, its definition has been influenced by the use of diversified semantic interpretations. In a general higher education environment dropout can be broadly characterized as the act of a student engaged in a course leaving the educational institution without finishing the course. This paper describes the proposal of the architecture of a computational system, PDE (Predicting Dropout Events), based on machine learning (ML) algorithms and specifically designed for predicting dropout events in a higher level educational environment. PDE’s main subsystem implements a group of instance-based learning (IBL) algorithms which, taking into account a particular university-course environment, and based on log files containing descriptions of previous dropouts events, is capable to predict when a student already engaged in the course, is prone to dropout, so preventive measures could be quickly implemented.

A robust cyberattack detection approach using optimal features of SCADA power systems in smart grids

Smart grids are a type of complex cyber–physical system (CPS) that integrates the communication capabilities of smart devices into the grid to facilitate remote operation and control of power systems. However, this integration exposes many existing vulnerabilities of conventional supervisory control and data acquisition (SCADA) systems, resulting in severe cyber threats to the smart grid and potential violation of security objectives. Stealing sensitive information, modifying firmware, or injecting function codes through compromised devices are examples of possible attacks on the smart grid. Therefore, early detection of cyberattacks on the grid is crucial to protect it from sabotage. Machine learning (ML) methods are conventional approaches for detecting cyberattacks that use features of smart grid networks. However, developing an effective, highly accurate detection method with reduced computational overload, is still a challenging research problem. In this work, an efficient and effective security control approach is proposed to detect cyberattacks on the smart grid. The proposed approach combines both feature reduction and detection techniques to reduce the extremely large number of features and achieve an improved detection rate. A correlation-based feature selection (CFS) method is used to remove irrelevant features, improving detection efficiency. An instance-based learning (IBL) algorithm classifies normal and cyberattack events using the selected optimal features. This study describes a set of experiments conducted on public datasets from a SCADA power system based on a 10-fold cross-validation technique. Experimental results show that the proposed approach achieves a high detection rate based on a small number of features drawn from SCADA power system measurements.

Large-width machine learning algorithm

We introduce an algorithm, called Large Width (LW), that produces a multi-category classifier (defined on a distance space) with the property that the classifier has a large ‘sample width.’ (Width is a notion similar to classification margin.) LW is an incremental instance-based (also known as ‘lazy’) learning algorithm. Given a sample of labeled and unlabeled examples, it iteratively picks the next unlabeled example and classifies it while maintaining a large distance between each labeled example and its nearest-unlike prototype. (A prototype is either a labeled example or an unlabeled example which has already been classified.) Thus, LW gives a higher priority to unlabeled points whose classification decision ‘interferes’ less with the labeled sample. On a collection UCI benchmark datasets, the LW algorithm ranks at the top when compared to 11 instance-based learning algorithms (or configurations). When compared to the best candidate from instance-based learners, MLP, SVM, decision tree learner (C4.5) and Naive Bayes, LW is ranked at second place after only MLP which comes at first place by a single extra win against LW. The LW algorithm can be implemented in parallel distributed processing to yield a high speedup factor and is suitable for any distance space, with a distance function which need not necessarily satisfy the conditions of a metric.

A Novel Forecastive Anomaly Based Botnet Revelation Framework for Competing Concerns in Internet of Things

With internet, billions and millions of devices in Internet of Things (IoT) are interconnected and are communicated with other devices through messaging bots. The messaging bots are sometimes controlled by the attackers so as to carry out several malicious activities. Thus bots become a serious cyber security hazard for the IoT devices. For this reason, it is crucial to detect the existence of malicious bots and other anomalies in the network. Thus to tackle with these bots and anomalies a Novel Forecastive Anomaly based Botnet Revelation Framework is designed in our proposed work. The approach works as a two way progression, i.e. first is the Instance Creation and the second is Cataloging. As an alternative to machine learning algorithm, in our work, an Ensemble based Stream Mining is being used to generate several instances with less memory and time. Once the instances are created, Graph Structure Based Detection of Anomaly (GSBDA) is initiated based on features derived by the stream mining algorithm to detect the presence of hazardous anomalies. In addition, the second phase utilizes a KNN (K Nearest neighbor) algorithm, a type of instance based learning algorithm. It is used to identify the Botnet accurately by observing the network flows.

A learnheuristic approach for the team orienteering problem with aerial drone motion constraints

This work proposes a learnheuristic approach (combination of heuristics with machine learning) to solve an aerial-drone team orienteering problem. The goal is to maximise the total reward collected from information gathering or surveillance observations of a set of known targets within a fixed amount of time. The aerial drone team orienteering problem has the complicating feature that the travel times between targets depend on a drone’s flight path between previous targets. This path-dependence is caused by the aerial surveillance drones flying under the influence of air-resistance, gravity, and the laws of motion. Sharp turns slow drones down and the angle of ascent and air-resistance influence the acceleration a drone is capable of. The route dependence of inter-target travel times motivates the consideration of a learnheuristic approach, in which the prediction of travel times is outsourced to a machine learning algorithm. This work proposes an instance-based learning algorithm with interpolated predictions as the learning module. We show that a learnheuristic approach can lead to higher quality solutions in a shorter amount of time than those generated from an equivalent metaheuristic algorithm, an effect attributed to the search-diversity enhancing consequence of the online learning process.