Abstract
Ensemble methods based on k-NN models minimise the effect of outliers in a training dataset by searching groups of the k closest data points to estimate the response of an unseen observation. However, traditional k-NN based ensemble methods use the arithmetic mean of the training points' responses for estimation which has several weaknesses. Traditional k-NN based models are also adversely affected by the presence of non-informative features in the data. This paper suggests a novel ensemble procedure consisting of a class of base k-NN models each constructed on a bootstrap sample drawn from the training dataset with a random subset of features. In the k nearest neighbours determined by each k-NN model, stepwise regression is fitted to predict the test point. The final estimate of the target observation is then obtained by averaging the estimates from all the models in the ensemble. The proposed method is compared with some other state-of-the-art procedures on 16 benchmark datasets in terms of coefficient of determination (R 2 ), Pearson's product-moment correlation coefficient (r), mean square predicted error (MSPE), root mean squared error (RMSE) and mean absolute error (MAE) as performance metrics. Furthermore, boxplots of the results are also constructed. The suggested ensemble procedure has outperformed the other procedures on almost all the datasets. The efficacy of the method has also been verified by assessing the proposed method in comparison with the other methods by adding non-informative features to the datasets considered. The results reveal that the proposed method is more robust to the issue of non-informative features in the data as compared to the rest of the methods.
Highlights
Supervised learning is a machine learning task dealing with functions that map an input to an output based on samples in pairs of input and outputs. k-nearest neighbours (k-NN) algorithm is considered as one of the top ten supervised learning methods used for classification and regression [1]–[3]
The proposed method is compared with k-NN, random k-NN, random forest and support vector machine on 16 datasets using R2, Pearson’s product moment correlation coefficient r, predicted mean square error root mean squared error (RMSE), root mean square error RMSE and mean absolute error MAE as performance metrics
K-NN and random k-NN have consistently performed poor as compared to the proposed optimal k-NN ensemble (Ok-NN-E) method due to their sensitivity to non-informative features in the data
Summary
Supervised learning is a machine learning task dealing with functions that map an input to an output based on samples in pairs of input and outputs. k-nearest neighbours (k-NN) algorithm is considered as one of the top ten supervised learning methods used for classification and regression [1]–[3]. K-nearest neighbours (k-NN) algorithm is considered as one of the top ten supervised learning methods used for classification and regression [1]–[3] It uses a set of k-nearest observations to decide on the response value of a test case trying to minimize the effect of outliers in a training dataset. This algorithm is fast, simple and easy to. Randomization techniques usually involves taking random samples from the training data and/or the given feature set for building the base k-NN models. This increases diversity in the base models reducing their chances
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
