Abstract

Data pre-processing is one of the crucial phases of data mining that enhances the efficiency of data mining techniques. One of the most important operations performed on data pre-processing is missing values imputation in incomplete datasets. This research presents a new imputation technique using K-means and samples weighting mechanism based on Grey relation (KWGI). The Grey-based K-means algorithm applicable to all samples of incomplete datasets clusters the similar samples, then an appropriate kernel function generates appropriate weights based on the Grey relation. The missing values estimation of the incomplete samples is done based on the weighted mean to reduce the impact of outlier and vague samples. In both clustering and imputation steps, a penalty mechanism has been considered to reduce the similarity of ambiguous samples with a high number of missing values, and consequently, increase the accuracy of clustering and imputation. The KWGI method has been applied on nine natural datasets with eight state-of-the-art and commonly used methods, namely CMIWD, KNNI, HotDeck, MeanI, KmeanI, RKmeanI, ICKmeanI, and FKMI. The imputation results are evaluated by the Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) criteria. In this study, the missing values are generated at two levels, namely sample and value, and the results are discussed in a wide range of missingness from low rate to high rate. Experimental results of the t-test show that the proposed method performs significantly better than all the other compared methods.

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.