Mining of classification patterns in clinical data through data mining algorithms

Abstract

Data mining on clinical data is a challenging area in the field of medical research, aiming at predicting and discovering patterns of disease occurrence and prognosis based on detected symptoms and reported health conditions. Data mining is the process of recovering related, significant and imperative information from a copious collection of comprehensive data. The main focus of this paper is to highlight the significance of machine learning and data mining techniques on classifying clinical datasets downloaded from the University of California, Irvine (UCI) Machine Learning Repository viz, Mammography masses, Orthopaedic (Vertebral Column) ailments, Dermatology infection, SPECTF (Single Proton Emission Computed Tomography) Heart and Thyroid diseases. We have made a careful selection of clinical data from various domains in order to identify the performance of the data mining algorithms on different types of clinical datasets. Our research work incorporates the design of a data mining framework that generates an efficient classifier that is trained on all the clinical datasets stated, by learning patterns and rules framed by executing classification algorithms. This enables the formulation of precise and accurate decisions to classify an unseen medical test data in the related field. Our results validate and confirm that Quinlan's C4.5 classification algorithm and the Random Tree algorithm yield 100 percent classification accuracy on SPECTF Heart, Orthopaedic (Vertebral Column) ailments, Thyroid and Dermatology infection datasets while Binary Logistic Regression and CS-MC4 also give 100 percent classifier accuracy on the SPECTF Heart Dataset and Multinomial Logistic Regression too classifies the Dermatology dataset with 100 percent accuracy. However on the Mammography training dataset, the classification accuracy produced by Random Tree and Quinlan's C4.5 is only 91.36%. We modify the parameters that control the decision tree size and the confidence level to achieve 100 percent classifier accuracy. The Decision tree generated by the Quinlan's C4.5 algorithm is smaller than the decision tree given by the Random Tree classification technique. Our research states that the Quinlan's C4.5 algorithm is most efficient in building a classifier trained on clinical datasets from diverse domains that could provide 100 percent classifier accuracy on a previously unseen test data.