Conditional functional dependency discovery and data repair based on decision tree

Abstract

In the era of big data, the data sources are complex, the data qualities are problematic, and some data is not accurate, missing or involves error. Incorrect data has seriously affected the quality of data mining, resulting in a significant impact on decision making. There are many repair methods about the missing data. In them conditional functional dependency is an effective one and many research findings on how to find conditional functional dependencies have been found. This paper presents a method of constructing conditional functional dependencies based on decision tree association rules and it proves first that decision tree is equivalent to association rules, and then it constructs conditional functional dependencies by association rules. The association rules based on decision tree are gotten by data mining and they have some hidden features and can not be found by usual ways. Thus they have a certain value of application. The paper gives a construction method and experiments prove its effectiveness.