Getting a clearer picture: a business application of data mining

Abstract

Businesses are beginning to realise that knowledge discovery, and data mining in particular, are becoming an essential part of day-to-day operations. Yet, fundamentally technical and statistical techniques are not usually part of the expertise of the user in a business environment. Techniques that do not rely so heavily on such specialist knowledge could prove more accessible, and thus, more rewarding. Furthermore, the problem often lies not only with the technology and tools, but with the data mining process itself. This paper explores the experience gained from a cooperative data mining effort between the authors and a health insurance company, in order to suggest ways of overcoming these challenges. Firstly, technique complexity is addressed through the use of visual link analysis. Secondly, the CRISP-DM methodology is used as a framework within which the problems experienced by this business and possible ways in which they can be overcome are described. Visual link analysis is fundamentally based on using visual perception in order to discover results and proved relatively easy to use and understand. Whilst it was found that experts can initially provide valuable assistance in helping the business users come to terms with the data mining process, it was also found that the users’ business knowledge is essential to the process. Assistance proved especially useful in the analytical understanding and preparation of data, as well as the modeling. Automation of more technical tasks was required. In spite of this assistance, a number of further challenges in the process of this company institutionalising data mining are highlighted.