FlexDM: Simple, parallel and fault-tolerant data mining using WEKA.

Madison Flannery,Alexandre Mendes,David M Budden

doi:10.1186/s13029-015-0045-3

Abstract

BackgroundWith the continued exponential growth in data volume, large-scale data mining and machine learning experiments have become a necessity for many researchers without programming or statistics backgrounds. WEKA (Waikato Environment for Knowledge Analysis) is a gold standard framework that facilitates and simplifies this task by allowing specification of algorithms, hyper-parameters and test strategies from a streamlined Experimenter GUI. Despite its popularity, the WEKA Experimenter exhibits several limitations that we address in our new FlexDM software.ResultsFlexDM addresses four fundamental limitations with the WEKA Experimenter: reliance on a verbose and difficult-to-modify XML schema; inability to meta-optimise experiments over a large number of algorithm hyper-parameters; inability to recover from software or hardware failure during a large experiment; and failing to leverage modern multicore processor architectures. Direct comparisons between the FlexDM and default WEKA XML schemas demonstrate a 10-fold improvement in brevity for a specification that allows finer control of experimental procedures. The stability of FlexDM has been tested on a large biological dataset (approximately 450 k attributes by 150 samples), and automatic parallelisation of tasks yields a quasi-linear reduction in execution time when distributed across multiple processor cores.ConclusionFlexDM is a powerful and easy-to-use extension to the WEKA package, which better handles the increased volume and complexity of data that has emerged during the 20 years since WEKA’s original development. FlexDM has been tested on Windows, OSX and Linux operating systems and is provided as a pre-configured virtual reference environment for trivial usage and extensibility. This software can substantially improve the productivity of any research group conducting large-scale data mining or machine learning tasks, in addition to providing non-programmers with improved control over specific aspects of their data analysis pipeline via a succinct and simplified XML schema.Electronic supplementary materialThe online version of this article (doi:10.1186/s13029-015-0045-3) contains supplementary material, which is available to authorized users.

Highlights

With the continued exponential growth in data volume, large-scale data mining and machine learning experiments have become a necessity for many researchers without programming or statistics backgrounds
Despite its proven success and widespread application, the WEKA Experimenter pipeline exhibits a number of limitations that make it both a) difficult to apply to non-trivial data mining challenges in modern research, and b) remain robust and reliable against the exponential growth of data volume in the two decades since its original development [6]
This results section is separated into two parts: a practical example of the improved FlexDM XML schema when compared to its WEKA Experimenter equivalent; and an empirical analysis of the time taken to perform a large data mining task when distributed across multiple CPUs

Summary

Results

This results section is separated into two parts: a practical example of the improved FlexDM XML schema when compared to its WEKA Experimenter equivalent; and an empirical analysis of the time taken to perform a large data mining task when distributed across multiple CPUs. FlexDM creates a summary file reporting the overall experimental outcomes This easy-to-read XML specification takes only 11 lines to define a non-trivial series of experiments and hyperparameter meta-optimisation tasks. The equivalent WEKA Experimenter-interpreted XML specification requires 10-fold as many lines and is difficult to modify without reliance upon the GUI These XML specifications are compared in Additional file 1. This test was completed on a desktop PC with a quad-core Intel i7 processor.

Conclusion

Background

Conclusions