Scalable Multi-Criteria Decision-Making: A MapReduce deployed Big Data Approach for Skill Analytics

Abstract

The main question in today's rapidly changing world is how fast and what sort of corresponding knowledge should an agent be adopted to?! This can be defined as knowledge mapping problem for decision based on large scale datasets with veracity and accuracy as key criteria, especially in safety-critical systems. The following paper proposes a hybrid ans scalable approach for Multi-Criteria Decision Making (MCDM) problems that is deployed in MapReduce. The main sector specific problem that is solved is to recommend training resources that efficiently improves skill gaps of job seekers. The main innovations of this work are: (1) the use of large scale semi-real skill analytics and training resources dataset (Dataset Perspective), (2) a hybrid MCDM approach that resolves skill gaps by matching required skills to the training resources (Decision Support Perspective). This can be applied to any other sector with the context of matching problems. (3) the use of MapReduce as scalable processing approach to deliver lower processing latency and higher quality for large scale datasets (Big Data and Scalability Perspective). The experimental results showed 89% accuracy in the clustering and matching results. The recommendation results have been tested and verified with the industrial partner.