Employee turnover reduction by analyzing textual data with SPARK

Abstract

The main purpose of this paper is to discuss the opportunities that big data analytics offer to HR practitioners. Big Data transforms organizations and affects various aspects of organizational activities, including the HR function. Traditional recruitment strategy is experiencing transformation due to social big data analytics, facilitating both, job providers and job seekers. The high turnover is a major expense for any organization because it requires additional time and budget to recruit, hire and train new employees. With big data analysis and suitable technological tools, HR experts could more quickly find the right candidates for a job profile taking into consideration various aspects. The conducted research is focused on analyzing online resumes, including unstructured data in textual format, to define the profile of employees that stay longer with the company. The methodology used is based on the CRISP-DM approach, adapted to the specificities of textual data processing. The experiments are conducted by using SPARK - analytical tool for big data processing. The k-means algorithm and bigrams generation method are used for cluster modelling. The experimental results reveal two clusters, defining specific characteristics of loyal employees. The extracted knowledge is relevant and could be further used for more effective selection in future recruitment to decrease the employee turnover rate.