Prediction of the Probability of Job Loss due to Digitalization and Comparison by Industry: Using Machine Learning Methods

Abstract

Purpose – The essential purpose of this study is to analyze the possibility of substitution of an individual job resulting from technological development represented by the 4th Industrial Resolution, considering the different effects of digital transformation on the labor market. Design/methodology – In order to estimate the substitution probability, this study used two data sets which the job characteristics data for individual occupations provided by KEIS and the information on occupational status of substitution provided by Frey and Osborne(2013). In total, 665 occupations were considered in this study. Of these, 80 occupations had data with labels of substitution status. The primary goal of estimation was to predict the degree of substitution for 607 of 665 occupations (excluding 58 with markers). It utilized three methods a principal component analysis, an unsupervised learning methodology of machine learning, and Ridge and Lasso from supervised learning methodology. After extracting significant variables based on the three methods, this study carried out logistics regression to estimate the probability of substitution for each occupation. Findings – The probability of substitution for other occupational groups did not significantly vary across individual models, and the rank order of the probabilities across occupational groups were similar across models. The mean of three methods of substitution probability was analyzed to be 45.3%. The highest value was obtained using the PCA method, and the lowest value was derived from the LASSO method. The average substitution probability of the trading industry was 45.1%, very similar to the overall average. Originality/value – This study has a significance in that it estimates the job substitution probability using various machine learning methods. The results of substitution probability estimation were compared by industry sector. In addition, This study attempts to compare between trade business and industry sector.