Methodological approach for measuring the effects of organisational-level interventions on employee withdrawal behaviour

Abstract

BackgroundTheoretical frameworks have recommended organisational-level interventions to decrease employee withdrawal behaviours such as sickness absence and employee turnover. However, evaluation of such interventions has produced inconclusive results. The aim of this study was to investigate if mixed-effects models in combination with time series analysis, process evaluation, and reference group comparisons could be used for evaluating the effects of an organisational-level intervention on employee withdrawal behaviour.MethodsMonthly data on employee withdrawal behaviours (sickness absence, employee turnover, employment rate, and unpaid leave) were collected for 58 consecutive months (before and after the intervention) for intervention and reference groups. In total, eight intervention groups with a total of 1600 employees participated in the intervention. Process evaluation data were collected by process facilitators from the intervention team. Overall intervention effects were assessed using mixed-effects models with an AR (1) covariance structure for the repeated measurements and time as fixed effect. Intervention effects for each intervention group were assessed using time series analysis. Finally, results were compared descriptively with data from process evaluation and reference groups to disentangle the organisational-level intervention effects from other simultaneous effects.ResultsAll measures of employee withdrawal behaviour indicated statistically significant time trends and seasonal variability. Applying these methods to an organisational-level intervention resulted in an overall decrease in employee withdrawal behaviour. Meanwhile, the intervention effects varied greatly between intervention groups, highlighting the need to perform analyses at multiple levels to obtain a full understanding. Results also indicated that possible delayed intervention effects must be considered and that data from process evaluation and reference group comparisons were vital for disentangling the intervention effects from other simultaneous effects.ConclusionsWhen analysing the effects of an intervention, time trends, seasonal variability, and other changes in the work environment must be considered. The use of mixed-effects models in combination with time series analysis, process evaluation, and reference groups is a promising way to improve the evaluation of organisational-level interventions that can easily be adopted by others.