Handling imbalance dataset issue in insider threat detection using machine learning methods

Abstract

Insider threats, characterized by their baleful impact and substantial costs, arise from internal factors within organizations. These threats are rare and usually unnoticed, as the malicious actions are often submerged in numerous normal activities, causing dataset imbalance and making detection hard. To address these challenges, in this paper we propose a Two-Step Insider Threat Detection (TSITD) approach. First, it preprocesses the CERT r4.2 and r5.2 datasets into day-long sequences. Second, it handles the dataset imbalance and detects threats by forming various combinations of sampling techniques and classifiers, referred to as TSITD models. When we compare these TSITD models to baseline models, we observe a significant improvement in anomaly detection rate and balanced accuracy. The TSITD models also achieve higher rankings when evaluated using the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method.