ProcmonML: Generating evasion resilient host-based behavioral analytics from tree ensembles

Abstract

Host-based analytics are useful for identifying nefarious activity and limiting the impact of an adversary's cyber attack on an endpoint. The majority of open-source host-based analytics are heuristic in nature and often rely on matching combinations of strings to produce an alert. Recent threat reports demonstrate that threat actors are able to easily evade these types of analytics via variances in attack techniques, implementation differences, or simple string/parameter modifications. This work introduces a novel machine learning-based approach (procmonML) to generate true behavioral host-based analytics that are more resilient to adversary evasion, thus imparting more workload on the adversary to successfully evade detection. This is accomplished by consolidating multiple system events into a single process event. Analytics are generated from a tree ensemble model using labeled host data from a lab environment and are validated on production enterprise endpoints. This approach can detect multiple variations of a single attack technique by capturing and generalizing system behaviors. The results demonstrate that the procmonML approach is able to effectively generate host-based analytics that are applicable to new environments and more resilient to adversary evasion.